Dom/OmniParser
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update readme

Browse Source
This commit is contained in:
yadonglu
2024-10-09 14:58:35 -07:00
parent 6d4c768c45
commit deec3bc5b0
1 changed files with 9 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
README.md
View File

@@ -1,12 +1,20 @@
# OmniParser: Screen Parsing tool for Pure Vision Based GUI Agent # OmniParser: Screen Parsing tool for Pure Vision Based GUI Agent
![Logo](imgs/logo.png) <div style="text-align: center;">
<img src="imgs/logo.png" alt="Logo">
</div>
[![arXiv](https://img.shields.io/badge/Paper-green)](https://arxiv.org/abs/2408.00203) [![arXiv](https://img.shields.io/badge/Paper-green)](https://arxiv.org/abs/2408.00203)
[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![License](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
📢 [[Project Page](https://microsoft.github.io/OmniParser/)] [[Blog Post](https://www.microsoft.com/en-us/research/articles/omniparser-for-pure-vision-based-gui-agent/)] [[Models](https://huggingface.co/microsoft/OmniParser)]
**OmniParser** is a comprehensive method for parsing user interface screenshots into structured and easy-to-understand elements, which significantly enhances the ability of GPT-4V to generate actions that can be accurately grounded in the corresponding regions of the interface. **OmniParser** is a comprehensive method for parsing user interface screenshots into structured and easy-to-understand elements, which significantly enhances the ability of GPT-4V to generate actions that can be accurately grounded in the corresponding regions of the interface.
## News
- [Upcoming] Both Interactive Region Detection Model and Icon functional description model will be release soon.
- [2024/09] OmniParser achieves the best performance on [[Windows Agent Arena](https://microsoft.github.io/WindowsAgentArena/)]!
## Install ## Install
```python ```python
conda create -n "omni" python==3.12 conda create -n "omni" python==3.12