A new project, GPT-4V-Act, combines machine learning and visual grounding strategy to analyze user interface screenshots and provide exact pixel coordinates for task completion. The AI agent can post on Reddit, conduct product searches, and initiate checkout processes. It also identifies and corrects auto-labeler errors. The technology aims to improve UI usability, automate workflows, and enable automated UI testing. However, a current ChatGPT Plus subscription is required for multimodal prompting on this project.
Read more at MarkTechPost…
