Latest posts tagged with #guiagents on Bluesky
PAL-UI boosts long‑horizon planning for vision‑based GUI agents
PAL‑UI (Planning with Active Look‑back) adds screenshot retrieval for GUI agents. PAL‑UI‑3B and PAL‑UI‑7B, trained on 8.6 K mobile navigation samples, improve success on mobile and web tasks. getnews.me/pal-ui-boosts-long-horiz... #palui #guiagents
Comprehensive Survey of GUI Agents Powered by Large Foundation Models
A survey of GUI agents with foundation models outlines four layers—perception, reasoning, planning, acting—and notes benchmarks and safety issues. DOI 10.48550/arXiv.2412.13501. Read more: getnews.me/comprehensive-survey-of-... #guiagents #ai
Retrieval‑Augmented GUI Agents Boosted by Generative Guidelines
RAG‑GUI, a plug‑in for vision‑language models, retrieves web‑tutorials to guide GUI agents and achieved 2.6%–13.3% performance gains across three benchmark tasks. Read more: getnews.me/retrieval-augmented-gui-... #raggui #visionlanguage #guiagents
ProRe: Proactive Reward System Boosts GUI Agent Evaluation
ProRe improves GUI agent reward assessment by adding targeted probing tasks; experiments show reward accuracy up to 5.3% higher and F1 scores improving by 19.4%. getnews.me/prore-proactive-reward-s... #prore #guiagents #aievaluation
VisualTrap Reveals Stealthy Backdoor Threats in GUI AI Agents
VisualTrap backdoor works after poisoning just 5% of training data and survives fine‑tuning, affecting GUI agents on mobile, web and desktop. Read more: getnews.me/visualtrap-reveals-steal... #visualtrap #backdoor #guiagents
Orcust Improves GUI Agent Performance with Stepwise-Feedback RL
Orcust, a stepwise-feedback RL framework, boosts GUI agent performance by 22.2% on the ScreenSpot benchmark and 23.9% on ScreenSpot-Pro versus Qwen2.5-VL-7B. getnews.me/orcust-improves-gui-agen... #orcust #guiagents
UIPro: New Generalist GUI Agent Enhances Interaction Across Platforms
UIPro, a GUI agent trained on 20.6 million tasks using a unified action space, outperforms prior agents on web navigation, desktop automation and mobile benchmarks. Read more: getnews.me/uipro-new-generalist-gui... #uipro #guiagents #automation
Blink-Think-Link Model Boosts AI GUI Agents in Human‑Like Interaction
The Blink‑Think‑Link (BTL) framework adds Blink Data Generation and a BTL Reward, letting its BTL‑UI agent achieve state‑of‑the‑art results on static and dynamic GUI benchmarks. getnews.me/blink-think-link-model-b... #blinkthinklink #guiagents
New research reveals AI systems perform worse on visual tasks when they "think harder." GUI agents achieve better accuracy by skipping the reasoning steps that typically help language models excel in text-based work. #AIVisionPerformance #GUIAgents
