Download
Agent S is an open-source agentic framework designed to enable autonomous computer use through an Agent-Computer Interface (ACI). Built to operate graphical user interfaces like a human, it allows AI agents to perceive screens, reason about tasks, and execute actions across macOS, Windows, and Linux systems. The latest version, Agent S3, surpasses human-level performance on the OSWorld benchmark, demonstrating state-of-the-art results in complex multi-step computer tasks. Agent S combines powerful foundation models (such as GPT-5) with grounding models like UI-TARS to translate visual inputs into precise executable actions. It supports flexible deployment via CLI, SDK, or cloud, and integrates with multiple model providers including OpenAI, Anthropic, Gemini, Azure, and Hugging Face endpoints. With optional local code execution, reflection mechanisms, and compositional planning, Agent S provides a scalable and research-driven framework for building advanced computer-use agents.
Features
- 🖥️ Human-Like GUI Interaction: Controls real computer interfaces by interpreting screenshots and executing actions autonomously.
- 🏆 State-of-the-Art Performance: Agent S3 surpasses human-level results on OSWorld and generalizes to WindowsAgentArena and AndroidWorld.
- 🤖 Multi-Model Support: Works with OpenAI, Anthropic, Gemini, Azure, Open Router, vLLM, and custom endpoints.
- 🎯 Grounding Model Integration: Uses models like UI-TARS to convert visual elements into precise coordinate-based actions.
- 🔁 Reflection & Planning Mechanisms: Includes optional reflection agents and trajectory control for improved decision-making.
- 💻 Optional Local Code Execution: Enables secure Python and Bash execution for automation, file operations, and system tasks (with safeguards).
Project Samples
