{"repo":"ADevillers/Pupil","free":true,"listed":false,"github":"https://github.com/ADevillers/Pupil","clone":"git clone https://github.com/ADevillers/Pupil.git","description":"Let agents perceive, indicate, and act in any application.","language":"JavaScript","stars":37,"topics":["ai-agents","automation","mcp","skills","windows"],"license":"MIT","category":"mcp-servers","readme_excerpt":"Pupil Pupil is a Windows MCP server that lets an AI agent perceive your UI as structured data, indicate with an on-screen overlay, and act on the desktop when you accept. Demo where the human operator only does : Tab , Tab , Tab ... Click the GIF for the full-quality MP4. Why Pupil Today, working with an agent on a real desktop usually means a chat back-and-forth: you describe what you see, the agent describes what to do, you do it, you describe again. It works, but it's slow and a lot gets lost in translation. Two things make that loop hard: - agents can't reliably see what's on screen, so context comes from your words (or repeated screenshots sent through the model); - and for many steps they need you to act — clicking a specific button, typing into a specific field, confirming a dialog — because they don't have hands on your machine. Pupil turns that chat into something more like working side by side. The agent gets a structured view of the UI instead of guessing from screenshots, and when it needs you it draws an overlay card on the exact control to click or field to fill. You stay in charge — you can accept, skip, or ignore — and as a bonus the agent can also execute the action itself when you let it, so the same channel covers \"show me\", \"do this\", and \"let me do it for you\". It's not a full autopilot. It's a tighter loop between what the agent sees, what it asks for, and what actually happens on your screen. Examples Overlay cards for each indicate type ( info , warnin","default_branch":null,"files":null,"tree":[],"storefront":"/r/ADevillers","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ADevillers/Pupil/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}