{"repo":"OthersideAI/self-operating-computer","free":true,"listed":false,"github":"https://github.com/OthersideAI/self-operating-computer","clone":"git clone https://github.com/OthersideAI/self-operating-computer.git","description":"A framework to enable a multimodal model to operate a computer.","language":"Python","stars":10285,"topics":["automation","openai","pyautogui"],"license":"MIT","category":"workflow-automation","readme_excerpt":"ome Self-Operating Computer Framework A framework to enable multimodal models to operate a computer. Using the same inputs and outputs as a human operator, the model views the screen and decides on a series of mouse and keyboard actions to reach an objective. Released Nov 2023, the Self-Operating Computer Framework was one of the first examples of full computer-use. Key Features - Compatibility : Designed for various multimodal models. - Integration : Currently integrated with GPT-4o, GPT-4.1, o1, Gemini Pro Vision, Claude 3, Qwen-VL and LLaVa. - Future Plans : Support for additional models. Demo https://github.com/OthersideAI/self-operating-computer/assets/42594239/9e8abc96-c76a-46fb-9b13-03678b3c67e0 Run Self-Operating Computer 1. Install the project 2. Run the project 3. Enter your OpenAI Key : If you don't have one, you can obtain an OpenAI key here. If you need you change your key at a later point, run vim .env to open the .env and replace the old key. 4. Give Terminal app the required permissions : As a last step, the Terminal app will ask for permission for \"Screen Recording\" and \"Accessibility\" in the \"Security & Privacy\" page of Mac's \"System Preferences\". Using operate Modes OpenAI models The default model for the project is gpt-4o which you can use by simply typing operate . To try running OpenAI's new o1 model, use the command below. To experiment with OpenAI's latest gpt-4.1 model, run: Multimodal Models -m Try Google's gemini-pro-vision by following the instruct","default_branch":null,"files":null,"tree":[],"storefront":"/r/OthersideAI","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/OthersideAI/self-operating-computer/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}