{"repo":"EachSheep/ShortcutsBench","free":true,"listed":false,"github":"https://github.com/EachSheep/ShortcutsBench","clone":"git clone https://github.com/EachSheep/ShortcutsBench.git","description":"ShortcutsBench: A Large-Scale Real-World Benchmark for API-Based Agents","language":"Python","stars":112,"topics":["agent","api","apple","ios","llm","macos","shortcut","watchos","wwdc24"],"license":"Apache-2.0","category":"mobile-apps","readme_excerpt":"🔧ShortcutsBench📱 Read this in 中文. What are Shortcuts? Shortcuts are workflows built by developers in the Shortcuts app using a user-friendly graphical interface 🖼️ with the provided basic actions. Apple describes them as \"a quick way to get one or more tasks done with your apps.\" 📱 Project Task List (Continuously Updated) 📋 All data, data acquisition processes, data generated during cleaning, cleaning scripts, experiment scripts, results, and related files can be found in the following documents: deves dataset/dataset src/README.md (English) or Chinese, deves dataset/dataset src valid apis/README.md (English) or Chinese, and experiments/README.md (English) or Chinese. - [x] ShortcutsBench Paper Main Text - [x] ShortcutsBench Paper Appendix - [x] Scripts for Data Acquisition, Data Cleaning and Processing, Experiment Code, and Experiment Results - [x] We provide shortcuts with bilingual explanations for regular users: listed in users dataset/${website name}/${category name}/README.md (English) or users dataset/${website name}/${category name}/README ZH.md (Chinese). Regular users can find suitable shortcuts for their work or life in our repository, which they can import into the Shortcuts app on Apple devices. Each shortcut includes: 1. The iCloud link for the shortcut 2. A description of the shortcut's functionality 3. The source of the shortcut For Shortcut Researchers : ShortcutsBench provides: (1) Shortcuts (i.e., sequences of actions in golden ); (2) Queries (i.e., ta","default_branch":null,"files":null,"tree":[],"storefront":"/r/EachSheep","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/EachSheep/ShortcutsBench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}