{"repo":"gersteinlab/ML-Bench","free":true,"listed":false,"github":"https://github.com/gersteinlab/ML-Bench","clone":"git clone https://github.com/gersteinlab/ML-Bench.git","description":"ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code (https://arxiv.org/abs/2311.09835)","language":"Python","stars":316,"topics":["gpt-4","llm","code-generation"],"license":"MIT","category":"ai-agents","readme_excerpt":"ML-Bench: Benchmarking Large Language Models and Agents for End-to-End Machine Learning Workflows Table of Contents - 📋 Prerequisites - 📊 Data Preparation - 🦙 ML-LLM-Bench - 📋 Prerequisites - 🌍 Environment Setup - 🛠️ Usage - 📞 API Calling - 🔧 Open Source Model Fine-tuning - 📋 Prerequisites - 🏋️ Fine-tuning - 🔍 Inference - 🤖 ML-Agent-Bench - 🌍 Environment Setup - 📝 Cite Us - 📜 License 📋 Prerequisites To clone this repository with all its submodules, use the --recurse-submodules flag: If you have already cloned the repository without the --recurse-submodules flag, you can run the following commands to fetch the submodules: Then run 📊 Data Preparation You can load the dataset using the following code: The dataset contains the following columns: - github id : The ID of the GitHub repository. - github : The URL of the GitHub repository. - repo id : The ID of the sample within each repository. - id : The unique ID of the sample in the entire dataset. - path : The path to the corresponding folder in LLM-Bench. - arguments : The arguments specified in the user requirements. - instruction : The user instructions for the task. - oracle : The oracle contents relevant to the task. - type : The expected output type based on the oracle contents. - output : The ground truth output generated based on the oracle contents. - prefix code : The code snippet for preparing the execution environment If you want to run ML-LLM-Bench, you need to do post-processing on the dataset. You","default_branch":null,"files":null,"tree":[],"storefront":"/r/gersteinlab","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/gersteinlab/ML-Bench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}