{"repo":"Humanity-s-Last-Code-Exam/HLCE","free":true,"listed":false,"github":"https://github.com/Humanity-s-Last-Code-Exam/HLCE","clone":"git clone https://github.com/Humanity-s-Last-Code-Exam/HLCE.git","description":"(EMNLP 2025 Findings) Source Evaluation scripts for Humanity's Last Code Exam","language":"Python","stars":96,"topics":["codegen","codellm","llm","llm-evaluation","benchmark"],"license":null,"category":"ai-agents","readme_excerpt":"Humanity's Last Code Exam: Can Advanced LLMs Conquer Human's Hardest Code Competition? Xiangyang Li · Xiaopeng Li · Kuicai Dong · Quanhu Zhang Rongju Ruan · Xinyi Dai · Yasheng Wang · Ruiming Tang 📖 Paper 🏠 Homepage&Leaderboard 🤗 Huggingface 👉 Github Code generation is a core capability of large language models (LLMs), yet mainstream benchmarks (e.g., APPs and LiveCodeBench) contain questions with medium-level difficulty and pose no challenge to advanced LLMs. To better reflect advanced reasoning and code generation ability, we introduce Humanity's Last Code Exam (HLCE), comprising 235 of the most challenging problems from the International Collegiate Programming Contest (ICPC World Finals) and the International Olympiad in Informatics (IOI) spanning 2010-2024. Why Do We Need HLCE? With the increasing capabilities of LLMs, many benchmarks have become too easy! News August 21, 2025 🎉Congratulations on our paper HLCE being accepted by EMNLP 2025 . 🛠️Dataset Usage - You can download dataset via this link: https://huggingface.co/HumanLastCodeExam 🔮Dataset Evaluation Prerequisites - Python 3.8 or higher - Git Setup Instructions 1. Clone the repository: 2. Install the package and its dependencies: - For IOI, kindly follow these instructions to obtain the definitive evaluation results. - For ICPC-World-Finals,kindly follow these instructions to obtain the definitive evaluation results. 📊 Leaderboard - If you wish to submit your model to the leaderboard, please follow the ins","default_branch":null,"files":null,"tree":[],"storefront":"/r/Humanity-s-Last-Code-Exam","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Humanity-s-Last-Code-Exam/HLCE/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}