{"repo":"ScalingIntelligence/KernelBench","free":true,"listed":false,"github":"https://github.com/ScalingIntelligence/KernelBench","clone":"git clone https://github.com/ScalingIntelligence/KernelBench.git","description":"KernelBench: Can LLMs Write GPU Kernels? - Benchmark + Toolkit with Torch -> CUDA (+ more DSLs)","language":"Jupyter Notebook","stars":1202,"topics":["benchmark","codegen","evaluation","gpu","tooling","rl-environment"],"license":null,"category":"dev-tools","readme_excerpt":"KernelBench: Can LLMs Write Efficient GPU Kernels? [ICML '25] A benchmark and environment for evaluating LLMs' ability to generate efficient GPU kernels Specifically we task LLM to generate correct and efficient CUDA / DSL kernels for PyTorch programs on a target GPU. arXiv blog post HuggingFace Dataset Versions The latest stable version will be on main branch. We continue to update and improve the repo. - v0.1 - See blog - v0 - Original Release The Huggingface dataset is updated to v0.1. This repo provides core functionality for KernelBench and an easy-to-use set of scripts for evaluation. It is not intended to provide complex agentic scaffolds that solve this task; we recommend cloning and modifying this repo for your experiment, or using it as a git submodule. 👋 Task Description We structure the problem for LLMs to transpile operators described in PyTorch to CUDA kernels, at whatever level of granularity they desire. We construct KernelBench to have 4 Levels of categories: - Level 1 🧱 : Single-kernel operators (100 Problems) The foundational building blocks of neural nets (Convolutions, Matrix multiplies, Layer normalization) - Level 2 🔗 : Simple fusion patterns (100 Problems) A fused kernel would be faster than separated kernels (Conv + Bias + ReLU, Matmul + Scale + Sigmoid) - Level 3 ⚛️ : Full model architectures (50 Problems) Optimize entire model architectures end-to-end (MobileNet, VGG, MiniGPT, Mamba) - Level 4 🤗 : Level Hugging Face Optimize whole model architec","default_branch":null,"files":null,"tree":[],"storefront":"/r/ScalingIntelligence","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ScalingIntelligence/KernelBench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}