{"repo":"Yale-BIDS-Chen-Lab/BioEval","free":true,"listed":false,"github":"https://github.com/Yale-BIDS-Chen-Lab/BioEval","clone":"git clone https://github.com/Yale-BIDS-Chen-Lab/BioEval.git","description":"A self-hosted web platform for benchmarking LLMs on medical NLP tasks.","language":"TypeScript","stars":11,"topics":[],"license":"MIT","category":"self-hosted-apps","readme_excerpt":"BioEval Benchmark LLMs on medical NLP — from setup to statistical comparison, in one platform. Evaluating LLMs on medical tasks usually means writing one-off scripts, wrangling provider APIs, and losing track of results across spreadsheets. BioEval is a self-hosted web platform that handles prompt engineering, inference, postprocessing & evaluation, head-to-head comparison, and statistical analysis in one place. 📊 Built-in Benchmarks 12 curated medical datasets covering QA, NER, relation extraction, classification, and summarization — including PubMedQA, MedQA, BC5CDR, and ChemProt. 🔀 One Workspace for the Entire Workflow Configure prompts, run inference across Azure OpenAI, Google Gemini, Anthropic, and local GPU models, auto-evaluate with 10+ metrics, and compare results side by side with statistical significance tests — all without leaving the browser. 📁 Bring Your Own Benchmark Upload custom datasets and evaluate them with the same pipeline and metrics. Benchmark Dashboard Head-to-Head Comparison & Statistical Analysis --- Prerequisites Requirement Notes --- --- Docker Desktop (macOS) or Docker Engine + Compose plugin (Linux) Required for the supported platforms below NVIDIA GPU + NVIDIA Container Toolkit Required only for local NVIDIA GPU inference ( Linux only ). Cloud providers (Azure OpenAI, Google Gemini, Anthropic) work without a GPU. --- Quick Start Open http://localhost:3000 , create an account, and add an integration (Azure OpenAI / Google Gemini / Anthropic A","default_branch":null,"files":null,"tree":[],"storefront":"/r/Yale-BIDS-Chen-Lab","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Yale-BIDS-Chen-Lab/BioEval/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}