{"repo":"Blackwellboy/model-serving-minefield","free":true,"listed":false,"github":"https://github.com/Blackwellboy/model-serving-minefield","clone":"git clone https://github.com/Blackwellboy/model-serving-minefield.git","description":"Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.","language":"Python","stars":85,"topics":["benchmarking","chat-template","llama-cpp","llm-serving","local-llm","model-evaluation","reasoning-models","vllm","cuda","lm-eval"],"license":null,"category":"deployment-docker-iac","readme_excerpt":"Model Serving Minefield You just lost hours to a \"model problem\" that turned out to be a serving or config bug. You are not crazy, and you are not the first: this is the registry where those bugs live, so the next person loses minutes instead of an evening. Every entry here produced a confidently wrong measurement on a real serving path: chat templates, tool parsers, reasoning fields, quantization kernel paths, container toolchains, memory allocation, eval harnesses, versioning. The common shape: the request looks correct, the response looks correct, and the number is still wrong, because something happened between the two that nobody inspected. Request-shaped checks cannot catch any of these. Each entry leads with the symptom you would actually observe, then the mechanism, the stacks and builds it bit, the check that catches it, and the fix. Each carries exactly one status from a closed vocabulary, and the status says how much weight the entry carries: reproduced here (we ran it, and you can check the result without asking us for anything), contributor-measured, conditions as reported (someone else measured it and published their conditions), reported by others (credited and linked, not independently reproduced), measured here, raw not published (we ran it, and you cannot check it), and under test . Entries that are reported, contributed or unreproduced are welcome here and labelled, never rejected; what the label protects is your ability to tell them apart at a glance. Diag","default_branch":null,"files":null,"tree":[],"storefront":"/r/Blackwellboy","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Blackwellboy/model-serving-minefield/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}