{"repo":"Uchebuzz/Naijaeval","free":true,"listed":false,"github":"https://github.com/Uchebuzz/Naijaeval","clone":"git clone https://github.com/Uchebuzz/Naijaeval.git","description":"Evaluation Metrics for African Languages","language":"Python","stars":150,"topics":["llm","llms","metrics"],"license":null,"category":"ai-agents","readme_excerpt":"NaijaEval Evaluation infrastructure for AI systems that mainstream benchmarks can't assess — built for African languages, code-switching, and dialectal robustness. --- Why this exists Standard NLP benchmarks — GLUE, HELM, XTREME — were built for high-resource languages and standard dialects. When you build a system for Nigerian English, Yoruba, Igbo, Hausa, Nigerian Pidgin, or Swahili, none of those benchmarks tell you whether your system actually works. The specific gaps NaijaEval addresses: - No metric exists for code-switch robustness. A model that scores 0.85 on clean English may collapse when a user switches mid-sentence from English to Yoruba. - No standard way to measure dialectal degradation. WER on standard British English says nothing about WER on Nigerian English. - Terminology preservation is unmeasured. BLEU doesn't weight medical or legal terms differently from \"the\" — but in practice, getting \"hypertension\" wrong matters more than getting word order slightly wrong. - Hallucination in low-resource translation is invisible. When a model is undertrained on Swahili, it hallucinates. Standard metrics don't flag this. NaijaEval provides composable, task-agnostic metrics that work on real African language evaluation challenges — out of the box. --- Quickstart --- Supported tasks and benchmarks Benchmark Task Languages Dataset --- --- --- --- naija mt v1 Machine translation English → Yoruba MENYO-20k coswitch asr v1 ASR robustness Nigerian English / Pidgin Common Voice","default_branch":null,"files":null,"tree":[],"storefront":"/r/Uchebuzz","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Uchebuzz/Naijaeval/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}