{"repo":"dmis-lab/ChroKnowledge","free":true,"listed":false,"github":"https://github.com/dmis-lab/ChroKnowledge","clone":"git clone https://github.com/dmis-lab/ChroKnowledge.git","description":"[ICLR 2025] ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains","language":"Python","stars":17,"topics":["benchmark","iclr","iclr2025","knowledge","multidomain","prompting","template","temporal"],"license":null,"category":"ai-agents","readme_excerpt":"\\[ICLR 2025\\] ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains Overview ⏰ ChroKnowledge is a research framework designed to evaluate and update the chronological knowledge of large language models (LLMs). It builds on the ChroKnowBench dataset, which enables testing LLMs' ability to handle chronologically accumulated knowledge across multiple domains, including general, biomedical, legal, commonsense, and mathematical facts. This repository also features ChroKnowPrompt , a technique for in-depth prompting to enhance temporal reasoning and improve the accuracy of LLMs over a timeline. 🏠 Homepage 📃 Paper 🤗 Datasets Updates [Mar 4, 2025] Our paper and homepage is now updated to the latest version for ICLR 2025. [Feb 19, 2025] We now update the latest version of framework and dataset. - Supported model is updated: Gemini series, Qwen 2.5 72B, mpt 7B, Pythia 7B, Nemotron3, Mistral 7B Chat and DeepSeek R1 Distll 70B (based on Llama3.3 70B). - TF template is now supported. - Biomedical ChroKnowBench is updated. - Parsing Codes for UMLS will be supported ASAP. - Minor errors are patched. [Jan 23, 2025] Our paper has been accepted to ICLR 2025! 🎉 [Nov 28, 2024] Our paper is now updated to the latest version. [Oct 18, 2024] We post a homepage for overview of our work. [Oct 11, 2024] We have released the code and data. Task Definition Our knowledge categorization is based on knowledge classification based on the Sampling-based Knowledge Categor","default_branch":null,"files":null,"tree":[],"storefront":"/r/dmis-lab","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/dmis-lab/ChroKnowledge/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}