{"repo":"atfortes/LLMSymbolicReasoningBench","free":true,"listed":false,"github":"https://github.com/atfortes/LLMSymbolicReasoningBench","clone":"git clone https://github.com/atfortes/LLMSymbolicReasoningBench.git","description":"Synthetic data generation for evaluating LLM symbolic and logic reasoning","language":"Python","stars":23,"topics":["chain-of-thought","data-engineering","dataset","dataset-generation","reasoning","symbolic-reasoning","random-generation","machine-learning","natural-language-processing","large-language-models"],"license":null,"category":"machine-learning","readme_excerpt":"Simple Synthetic Benchmarks for Language Models This repository is dedicated to generating datasets for symbolic and logic reasoning tasks in LLMs. Our goal is to provide an implementation for the generation of datasets that are not public or do not have an official implementation, and to make it easy for people to create and reproduce them. We encourage contributions and feedback to make this repository a valuable resource for the community. If you have any questions or suggestions, please feel free to open an issue or submit a pull request. Make Your Pull Requests (PRs) If you want to contribute, we encourage you to make a PR to this repository according to the following guidelines. Directory Structure Each PR should include the code and markdown description in a subdirectory. An example subdirectory tree: Please exclude large data files in the PR as they take up too much space. Instead, describe the method to acquire the data in your README.md and optionally provide a small generation snippet ( example.json ). See the coin-flip and last-letter-concatenation subdirectories for an example. Task Description (README.md) Please include the following sections in your README to help its better use: + Dataset Name : Serves as the markdown title. + Authors : Your name(s), contact and/or url to your homepage (if available). + Task Description : A short paragraph to briefly introduce what the dataset and corresponding task is about. + Running Commands : Instructions for generating th","default_branch":null,"files":null,"tree":[],"storefront":"/r/atfortes","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/atfortes/LLMSymbolicReasoningBench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}