{"repo":"kyegomez/OpenMythos","free":true,"listed":false,"github":"https://github.com/kyegomez/OpenMythos","clone":"git clone https://github.com/kyegomez/OpenMythos.git","description":"A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.","language":"Python","stars":14796,"topics":["ai","attention","claude","claude-ai","claude-code","gpt-5","looped-transformers","ml","sonnet","gpt-7"],"license":"MIT","category":"machine-learning","readme_excerpt":"OpenMythos Disclaimer: OpenMythos is an independent, community-driven theoretical reconstruction based solely on publicly available research and speculation. It is not affiliated with, endorsed by, or connected to Anthropic or any of their proprietary systems. OpenMythos is an open-source, theoretical implementation of the Claude Mythos model. It implements a Recurrent-Depth Transformer (RDT) with three stages: Prelude (transformer blocks), a looped Recurrent Block (up to max loop iters ), and a final Coda . Attention is switchable between MLA and GQA, and the feed-forward uses a sparse MoE with routed and shared experts ideal for exploring compute-adaptive, depth-variable reasoning. Installation To enable Flash Attention 2 in GQAttention (requires CUDA and build tools): Usage Model Variants Pre-configured scales from 1B to 1T parameters: Variant dim Experts expert dim Loop iters Context Max output --- --- --- --- --- --- --- mythos 1b 2048 64 2048 16 4k 4k mythos 3b 3072 64 4096 16 4k 4k mythos 10b 4096 128 5632 24 8k 4k mythos 50b 6144 256 9728 32 8k 4k mythos 100b 8192 256 13568 32 1M 128k mythos 500b 12288 512 23040 48 1M 128k mythos 1t 16384 512 34560 64 1M 128k --- Training The training script for the 3B model on FineWeb-Edu is at training/3b fine web edu.py . Single GPU: Multi-GPU (auto-detects GPU count): Key design choices: Feature Detail --- --- Optimizer AdamW Dataset HuggingFaceFW/fineweb-edu ( sample-10BT by default, swap to sample-100BT or default for full run) ","default_branch":null,"files":null,"tree":[],"storefront":"/r/kyegomez","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/kyegomez/OpenMythos/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}