{"repo":"Aryan1718/md2LLM","free":true,"listed":false,"github":"https://github.com/Aryan1718/md2LLM","clone":"git clone https://github.com/Aryan1718/md2LLM.git","description":"md2LLM enables fine-tuning of open-source language models using personal Markdown files. It includes an observability and management layer for the fine-tuning process, allowing users to generate training data, manage models, and track and store each state of the fine-tuned model.","language":"Python","stars":12,"topics":["fine-tuning-llm","markdown","observability","open-source-llm"],"license":null,"category":"analytics","readme_excerpt":"md2LLM Turn your markdown notes into a small local model you can chat with. md2LLM reads an Obsidian vault or any folder of Markdown files, scores the notes, generates fine-tuning data, recommends a training path based on your hardware, and helps you run the resulting model locally with Ollama. Overview • Features • Getting Started • Run The App • Observability • Workflow • Training Options • Project Structure • Troubleshooting [!TIP] You can use the full browser workflow, or generate training data.jsonl from the CLI with python -m pipeline.main . Overview The project is split into three stages: Typical flow: 1. Select a vault, folder upload, zip archive, or an existing training data.jsonl file. 2. Choose the training goal: knowledge , style , reasoning , or chatbot . 3. Generate JSONL examples from your notes using an LLM provider. 4. Pick a base model from the recommended Hugging Face repos. 5. Fine-tune locally on Apple Silicon with MLX, on NVIDIA with Unsloth, or use the Colab fallback. 6. Export or load the trained model and chat with it locally. Observability And Model Registry md2LLM keeps a local operational registry so generated datasets, runs, models, evaluations, and chat telemetry can be inspected after a server restart. The registry is stored in: Registry APIs: - GET /api/registry/summary - GET /api/registry/datasets - GET /api/registry/runs - GET /api/registry/runs/{run id} - GET /api/registry/runs/{run id}/logs - GET /api/registry/models - GET /api/registry/mod","default_branch":null,"files":null,"tree":[],"storefront":"/r/Aryan1718","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Aryan1718/md2LLM/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}