{"repo":"YouAreSpecialToMe/ResearchArena","free":true,"listed":false,"github":"https://github.com/YouAreSpecialToMe/ResearchArena","clone":"git clone https://github.com/YouAreSpecialToMe/ResearchArena.git","description":null,"language":"Python","stars":42,"topics":["automated-research","autoresearch","claude-code","cli-agents","codex","kimi","llm-agents","research-benchmark"],"license":"MIT","category":"ai-agents","readme_excerpt":"How Far Are We From True Auto-Research? An in-depth analysis of frontier CLI agents — Claude Code (Opus 4.6) , Codex (GPT-5.4) , Kimi Code (K2.5) — conducting end-to-end research across diverse fields and compute resources. - 117 agent-generated papers — 39 per agent (Claude Code, Codex, Kimi Code), 3 trials × 13 seeds, spanning both GPU and CPU domains - 351 code-aware peer reviews (3 CLI-agent reviewers per paper) + 117 Stanford Agentic Reviewer scores - Human analysis of every paper , its artifacts, and agentic reviews 📄 Paper: arXiv:2605.19156 ➡️ Read the full write-up for scores, per-domain breakdowns, case studies, and the human-inspection findings. What this does Given a seed field (e.g., \"computer vision\", \"compiler optimization\"), each CLI agent follows a standardized pipeline: 1. Ideation — Generate a research idea and experiment plan; self-review for up to 3 iterations. 2. Experiments — Write and execute code, collect results; self-review for up to 3 iterations. 3. Paper Writing — Produce a paper; self-review for up to 3 iterations. 4. Review — Evaluate via Stanford Agentic Reviewer and triple peer review (all three agents review each paper alongside its code). Conferences & areas Seeds span multiple CS conferences and two compute platforms. Hardware: 1× RTX A6000 (48GB), 4 CPUs, 60GB RAM (main experiments); H100 (80GB) re-run for GPU seeds. Platform Seeds Target conferences --- --- --- GPU (8) AI for Biology, Computer Vision, Datasets & Benchmarks, Generative Mod","default_branch":null,"files":null,"tree":[],"storefront":"/r/YouAreSpecialToMe","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/YouAreSpecialToMe/ResearchArena/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}