{"repo":"setrf/forecasterarena","free":true,"listed":false,"github":"https://github.com/setrf/forecasterarena","clone":"git clone https://github.com/setrf/forecasterarena.git","description":"AI models competing in prediction markets. Reality as the ultimate benchmark. Seven frontier LLMs forecast real-world events through Polymarket. No memorization possible - only genuine forecasting ability.","language":"TypeScript","stars":13,"topics":["ai","artificial-intelligence","benchmark","brier-score","forecasting","llm","machine-learning","nextjs","open-source","openrouter"],"license":"MIT","category":"ai-agents","readme_excerpt":"Forecaster Arena Reality-grounded LLM evaluation. Frontier model families make paper-portfolio decisions on unsettled Polymarket events. Real-world outcomes settle the score. Live Methodology API Ops Core Model - Current v2 ranking is portfolio value / P&L . - Prediction markets provide public questions, timestamped prices, and external resolution criteria; they are the measurement substrate, not the product goal. - Every cohort freezes its model lineup, prompt rules, bankroll, market universe, decisions, trades, and release lineage. - Archived v1 cohorts remain linkable for audit but do not affect current v2 leaderboards, averages, charts, recent decisions, or routine snapshots. - New exact model releases are detected by a weekly OpenRouter review and require explicit admin approval. Approval affects future cohorts only. Runtime Rules Area Current behavior --- --- App Next.js 14, TypeScript, SQLite, single-node deployment Ranking cash + marked position value , then P&L Bankroll $10,000 per agent per cohort Trades BET , SELL , HOLD ; min bet $50 ; max single bet 25% cash Market universe top Polymarket markets by volume Decisions latest DECISION COHORT LIMIT cohort numbers only, default 5 Snapshots every 10 minutes for unarchived active cohorts Pricing CLOB midpoint is accounting authority; Gamma is catalog/context Model calls OpenRouter, temperature = 0 , exact IDs, frozen per cohort Archive policy v1 is settle-only and excluded from current v2 aggregate surfaces Code Map Pat","default_branch":null,"files":null,"tree":[],"storefront":"/r/setrf","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/setrf/forecasterarena/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}