{"repo":"mage-ai/mage-ai","free":true,"listed":false,"github":"https://github.com/mage-ai/mage-ai","clone":"git clone https://github.com/mage-ai/mage-ai.git","description":"πŸ§™ Build, run, and manage data pipelines for integrating and transforming data.","language":"Python","stars":8800,"topics":["artificial-intelligence","data","data-engineering","data-integration","data-pipelines","data-science","dbt","elt","etl","machine-learning","orchestration","pipeline","pipelines","python","reverse-etl","spark","sql","transformation"],"license":"Apache-2.0","category":"data_orchestration","readme_excerpt":"# Mage OSS\n\n### Build modern data pipelines locally β€” fast, visual, and production-ready.\n\n
\n\nMage OSS is a self-hosted development environment designed to help teams create production-grade data pipelines with confidence.\n\nIdeal for automating ETL tasks, architecting data flow, or orchestrating transformations β€” all in a fast, notebook-style interface powered by modular code.\n\nWhen it’s time to scale, [Mage Pro](https://mage.ai) β€” our core platform β€” unlocks enterprise orchestration, collaboration, and AI-powered workflows.\n\n
\n\n\"Mage\n\"Mage\n\"Mage\n\"Join\n\n
\n\n## What you can do with Mage OSS\n\n- Build pipelines locally with Python, SQL, or R in a modular notebook-style UI\n\n- Run jobs manually or on a schedule (cron supported)\n\n- Connect to databases, APIs, and cloud storage with prebuilt connectors\n\n- Debug visually with logs, live previews, and step-by-step execution\n\n- Set up quickly with Docker, pip, or conda β€” no cloud account required\n\n- Your go-to workspace for local pipeline development β€” fully in your control.\n \n\"mage\"\n\n\n

\n\n## Start local. Scale when you're ready.\n\nUse Mage OSS to build and run pipelines on your machine. When you're ready for advanced tooling, performance, and AI-assisted productivity, Mage Pro is just one click away.\n\n[**Try Mage Pro free β†’**](https://mage.ai)\n\n
\n\n### Quickstart\n\nInstall using Docker (recommended):\n\n```bash\ndocker pull mageai/mageai:latest\n```\n\nOr with pip:\n\n```bash\npip install mage-ai\n```\n\nOr with conda:\n\n```bash\nconda install -c conda-forge mage-ai\n```\n\nFull setup guide and docs: [docs.mage.ai](https://docs.mage.ai/getting-started/setup#%E2%9B%B5%EF%B8%8F-mage-oss-overview)\n\n
\n\n## Core Features\n\n| Feature | Description |\n| :- | :- |\n| Modular pipelines | Build pipelines block-by-block using Python, SQL, or R |\n| Notebook UI | Interactive editor for writing and documenting logic |\n| Data integrations | Prebuilt connectors to databases, APIs, and cloud storage |\n| Scheduling | Trigger pipelines manually or on a schedule |\n| Visual debugging | Step-by-step logs, data previews, and error handling |\n| dbt support | Build and run dbt models directly inside Mage |\n\n
\n\n## Example Use Cases\n\n- Move data from Google Sheets to Snowflake with a Python transform\n- Schedule a daily SQL pipeline to clean and aggregate product data\n- Develop dbt models in a visual notebook-style interface\n- Run simple ETL/ELT jobs locally with full transparency\n\n
\n\n## Documentation\n\nLooking for how-to guides, examples, or advanced configuration?\n\nExplore our full documentation at [docs.mage.ai](https://docs.mage.ai).\n\n\n
\n\n## Contributing\n\nWe welcome contributions of all kinds β€” bug fixes, docs, new features, or community examples.\n\nStart with our [contributing guide](https://docs.mage.ai/contributing/overview), check out open issues, or suggest improvements.\n\n
\n\n## Ready to scale? Mage Pro has you covered.\n\nMage Pro is a powered-up platform built for teams.\nIt adds everything you need for production pipelines, at scale.\n\n- Magical AI-assisted development and debugging\n- Multi-environment orchestration\n- Role-based access control\n- Real-time monitoring & alerts\n- Powerful CI/CD & version control\n- Powerful enterprise features\n- Available fully managed, hybrid, or on-premises\n\n[**Try Mage Pro free β†’**](https://mage.ai)\n","default_branch":"master","files":5938,"tree":[".flake8",".git-dev/hooks/prepare-commit-msg",".github/ISSUE_TEMPLATE/bug_report.yml",".github/ISSUE_TEMPLATE/custom.md",".github/ISSUE_TEMPLATE/feature_request.md",".github/pull_request_template.md",".github/release.yml",".github/workflows/build_and_test.yml",".github/workflows/codeql-analysis.yml",".github/workflows/export_frontend.yml",".github/workflows/lint_docker_image.yml",".github/workflows/publish_docker_image.yml",".github/workflows/publish_to_pypi.yml",".github/workflows/trivy.yml",".gitignore",".pre-commit-config.yaml","Dockerfile","LICENSE","MANIFEST.in","Makefile","README.md","README_dev.md","SECURITY.md","SUMMARY.md","aws","dev.Dockerfile","dev.requirements.txt","dev.spark.Dockerfile","docker-compose.yml","docs/_snippets/configure-git-sync.mdx","docs/_snippets/contribution-process.mdx","docs/_snippets/debugging-backend.mdx","docs/_snippets/google-auth.mdx","docs/_snippets/initialize-session.mdx","docs/_snippets/overview-shields.mdx","docs/_snippets/test-connection.mdx","docs/_snippets/whats-a-block.mdx","docs/about/changelog.mdx","docs/about/code-of-conduct.mdx","docs/about/features.mdx","docs/about/frequently-asked-questions.mdx","docs/about/releases.mdx","docs/about/roadmap.mdx","docs/about/statistics.mdx","docs/agent/authentication.mdx","docs/agent/cli-reference.mdx","docs/agent/ide-integration.mdx","docs/agent/installation.mdx","docs/agent/introduction.mdx","docs/agent/mage-agent-plugin.mdx","docs/agent/mcp-tools.mdx","docs/agent/quickstart.mdx","docs/agent/sync.mdx","docs/agent/troubleshooting.mdx","docs/ai/ai-client.mdx","docs/ai/blocks.mdx","docs/ai/custom-resources.mdx","docs/ai/ml/train-model.mdx","docs/ai/rag-pipeline.mdx","docs/ai/setup.mdx","docs/ai/sidekick/index.mdx","docs/background.png","docs/community/blog.mdx","docs/community/contact.mdx","docs/community/github.mdx","docs/community/linkedin.mdx","docs/community/slack.mdx","docs/community/twitter.mdx","docs/contributing/backend/io/adding-a-class.mdx","docs/contributing/backend/overview.mdx","docs/contributing/backend/streaming/sources-and-destinations.mdx","docs/contributing/backend/testing/overview.mdx","docs/contributing/charts/how-to-add.mdx","docs/contributing/data-integrations/adapt-existing-source.mdx","docs/contributing/data-integrations/add-new-destination.mdx","docs/contributing/data-integrations/add-new-source.mdx","docs/contributing/development-environment.mdx","docs/contributing/documentation/overview.mdx","docs/contributing/frontend/overview.mdx","docs/contributing/frontend/testing.mdx","docs/contributing/overview.mdx","docs/contributing/styling.mdx","docs/data-integrations/batch-pipelines.mdx","docs/data-integrations/configuration.mdx","docs/data-integrations/destinations/airtable.mdx","docs/data-integrations/destinations/amazon_s3.mdx","docs/data-integrations/destinations/bigquery.mdx","docs/data-integrations/destinations/clickhouse.mdx","docs/data-integrations/destinations/databricks.mdx","docs/data-integrations/destinations/delta_lake_azure.mdx","docs/data-integrations/destinations/delta_lake_s3.mdx","docs/data-integrations/destinations/doris.mdx","docs/data-integrations/destinations/duckdb.mdx","docs/data-integrations/destinations/elasticsearch.mdx","docs/data-integrations/destinations/google_cloud_storage.mdx","docs/data-integrations/destinations/iceberg_s3.mdx","docs/data-integrations/destinations/kafka.mdx","docs/data-integrations/destinations/mongodb.mdx","docs/data-integrations/destinations/mssql.mdx","docs/data-integrations/destinations/mysql.mdx","docs/data-integrations/destinations/opensearch.mdx","docs/data-integrations/destinations/oracledb.mdx","docs/data-integrations/destinations/overview.mdx","docs/data-integrations/destinations/postgresql.mdx","docs/data-integrations/destinations/redshift.mdx","docs/data-integrations/destinations/salesforce.mdx","docs/data-integrations/destinations/snowflake.mdx","docs/data-integrations/destinations/teradata.mdx","docs/data-integrations/destinations/trino.mdx","docs/data-integrations/overview.mdx","docs/data-integrations/sources/airtable.mdx","docs/data-integrations/sources/amazon_s3.mdx","docs/data-integrations/sources/amplitude.mdx","docs/data-integrations/sources/api.mdx","docs/data-integrations/sources/azure_blob_storage.mdx","docs/data-integrations/sources/bigquery.mdx","docs/data-integrations/sources/chargebee.mdx","docs/data-integrations/sources/commercetools.mdx","docs/data-integrations/sources/couchbase.mdx","docs/data-integrations/sources/custom_source.mdx","docs/data-integrations/sources/datadog.mdx","docs/data-integrations/sources/doris.mdx","docs/data-integrations/sources/dremio.mdx","docs/data-integrations/sources/dynamodb.mdx","docs/data-integrations/sources/facebook_ads.mdx","docs/data-integrations/sources/facebook_pages.mdx","docs/data-integrations/sources/freshdesk.mdx","docs/data-integrations/sources/front.mdx","docs/data-integrations/sources/github.mdx","docs/data-integrations/sources/google_ads.mdx","docs/data-integrations/sources/google_analytics.mdx","docs/data-integrations/sources/google_campaign_manager_360.mdx","docs/data-integrations/sources/google_cloud_storage.mdx","docs/data-integrations/sources/google_display_video_360.mdx","docs/data-integrations/sources/google_search_console.mdx","docs/data-integrations/sources/google_sheets.mdx","docs/data-integrations/sources/hubspot.mdx","docs/data-integrations/sources/instagram.mdx","docs/data-integrations/sources/intercom.mdx","docs/data-integrations/sources/knowi.mdx","docs/data-integrations/sources/linkedin_ads.mdx","docs/data-integrations/sources/microsoft_ads.mdx","docs/data-integrations/sources/mode.mdx","docs/data-integrations/sources/monday.mdx","docs/data-integrations/sources/mongodb.mdx","docs/data-integrations/sources/mssql.mdx","docs/data-integrations/sources/mysql.mdx","docs/data-integrations/sources/oracledb.mdx","docs/data-integrations/sources/outreach.mdx","docs/data-integrations/sources/overview.mdx","docs/data-integrations/sources/paystack.mdx","docs/data-integrations/sources/pipedrive.mdx","docs/data-integrations/sources/postgresql.mdx","docs/data-integrations/sources/postmark.mdx","docs/data-integrations/sources/powerbi.mdx","docs/data-integrations/sources/quickbooks.mdx","docs/data-integrations/sources/redshift.mdx","docs/data-integrations/sources/salesforce.mdx","docs/data-integrations/sources/salesforce_marketing_cloud.mdx","docs/data-integrations/sources/sftp.mdx","docs/data-integrations/sources/snowflake.mdx","docs/data-integrations/sources/stripe.mdx","docs/data-integrations/sources/tableau.mdx","docs/data-integrations/sources/teradata.mdx","docs/data-integrations/sources/tiktok_ads.mdx","docs/data-integrations/sources/twitter_ads.mdx","docs/data-integrations/sources/xero.mdx","docs/data-integrations/sources/youtube_analytics.mdx","docs/data-integrations/sources/zendesk.mdx","docs/design/abstractions/project-structure.mdx","docs/design/blocks.mdx","docs/design/blocks/callbacks.mdx","docs/design/blocks/conditionals.mdx","docs/design/blocks/data-exporter.mdx","docs/design/blocks/data-loader.mdx","docs/design/blocks/dynamic-blocks.mdx","docs/design/blocks/scratchpad.mdx","docs/design/blocks/sensor.mdx","docs/design/blocks/transformer-blocks.mdx","docs/design/blocks/transformer.mdx","docs/design/core-abstractions.mdx","docs/design/core-design-principles.mdx","docs/design/data-loading.mdx","docs/design/data-pipeline-management.mdx","docs/development/blocks/callbacks/templates.mdx","docs/development/blocks/conditionals/templates.mdx","docs/development/blocks/data_exporters/templates.mdx","docs/development/blocks/data_loaders/templates.mdx","docs/development/blocks/sensors/templates.mdx","docs/development/blocks/transformers/templates.mdx","docs/development/cli-commands.mdx","docs/development/databases/default.mdx","docs/development/dependencies/custom-files.mdx","docs/development/dependencies/dependency-management.mdx","docs/development/dependencies/requirements.mdx","docs/development/docker/connecting-a-database.mdx","docs/development/docker/non-root-user.mdx","docs/development/git/pro-overview.mdx","docs/development/git/setting-up-git.mdx","docs/development/io_config.mdx","docs/development/io_config_setup.mdx","docs/development/project/landing-page.mdx","docs/development/project/local-timezone.mdx","docs/development/project/metadata.mdx","docs/development/project/setup.mdx","docs/development/settings-backend.mdx","docs/development/troubleshooting.mdx","docs/development/updating-mage.mdx","docs/development/variables/block-variables.mdx","docs/development/variables/environment-variables.mdx","docs/development/variables/overview.mdx","docs/development/variables/referencing-variables.mdx","docs/development/variables/secrets.mdx","docs/docs.json","docs/docs.sh","docs/extensibility/api-reference/backfills/create-backfills.mdx","docs/extensibility/api-reference/backfills/delete-backfill.mdx","docs/extensibility/api-reference/backfills/overview.mdx","docs/extensibility/api-reference/backfills/read-backfills.mdx","docs/extensibility/api-reference/backfills/update-backfill.mdx","docs/extensibility/api-reference/blocks/create-block.mdx","docs/extensibility/api-reference/blocks/delete-block.mdx","docs/extensibility/api-reference/blocks/overview.mdx","docs/extensibility/api-reference/blocks/read-block.mdx","docs/extensibility/api-reference/blocks/update-block.mdx","docs/extensibility/api-reference/logs/overview.mdx","docs/extensibility/api-reference/logs/read-logs.mdx","docs/extensibility/api-reference/oauth-access-tokens/overview.mdx","docs/extensibility/api-reference/oauth-access-tokens/read-oauth-access-tokens.mdx","docs/extensibility/api-reference/overview.mdx","docs/extensibility/api-reference/pipeline-runs/overview.mdx","docs/extensibility/api-reference/pipeline-runs/read-pipeline-runs.mdx","docs/extensibility/api-reference/pipeline-runs/trigger-pipeline.mdx","docs/extensibility/api-reference/pipeline-schedules/create-pipeline-schedule.mdx","docs/extensibility/api-reference/pipeline-schedules/overview.mdx","docs/extensibility/api-reference/pipeline-schedules/read-pipeline-schedules.mdx","docs/extensibility/api-reference/pipeline-schedules/update-pipeline-schedule.mdx","docs/extensibility/api-reference/pipelines/create-pipeline.mdx","docs/extensibility/api-reference/pipelines/delete-pipeline.mdx","docs/extensibility/api-reference/pipelines/overview.mdx","docs/extensibility/api-reference/pipelines/read-pipeline.mdx","docs/extensibility/api-reference/pipelines/update-pipeline.mdx","docs/extensibility/api-reference/policies.mdx","docs/extensibility/api-reference/presenters.mdx","docs/extensibility/api-reference/resources.mdx","docs/extensibility/api-reference/sessions/create-session.mdx","docs/extensibility/api-reference/sessions/overview.mdx","docs/extensibility/env-config/pipeline.mdx","docs/extensibility/env-config/project.mdx","docs/extensibility/global-data-products/overview.mdx","docs/extensibility/global-hooks/guides.mdx","docs/extensibility/global-hooks/overview.mdx","docs/extensibility/pro-api-reference/deployments/overview.mdx","docs/extensibility/pro-api-reference/overview.mdx","docs/extensibility/pro-api-reference/workspaces/update-workspace.mdx","docs/favicons/c1.ico","docs/favicons/c1.svg","docs/favicons/logo/m0.svg","docs/favicons/logo/m1.svg","docs/favicons/logo/m2.svg","docs/favicons/logo/m3.svg","docs/favicons/logo/m4.svg","docs/favicons/logo/m5.svg","docs/favicons/logo/pro.svg","docs/favicons/m1.ico","docs/favicons/m1.svg","docs/favicons/mb.ico","docs/favicons/mb.svg","docs/getting-started/build-pro-pipeline.mdx","docs/getting-started/cluster-management-portal.mdx","docs/getting-started/install-dependencies.mdx","docs/getting-started/kernels.mdx","docs/getting-started/mage-pro-api.mdx","docs/getting-started/runtime-variable.mdx","docs/getting-started/setup.mdx","docs/getting-started/trigger-pipeline.mdx","docs/guides/batch-pipeline.mdx","docs/guides/best-practices.mdx","docs/guides/blocks/batch-read-write.mdx","docs/guides/blocks/custom-blocks.mdx","docs/guides/blocks/detach-blocks.mdx","docs/guides/blocks/dynamic-blocks-2.mdx","docs/guides/blocks/dynamic-blocks.mdx","docs/guides/blocks/markdown-blocks.mdx","docs/guides/blocks/polars.mdx","docs/guides/blocks/r-blocks.mdx","docs/guides/blocks/reorder-blocks.mdx","docs/guides/blocks/replicate-blocks.mdx","docs/guides/blocks/reuse-blocks.mdx","docs/guides/blocks/sql-blocks.mdx","docs/guides/blocks/transformer-blocks.mdx","docs/guides/blocks/user-defined-templates.mdx","docs/guides/community-examples.mdx","docs/guides/data-integration-pipeline.mdx","docs/guides/data-sync/cicd-workflow.mdx","docs/guides/data-sync/git-client.mdx","docs/guides/data-sync/git-integration.mdx","docs/guides/data-sync/git-sync.mdx","docs/guides/data-sync/git-terminal.mdx","docs/guides/data-sync/version-control-guide.mdx","docs/guides/data-sync/version-control.mdx","docs/guides/dbt/add-existing-dbt.mdx","docs/guides/dbt/connection-profiles.mdx","docs/guides/dbt/developing-dbt-in-mage.mdx","docs/guides/dbt/docs.mdx","docs/guides/dbt/incremental-models.mdx","docs/guides/dbt/overview.mdx","docs/guides/dbt/run-dbt-spark.mdx","docs/guides/dbt/run-selected-model.mdx","docs/guides/dbt/run-single-model.mdx","docs/guides/dbt/running-dbt-tests.mdx","docs/guides/dbt/setup-dbt.mdx","docs/guides/dbt/snapshots.mdx","docs/guides/dbt/sources.mdx","docs/guides/dbt/variable-interpolation.mdx","docs/guides/developer-ux/block-dependency-tree.mdx","docs/guides/developer-ux/code-browser.mdx","docs/guides/developer-ux/code-canvas.mdx","docs/guides/developer-ux/code-editor.mdx","docs/guides/developer-ux/file-browser.mdx","docs/guides/developer-ux/personalize.mdx","docs/guides/developer-ux/pipeline-dependencies.mdx","docs/guides/developer-ux/sidekick.mdx","docs/guides/developer-ux/visual-studio-code.mdx","docs/guides/developer-ux/workspaces.mdx","docs/guides/git/configure.mdx","docs/guides/load-api-data.mdx","docs/guides/overview.mdx","docs/guides/pipelines/importing-pipelines.mdx","docs/guides/streaming/destinations/activemq.mdx","docs/guides/streaming/destinations/amazon-s3.mdx","docs/guides/streaming/destinations/azure_data_lake.mdx","docs/guides/streaming/destinations/bigquery.mdx","docs/guides/streaming/destinations/clickhouse.mdx","docs/guides/streaming/destinations/duckdb.mdx","docs/guides/streaming/destinations/dummy.mdx","docs/guides/streaming/destinations/elasticsearch.mdx","docs/guides/streaming/destinations/google-cloud-pubsub.mdx","docs/guides/streaming/destinations/google-cloud-storage.mdx","docs/guides/streaming/destinations/influxdb.mdx","docs/guides/streaming/destinations/kafka.mdx","docs/guides/streaming/destinations/kinesis.mdx","docs/guides/streaming/destinations/mongodb.mdx","docs/guides/streaming/destinations/mssql.mdx","docs/guides/streaming/destinations/mysql.mdx","docs/guides/streaming/destinations/opensearch.mdx","docs/guides/streaming/destinations/oracledb.mdx","docs/guides/streaming/destinations/postgres.mdx","docs/guides/streaming/destinations/rabbitmq.mdx","docs/guides/streaming/destinations/redshift.mdx","docs/guides/streaming/destinations/snowflake.mdx","docs/guides/streaming/destinations/trino.mdx","docs/guides/streaming/introduction.mdx","docs/guides/streaming/sources/activemq.mdx","docs/guides/streaming/sources/amazon-sqs.mdx","docs/guides/streaming/sources/azure-event-hub.mdx","docs/guides/streaming/sources/google-cloud-pubsub.mdx","docs/guides/streaming/sources/google_cloud_pubsub_source.png","docs/guides/streaming/sources/influxdb.mdx","docs/guides/streaming/sources/kafka.mdx","docs/guides/streaming/sources/kinesis.mdx","docs/guides/streaming/sources/mongodb.mdx","docs/guides/streaming/sources/mssql-cdc.mdx","docs/guides/streaming/sources/mysql-cdc.mdx","docs/guides/streaming/sources/nats.mdx","docs/guides/streaming/sources/oracledb.mdx","docs/guides/streaming/sources/postgres-cdc.mdx","docs/guides/streaming/sources/rabbitmq.mdx","docs/guides/streaming/tutorials/magic-devcontainer.mdx","docs/guides/streaming/tutorials/streaming-pipeline-rabbitmq.mdx","docs/guides/streaming/tutorials/streaming-pipeline.mdx","docs/guides/streaming/tutorials/streaming-stateful-store.mdx","docs/guides/tips-and-tricks.mdx","docs/guides/train/complete-project.mdx","docs/guides/version-control/file-versions.mdx","docs/integrations/airbyte.mdx","docs/integrations/airflow/index.mdx","docs/integrations/airflow/integrate-mage-airflow.mdx","docs/integrations/compute/databricks.mdx","docs/integrations/compute/management.mdx","docs/integrations/compute/spark-pro.mdx","docs/integrations/compute/spark-pyspark.mdx","docs/integrations/databases/Algolia.mdx","docs/integrations/databases/Athena.mdx","docs/integrations/databases/AzureBlobStorage.mdx","docs/integrations/databases/AzureDataLakeStorage.mdx","docs/integrations/databases/BackblazeB2.mdx","docs/integrations/databases/BigQuery.mdx","docs/integrations/databases/Chroma.mdx","docs/integrations/databases/ClickHouse.mdx","docs/integrations/databases/Databricks.mdx","docs/integrations/databases/Druid.mdx","docs/integrations/databases/DuckDB.mdx","docs/integrations/databases/GoogleCloudStorage.mdx","docs/integrations/databases/GoogleSheets.mdx","docs/integrations/databases/Iceberg.mdx","docs/integrations/databases/MicrosoftFabricWarehouse.mdx","docs/integrations/databases/MicrosoftSQLServer.mdx","docs/integrations/databases/MongoDB.mdx","docs/integrations/databases/MotherDuck.mdx"],"storefront":"/r/mage-ai","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/mage-ai/mage-ai/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}