{"repo":"rmurai0610/MASt3R-SLAM","free":true,"listed":false,"github":"https://github.com/rmurai0610/MASt3R-SLAM","clone":"git clone https://github.com/rmurai0610/MASt3R-SLAM.git","description":"[CVPR 2025] MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors","language":"Python","stars":3161,"topics":["computer-vision","cvpr2025","robotics","slam"],"license":null,"category":"robotics_slam","readme_excerpt":"[comment]: <> (# MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors)\n\n<p align=\"center\">\n  <h1 align=\"center\">MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors</h1>\n  <p align=\"center\">\n    <a href=\"https://rmurai.co.uk/\"><strong>Riku Murai*</strong></a>\n    ·\n    <a href=\"https://edexheim.github.io/\"><strong>Eric Dexheimer*</strong></a>\n    ·\n    <a href=\"https://www.doc.ic.ac.uk/~ajd/\"><strong>Andrew J. Davison</strong></a>\n  </p>\n  <p align=\"center\">(* Equal Contribution)</p>\n\n[comment]: <> (  <h2 align=\"center\">PAPER</h2>)\n  <h3 align=\"center\"><a href=\"https://arxiv.org/abs/2412.12392\">Paper</a> | <a href=\"https://youtu.be/wozt71NBFTQ\">Video</a> | <a href=\"https://edexheim.github.io/mast3r-slam/\">Project Page</a></h3>\n  <div align=\"center\"></div>\n\n<p align=\"center\">\n    <img src=\"./media/teaser.gif\" alt=\"teaser\" width=\"100%\">\n</p>\n<br>\n\n# Getting Started\n## Installation\n```\nconda create -n mast3r-slam python=3.11\nconda activate mast3r-slam\n```\nCheck the system's CUDA version with nvcc\n```\nnvcc --version\n```\nInstall pytorch with **matching** CUDA version following:\n```\n# CUDA 11.8\nconda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1  pytorch-cuda=11.8 -c pytorch -c nvidia\n# CUDA 12.1\nconda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.1 -c pytorch -c nvidia\n# CUDA 12.4\nconda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.4 -c pytorch -c nvidia\n```\n\nClone the repo and install the dependencies.\n```\ngit clone https://github.com/rmurai0610/MASt3R-SLAM.git --recursive\ncd MASt3R-SLAM/\n\n# if you've clone the repo without --recursive run\n# git submodule update --init --recursive\n\npip install -e thirdparty/mast3r\npip install -e thirdparty/in3d\npip install --no-build-isolation -e .\n \n\n# Optionally install torchcodec for faster mp4 loading\npip install torchcodec==0.1\n```\n\nSetup the checkpoints for MASt3R and retrieval.  The license for the checkpoints and more information on the datasets used is written [here](https://github.com/naver/mast3r/blob/mast3r_sfm/CHECKPOINTS_NOTICE).\n```\nmkdir -p checkpoints/\nwget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth -P checkpoints/\nwget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_trainingfree.pth -P checkpoints/\nwget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_codebook.pkl -P checkpoints/\n```\n\n## WSL Users\nWe have primarily tested on Ubuntu.  If you are using WSL, please checkout to the windows branch and follow the above installation.\n```\ngit checkout windows\n```\nThis disables multiprocessing which causes an issue with shared memory as discussed [here](https://github.com/rmurai0610/MASt3R-SLAM/issues/21).\n\n## Examples\n```\nbash ./scripts/download_tum.sh\npython main.py --dataset datasets/tum/rgbd_dataset_freiburg1_room/ --config config/calib.yaml\n```\n## Live Demo\nConnect a realsense camera to the PC and run\n```\npython main.py --dataset realsense --config config/base.yaml\n```\n## Running on a video\nOur system can process either MP4 videos or folders containing RGB images.\n```\npython main.py --dataset <path/to/video>.mp4 --config config/base.yaml\npython main.py --dataset <path/to/folder> --config config/base.yaml\n```\nIf the calibration parameters are known, you can specify them in intrinsics.yaml\n```\npython main.py --dataset <path/to/video>.mp4 --config config/base.yaml --calib config/intrinsics.yaml\npython main.py --dataset <path/to/folder> --config config/base.yaml --calib config/intrinsics.yaml\n```\n\n## Downloading Dataset\n### TUM-RGBD Dataset\n```\nbash ./scripts/download_tum.sh\n```\n\n### 7-Scenes Dataset\n```\nbash ./scripts/download_7_scenes.sh\n```\n\n### EuRoC Dataset\n```\nbash ./scripts/download_euroc.sh\n```\n### ETH3D SLAM Dataset\n```\nbash ./scripts/download_eth3d.sh\n```\n\n## Running Evaluations\nAll evaluation script will run our system in a single-threaded, headless mode.\nWe can run evaluations with/without calibration:\n### TUM-RGBD Dataset\n```\nbash ./scripts/eval_tum.sh \nbash ./scripts/eval_tum.sh --no-calib\n```\n\n### 7-Scenes Dataset\n```\nbash ./scripts/eval_7_scenes.sh \nbash ./scripts/eval_7_scenes.sh --no-calib\n```\n\n### EuRoC Dataset\n```\nbash ./scripts/eval_euroc.sh \nbash ./scripts/eval_euroc.sh --no-calib\n```\n### ETH3D SLAM Dataset\n```\nbash ./scripts/eval_eth3d.sh \n```\n\n## Reproducibility\nThere might be minor differences between the released version and the results in the paper after developing this multi-processing version. \nWe run all our experiments on an RTX 4090, and the performance may differ when running with a different GPU.\n\n## Acknowledgement\nWe sincerely thank the developers and contributors of the many open-source projects that our code is built upon.\n- [MASt3R](https://github.com/naver/mast3r)\n- [MASt3R-SfM](https://github.com/naver/mast3r/tree/mast3r_sfm)\n- [DROID-SLAM](https://github.com/princeton-vl/DROID-SLAM)\n- [ModernGL](https://github.com/moderngl/moderngl)\n\n# Citation\nIf you found this code/work to be useful in your own research, please considering citing the following:\n\n```bibtex\n@inproceedings{murai2024_mast3rslam,\n  title={{MASt3R-SLAM}: Real-Time Dense {SLAM} with {3D} Reconstruction Priors},\n  author={Murai, Riku and Dexheimer, Eric and Davison, Andrew J.},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  year={2025},\n}\n```\n","default_branch":"main","files":274,"tree":[".gitignore",".gitmodules","Dependencies.md","LICENSE.md","README.md","config/base.yaml","config/calib.yaml","config/eth3d.yaml","config/eval_calib.yaml","config/eval_no_calib.yaml","config/intrinsics.yaml","groundtruths/7-scenes/chess.txt","groundtruths/7-scenes/fire.txt","groundtruths/7-scenes/heads.txt","groundtruths/7-scenes/office.txt","groundtruths/7-scenes/pumpkin.txt","groundtruths/7-scenes/redkitchen.txt","groundtruths/7-scenes/stairs.txt","groundtruths/euroc/MH_01_easy.txt","groundtruths/euroc/MH_02_easy.txt","groundtruths/euroc/MH_03_medium.txt","groundtruths/euroc/MH_04_difficult.txt","groundtruths/euroc/MH_05_difficult.txt","groundtruths/euroc/V1_01_easy.txt","groundtruths/euroc/V1_02_medium.txt","groundtruths/euroc/V1_03_difficult.txt","groundtruths/euroc/V2_01_easy.txt","groundtruths/euroc/V2_02_medium.txt","groundtruths/euroc/V2_03_difficult.txt","main.py","mast3r_slam/__init__.py","mast3r_slam/backend/include/gn.h","mast3r_slam/backend/src/gn.cpp","mast3r_slam/backend/src/gn_kernels.cu","mast3r_slam/backend/src/matching_kernels.cu","mast3r_slam/config.py","mast3r_slam/dataloader.py","mast3r_slam/evaluate.py","mast3r_slam/frame.py","mast3r_slam/geometry.py","mast3r_slam/global_opt.py","mast3r_slam/image.py","mast3r_slam/lietorch_utils.py","mast3r_slam/mast3r_utils.py","mast3r_slam/matching.py","mast3r_slam/multiprocess_utils.py","mast3r_slam/nonlinear_optimizer.py","mast3r_slam/retrieval_database.py","mast3r_slam/tictoc.py","mast3r_slam/tracker.py","mast3r_slam/visualization.py","mast3r_slam/visualization_utils.py","media/teaser.gif","pyproject.toml","resources/programs/surfelmap.glsl","resources/programs/trianglemap.glsl","scripts/download_7_scenes.sh","scripts/download_eth3d.sh","scripts/download_euroc.sh","scripts/download_tum.sh","scripts/eval_7_scenes.sh","scripts/eval_eth3d.sh","scripts/eval_euroc.sh","scripts/eval_tum.sh","setup.py","thirdparty/in3d/.gitignore","thirdparty/in3d/in3d/__init__.py","thirdparty/in3d/in3d/camera.py","thirdparty/in3d/in3d/color.py","thirdparty/in3d/in3d/geometry.py","thirdparty/in3d/in3d/image.py","thirdparty/in3d/in3d/imgui.py","thirdparty/in3d/in3d/logger.py","thirdparty/in3d/in3d/pose_utils.py","thirdparty/in3d/in3d/program.py","thirdparty/in3d/in3d/viewport_window.py","thirdparty/in3d/in3d/window.py","thirdparty/in3d/main.py","thirdparty/in3d/pyproject.toml","thirdparty/in3d/resources/fonts/Cousine-Regular.ttf","thirdparty/in3d/resources/fonts/DroidSans.ttf","thirdparty/in3d/resources/fonts/Roboto-Medium.ttf","thirdparty/in3d/resources/meshes/bunny.obj","thirdparty/in3d/resources/meshes/lucy.obj","thirdparty/in3d/resources/meshes/mustard_bottle.ply","thirdparty/in3d/resources/programs/cube_simple.glsl","thirdparty/in3d/resources/programs/depth2mesh.glsl","thirdparty/in3d/resources/programs/lines.glsl","thirdparty/in3d/resources/programs/lines_debug.glsl","thirdparty/in3d/resources/programs/normals.glsl","thirdparty/in3d/resources/programs/phong.glsl","thirdparty/in3d/resources/programs/points.glsl","thirdparty/in3d/setup.py","thirdparty/in3d/test.py","thirdparty/mast3r/.gitignore","thirdparty/mast3r/.gitmodules","thirdparty/mast3r/CHECKPOINTS_NOTICE","thirdparty/mast3r/LICENSE","thirdparty/mast3r/NOTICE","thirdparty/mast3r/README.md","thirdparty/mast3r/asmk/.gitignore","thirdparty/mast3r/asmk/LICENSE","thirdparty/mast3r/asmk/README.md","thirdparty/mast3r/asmk/asmk/__init__.py","thirdparty/mast3r/asmk/asmk/asmk_method.py","thirdparty/mast3r/asmk/asmk/codebook.py","thirdparty/mast3r/asmk/asmk/functional.py","thirdparty/mast3r/asmk/asmk/index.py","thirdparty/mast3r/asmk/asmk/inverted_file.py","thirdparty/mast3r/asmk/asmk/io_helpers.py","thirdparty/mast3r/asmk/asmk/kernel.py","thirdparty/mast3r/asmk/cython/build.sh","thirdparty/mast3r/asmk/cython/hamming.c","thirdparty/mast3r/asmk/cython/hamming.pyx","thirdparty/mast3r/asmk/examples/demo_how.py","thirdparty/mast3r/asmk/examples/params/_eccv20_how.yml","thirdparty/mast3r/asmk/examples/params/eccv20_how_r18_1000.yml","thirdparty/mast3r/asmk/examples/params/eccv20_how_r50-_1000.yml","thirdparty/mast3r/asmk/examples/params/eccv20_how_r50-_2000.yml","thirdparty/mast3r/asmk/pyproject.toml","thirdparty/mast3r/asmk/setup.py","thirdparty/mast3r/asmk/test/test_hamming.py","thirdparty/mast3r/assets/NLE_tower/01D90321-69C8-439F-B0B0-E87E7634741C-83120-000041DAE419D7AE.jpg","thirdparty/mast3r/assets/NLE_tower/1AD85EF5-B651-4291-A5C0-7BDB7D966384-83120-000041DADF639E09.jpg","thirdparty/mast3r/assets/NLE_tower/2679C386-1DC0-4443-81B5-93D7EDE4AB37-83120-000041DADB2EA917.jpg","thirdparty/mast3r/assets/NLE_tower/28EDBB63-B9F9-42FB-AC86-4852A33ED71B-83120-000041DAF22407A1.jpg","thirdparty/mast3r/assets/NLE_tower/91E9B685-7A7D-42D7-B933-23A800EE4129-83120-000041DAE12C8176.jpg","thirdparty/mast3r/assets/NLE_tower/CDBBD885-54C3-4EB4-9181-226059A60EE0-83120-000041DAE0C3D612.jpg","thirdparty/mast3r/assets/NLE_tower/FF5599FD-768B-431A-AB83-BDA5FB44CB9D-83120-000041DADDE35483.jpg","thirdparty/mast3r/assets/demo.jpg","thirdparty/mast3r/assets/examples.jpg","thirdparty/mast3r/assets/mast3r.jpg","thirdparty/mast3r/assets/mast3r_archi.jpg","thirdparty/mast3r/assets/matching.jpg","thirdparty/mast3r/demo.py","thirdparty/mast3r/demo_dust3r_ga.py","thirdparty/mast3r/docker/docker-compose-cpu.yml","thirdparty/mast3r/docker/docker-compose-cuda.yml","thirdparty/mast3r/docker/files/cpu.Dockerfile","thirdparty/mast3r/docker/files/cuda.Dockerfile","thirdparty/mast3r/docker/files/entrypoint.sh","thirdparty/mast3r/docker/run.sh","thirdparty/mast3r/dust3r/.gitignore","thirdparty/mast3r/dust3r/.gitmodules","thirdparty/mast3r/dust3r/LICENSE","thirdparty/mast3r/dust3r/NOTICE","thirdparty/mast3r/dust3r/README.md","thirdparty/mast3r/dust3r/assets/demo.jpg","thirdparty/mast3r/dust3r/assets/dust3r.jpg","thirdparty/mast3r/dust3r/assets/dust3r_archi.jpg","thirdparty/mast3r/dust3r/assets/matching.jpg","thirdparty/mast3r/dust3r/assets/pipeline1.jpg","thirdparty/mast3r/dust3r/croco/LICENSE","thirdparty/mast3r/dust3r/croco/NOTICE","thirdparty/mast3r/dust3r/croco/README.MD","thirdparty/mast3r/dust3r/croco/assets/Chateau1.png","thirdparty/mast3r/dust3r/croco/assets/Chateau2.png","thirdparty/mast3r/dust3r/croco/assets/arch.jpg","thirdparty/mast3r/dust3r/croco/croco-stereo-flow-demo.ipynb","thirdparty/mast3r/dust3r/croco/demo.py","thirdparty/mast3r/dust3r/croco/interactive_demo.ipynb","thirdparty/mast3r/dust3r/croco/models/blocks.py","thirdparty/mast3r/dust3r/croco/models/criterion.py","thirdparty/mast3r/dust3r/croco/models/croco.py","thirdparty/mast3r/dust3r/croco/models/croco_downstream.py","thirdparty/mast3r/dust3r/croco/models/curope/__init__.py","thirdparty/mast3r/dust3r/croco/models/curope/curope.cpp","thirdparty/mast3r/dust3r/croco/models/curope/curope2d.py","thirdparty/mast3r/dust3r/croco/models/curope/kernels.cu","thirdparty/mast3r/dust3r/croco/models/curope/setup.py","thirdparty/mast3r/dust3r/croco/models/dpt_block.py","thirdparty/mast3r/dust3r/croco/models/head_downstream.py","thirdparty/mast3r/dust3r/croco/models/masking.py","thirdparty/mast3r/dust3r/croco/models/pos_embed.py","thirdparty/mast3r/dust3r/croco/pretrain.py","thirdparty/mast3r/dust3r/croco/stereoflow/README.MD","thirdparty/mast3r/dust3r/croco/stereoflow/augmentor.py","thirdparty/mast3r/dust3r/croco/stereoflow/criterion.py","thirdparty/mast3r/dust3r/croco/stereoflow/datasets_flow.py","thirdparty/mast3r/dust3r/croco/stereoflow/datasets_stereo.py","thirdparty/mast3r/dust3r/croco/stereoflow/download_model.sh","thirdparty/mast3r/dust3r/croco/stereoflow/engine.py","thirdparty/mast3r/dust3r/croco/stereoflow/test.py","thirdparty/mast3r/dust3r/croco/stereoflow/train.py","thirdparty/mast3r/dust3r/croco/utils/misc.py","thirdparty/mast3r/dust3r/datasets_preprocess/habitat/README.md","thirdparty/mast3r/dust3r/datasets_preprocess/habitat/find_scenes.py","thirdparty/mast3r/dust3r/datasets_preprocess/habitat/habitat_renderer/__init__.py","thirdparty/mast3r/dust3r/datasets_preprocess/habitat/habitat_renderer/habitat_sim_envmaps_renderer.py","thirdparty/mast3r/dust3r/datasets_preprocess/habitat/habitat_renderer/multiview_crop_generator.py","thirdparty/mast3r/dust3r/datasets_preprocess/habitat/habitat_renderer/projections.py","thirdparty/mast3r/dust3r/datasets_preprocess/habitat/habitat_renderer/projections_conversions.py","thirdparty/mast3r/dust3r/datasets_preprocess/habitat/preprocess_habitat.py","thirdparty/mast3r/dust3r/datasets_preprocess/path_to_root.py","thirdparty/mast3r/dust3r/datasets_preprocess/preprocess_arkitscenes.py","thirdparty/mast3r/dust3r/datasets_preprocess/preprocess_blendedMVS.py","thirdparty/mast3r/dust3r/datasets_preprocess/preprocess_co3d.py","thirdparty/mast3r/dust3r/datasets_preprocess/preprocess_megadepth.py","thirdparty/mast3r/dust3r/datasets_preprocess/preprocess_scannetpp.py","thirdparty/mast3r/dust3r/datasets_preprocess/preprocess_staticthings3d.py","thirdparty/mast3r/dust3r/datasets_preprocess/preprocess_waymo.py","thirdparty/mast3r/dust3r/datasets_preprocess/preprocess_wildrgbd.py","thirdparty/mast3r/dust3r/demo.py","thirdparty/mast3r/dust3r/docker/docker-compose-cpu.yml","thirdparty/mast3r/dust3r/docker/docker-compose-cuda.yml","thirdparty/mast3r/dust3r/docker/files/cpu.Dockerfile","thirdparty/mast3r/dust3r/docker/files/cuda.Dockerfile","thirdparty/mast3r/dust3r/docker/files/entrypoint.sh","thirdparty/mast3r/dust3r/docker/run.sh","thirdparty/mast3r/dust3r/dust3r/__init__.py","thirdparty/mast3r/dust3r/dust3r/cloud_opt/__init__.py","thirdparty/mast3r/dust3r/dust3r/cloud_opt/base_opt.py","thirdparty/mast3r/dust3r/dust3r/cloud_opt/commons.py","thirdparty/mast3r/dust3r/dust3r/cloud_opt/init_im_poses.py","thirdparty/mast3r/dust3r/dust3r/cloud_opt/modular_optimizer.py","thirdparty/mast3r/dust3r/dust3r/cloud_opt/optimizer.py","thirdparty/mast3r/dust3r/dust3r/cloud_opt/pair_viewer.py","thirdparty/mast3r/dust3r/dust3r/demo.py","thirdparty/mast3r/dust3r/dust3r/heads/__init__.py","thirdparty/mast3r/dust3r/dust3r/heads/dpt_head.py","thirdparty/mast3r/dust3r/dust3r/heads/linear_head.py","thirdparty/mast3r/dust3r/dust3r/heads/postprocess.py","thirdparty/mast3r/dust3r/dust3r/image_pairs.py","thirdparty/mast3r/dust3r/dust3r/inference.py","thirdparty/mast3r/dust3r/dust3r/losses.py","thirdparty/mast3r/dust3r/dust3r/model.py","thirdparty/mast3r/dust3r/dust3r/optim_factory.py","thirdparty/mast3r/dust3r/dust3r/patch_embed.py","thirdparty/mast3r/dust3r/dust3r/post_process.py","thirdparty/mast3r/dust3r/dust3r/training.py","thirdparty/mast3r/dust3r/dust3r/utils/__init__.py","thirdparty/mast3r/dust3r/dust3r/utils/device.py","thirdparty/mast3r/dust3r/dust3r/utils/geometry.py","thirdparty/mast3r/dust3r/dust3r/utils/image.py","thirdparty/mast3r/dust3r/dust3r/utils/misc.py","thirdparty/mast3r/dust3r/dust3r/utils/parallel.py","thirdparty/mast3r/dust3r/dust3r/utils/path_to_croco.py","thirdparty/mast3r/dust3r/dust3r/viz.py","thirdparty/mast3r/dust3r/dust3r_visloc/README.md","thirdparty/mast3r/dust3r/dust3r_visloc/__init__.py","thirdparty/mast3r/dust3r/dust3r_visloc/evaluation.py","thirdparty/mast3r/dust3r/dust3r_visloc/localization.py","thirdparty/mast3r/dust3r/requirements.txt","thirdparty/mast3r/dust3r/requirements_optional.txt","thirdparty/mast3r/dust3r/train.py","thirdparty/mast3r/dust3r/visloc.py","thirdparty/mast3r/mast3r/__init__.py","thirdparty/mast3r/mast3r/catmlp_dpt_head.py","thirdparty/mast3r/mast3r/cloud_opt/__init__.py","thirdparty/mast3r/mast3r/cloud_opt/sparse_ga.py","thirdparty/mast3r/mast3r/cloud_opt/triangulation.py","thirdparty/mast3r/mast3r/cloud_opt/tsdf_optimizer.py","thirdparty/mast3r/mast3r/cloud_opt/utils/__init__.py","thirdparty/mast3r/mast3r/cloud_opt/utils/losses.py","thirdparty/mast3r/mast3r/cloud_opt/utils/schedules.py","thirdparty/mast3r/mast3r/colmap/__init__.py","thirdparty/mast3r/mast3r/colmap/database.py","thirdparty/mast3r/mast3r/demo.py","thirdparty/mast3r/mast3r/fast_nn.py","thirdparty/mast3r/mast3r/losses.py","thirdparty/mast3r/mast3r/model.py","thirdparty/mast3r/mast3r/retrieval/graph.py","thirdparty/mast3r/mast3r/retrieval/model.py","thirdparty/mast3r/mast3r/retrieval/processor.py","thirdparty/mast3r/mast3r/utils/__init__.py","thirdparty/mast3r/mast3r/utils/coarse_to_fine.py","thirdparty/mast3r/mast3r/utils/collate.py","thirdparty/mast3r/mast3r/utils/misc.py","thirdparty/mast3r/mast3r/utils/path_to_dust3r.py","thirdparty/mast3r/pyproject.toml","thirdparty/mast3r/requirements_old.txt","thirdparty/mast3r/setup.py","thirdparty/mast3r/train.py","thirdparty/mast3r/visloc.py"],"storefront":"/r/rmurai0610","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/rmurai0610/MASt3R-SLAM/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}