Add model card with metadata (#1)
Browse files- Add model card with metadata (b2bb224fb94313ff6db360340a2b583ebf2bb2e3)
Co-authored-by: Niels Rogge <[email protected]>
README.md
ADDED
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
pipeline_tag: image-to-3d
|
3 |
+
library_name: pytorch
|
4 |
+
license: apache-2.0
|
5 |
+
---
|
6 |
+
|
7 |
+
# FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views
|
8 |
+
|
9 |
+
[](https://zhanghe3z.github.io/FLARE/)
|
10 |
+
[](https://huggingface.co/AntResearch/FLARE)
|
11 |
+
[](https://zhanghe3z.github.io/FLARE/videos/teaser_video.mp4)
|
12 |
+
|
13 |
+
This repository contains the FLARE model, as presented in [FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views](https://hf.co/papers/2502.12138). FLARE is a feed-forward model that estimates high-quality camera poses, 3D geometry, and appearance from as few as 2-8 uncalibrated images.
|
14 |
+
|
15 |
+
Project Page: https://zhanghe3z.github.io/FLARE/
|
16 |
+
|
17 |
+
## Run a Demo (Point Cloud and Camera Pose Estimation)
|
18 |
+
|
19 |
+
To run a demo, follow these steps:
|
20 |
+
|
21 |
+
1. **Install Dependencies:** Ensure you have PyTorch and other necessary libraries installed as detailed in the [installation instructions](https://github.com/zhanghe3z/FLARE#installation).
|
22 |
+
2. **Download Checkpoint:** Download the checkpoint from [Hugging Face](https://huggingface.co/AntResearch/FLARE/blob/main/geometry_pose.pth) and place it in the `/checkpoints/geometry_pose.pth` directory.
|
23 |
+
3. **Run the Script:** Execute the following command, replacing `"Your/Data/Path"` and `"Your/Checkpoint/Path"` with the appropriate paths:
|
24 |
+
|
25 |
+
```bash
|
26 |
+
torchrun --nproc_per_node=1 run_pose_pointcloud.py \
|
27 |
+
--test_dataset "1 @ CustomDataset(split='train', ROOT='Your/Data/Path', resolution=(512,384), seed=1, num_views=8, gt_num_image=0, aug_portrait_or_landscape=False, sequential_input=False)" \
|
28 |
+
--model "AsymmetricMASt3R(pos_embed='RoPE100', patch_embed_cls='ManyAR_PatchEmbed', img_size=(512, 512), head_type='catmlp+dpt', output_mode='pts3d+desc24', depth_mode=('exp', -inf, inf), conf_mode=('exp', 1, inf), enc_embed_dim=1024, enc_depth=24, enc_num_heads=16, dec_embed_dim=768, dec_depth=12, dec_num_heads=12, two_confs=True, desc_conf_mode=('exp', 0, inf))" \
|
29 |
+
--pretrained "Your/Checkpoint/Path" \
|
30 |
+
--test_criterion "MeshOutput(sam=False)" --output_dir "log/" --amp 1 --seed 1 --num_workers 0
|
31 |
+
```
|
32 |
+
|
33 |
+
## Visualization
|
34 |
+
|
35 |
+
After running the demo, you can visualize the results using the following command:
|
36 |
+
|
37 |
+
```bash
|
38 |
+
sh ./visualizer/vis.sh
|
39 |
+
```
|
40 |
+
|
41 |
+
This will run a visualization script. Refer to the Github README for more details on visualization options.
|
42 |
+
|
43 |
+
## Citation
|
44 |
+
|
45 |
+
```bibtex
|
46 |
+
@misc{zhang2025flarefeedforwardgeometryappearance,
|
47 |
+
title={FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views},
|
48 |
+
author={Shangzhan Zhang and Jianyuan Wang and Yinghao Xu and Nan Xue and Christian Rupprecht and Xiaowei Zhou and Yujun Shen and Gordon Wetzstein},
|
49 |
+
year={2025},
|
50 |
+
eprint={2502.12138},
|
51 |
+
archivePrefix={arXiv},
|
52 |
+
primaryClass={cs.CV},
|
53 |
+
url={https://arxiv.org/abs/2502.12138},
|
54 |
+
}
|
55 |
+
```
|