Compare commits
10 Commits
huggingface
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 064baef0e4 | |||
| 5f0df54b5c | |||
| 2c23b43ee9 | |||
| a0ed7a8f66 | |||
| f8a42bbe39 | |||
| 3e3a0dbafd | |||
| d3e3c10f48 | |||
| 0ed28c85e8 | |||
| 73cd8012eb | |||
| 3ebd70a873 |
@@ -1,5 +1,8 @@
|
||||
# SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimation
|
||||
|
||||
## Check out [SMPLest-X](https://github.com/wqyin/SMPLest-X), an extension of SMPLer-X with stronger foundation models!
|
||||
|
||||
|
||||

|
||||
|
||||
## Useful links
|
||||
@@ -9,10 +12,17 @@
|
||||
<a href="https://huggingface.co/spaces/caizhongang/SMPLer-X" class="button"><b>[HuggingFace]</b></a>
|
||||
<a href="https://arxiv.org/abs/2309.17448" class="button"><b>[arXiv]</b></a>
|
||||
<a href="https://youtu.be/DepTqbPpVzY" class="button"><b>[Video]</b></a>
|
||||
<a href="https://github.com/wqyin/SMPLest-X" class="button"><b>[SMPLest-X]</b></a>
|
||||
<a href="https://github.com/open-mmlab/mmhuman3d" class="button"><b>[MMHuman3D]</b></a>
|
||||
</div>
|
||||
|
||||
## News
|
||||
- [2025-10-21] [SMPLest-X](https://github.com/wqyin/SMPLest-X) accepted to TPAMI.
|
||||
- [2025-02-17] Pretrained models of [SMPLest-X](https://github.com/wqyin/SMPLest-X) available for download.
|
||||
- [2025-02-14] Brand new codebase of [SMPLest-X](https://github.com/wqyin/SMPLest-X) released for trainig, testing and inference.
|
||||
- [2025-01-20] [SMPLest-X](https://github.com/wqyin/SMPLest-X) released on [arXiv](https://arxiv.org/abs/2501.09782).
|
||||
- [2025-01-08] Project page of [SMPLest-X](https://github.com/wqyin/SMPLest-X) created.
|
||||
- [2024-03-29] An updated version of SMPLer-X-H32 is released to fix camera estimation on 3DPW-like data.
|
||||
- [2024-02-29] [HuggingFace](https://huggingface.co/spaces/caizhongang/SMPLer-X) demo is online!
|
||||
- [2023-10-23] Support visualization through SMPL-X mesh overlay and add inference docker.
|
||||
- [2023-10-02] [arXiv](https://arxiv.org/abs/2309.17448) preprint is online!
|
||||
@@ -56,14 +66,16 @@ docker run --gpus all -v <vid_input_folder>:/smplerx_inference/vid_input \
|
||||
|
||||
|
||||
## Pretrained Models
|
||||
| Model | Backbone | #Datasets | #Inst. | #Params | MPE | Download | FPS |
|
||||
|:------------:|:--------:|:---------:|:------:|:-------:|:----:|:--------:|:-----:|
|
||||
| SMPLer-X-S32 | ViT-S | 32 | 4.5M | 32M | 82.6 | [model](https://pjlab-my.sharepoint.cn/:u:/g/personal/openmmlab_pjlab_org_cn/EbkyKOS5PclHtDSxdZDmsu0BNviaTKUbF5QUPJ08hfKuKg?e=LQVvzs) | 36.17 |
|
||||
| SMPLer-X-B32 | ViT-B | 32 | 4.5M | 103M | 74.3 | [model](https://pjlab-my.sharepoint.cn/:u:/g/personal/openmmlab_pjlab_org_cn/EVcRBwNOQl9OtWhnCU54l58BzJaYEPxcFIw7u_GnnlPZiA?e=nPqMjz) | 33.09 |
|
||||
| SMPLer-X-L32 | ViT-L | 32 | 4.5M | 327M | 66.2 | [model](https://pjlab-my.sharepoint.cn/:u:/g/personal/openmmlab_pjlab_org_cn/EWypJXfmJ2dEhoC0pHFFd5MBoSs7LCZmWQjHjbcQJF72fQ?e=Gteus3) | 24.44 |
|
||||
| SMPLer-X-H32 | ViT-H | 32 | 4.5M | 662M | 63.0 | [model](https://pjlab-my.sharepoint.cn/:u:/g/personal/openmmlab_pjlab_org_cn/Eco7AAc_ZmtBrhAat2e5Ti8BonrR3NVNx-tNSck45ixT4Q?e=nudXrR) | 17.47 |
|
||||
| Model | Backbone | #Datasets | #Inst. | #Params | MPE | Download | FPS |
|
||||
|:-------------:|:--------:|:---------:|:------:|:-------:|:----:|:--------:|:-----:|
|
||||
| SMPLer-X-S32 | ViT-S | 32 | 4.5M | 32M | 82.6 | [model](https://huggingface.co/caizhongang/SMPLer-X/resolve/main/smpler_x_s32.pth.tar?download=true) | 36.17 |
|
||||
| SMPLer-X-B32 | ViT-B | 32 | 4.5M | 103M | 74.3 | [model](https://huggingface.co/caizhongang/SMPLer-X/resolve/main/smpler_x_b32.pth.tar?download=true) | 33.09 |
|
||||
| SMPLer-X-L32 | ViT-L | 32 | 4.5M | 327M | 66.2 | [model](https://huggingface.co/caizhongang/SMPLer-X/resolve/main/smpler_x_l32.pth.tar?download=true) | 24.44 |
|
||||
| SMPLer-X-H32 | ViT-H | 32 | 4.5M | 662M | 63.0 | [model](https://huggingface.co/caizhongang/SMPLer-X/resolve/main/smpler_x_h32.pth.tar?download=true) | 17.47 |
|
||||
| SMPLer-X-H32* | ViT-H | 32 | 4.5M | 662M | 59.7 | [model](https://huggingface.co/caizhongang/SMPLer-X/resolve/main/smpler_x_h32_correct.pth.tar?download=true) | 17.47 |
|
||||
* MPE (Mean Primary Error): the average of the primary errors on five benchmarks (AGORA, EgoBody, UBody, 3DPW, and EHF)
|
||||
* FPS (Frames Per Second): the average inference speed on a single Tesla V100 GPU, batch size = 1
|
||||
* SMPLer-X-H32* is the updated version of SMPLer-X-H32, which fixes the camera estimation issue on 3DPW-like data.
|
||||
|
||||
## Preparation
|
||||
- download all datasets
|
||||
@@ -246,3 +258,53 @@ sh slurm_test.sh {JOB_NAME} {NUM_GPU} {TRAIN_OUTPUT_DIR} {CKPT_ID}
|
||||
- [Hand4Whole](https://github.com/mks0601/Hand4Whole_RELEASE)
|
||||
- [OSX](https://github.com/IDEA-Research/OSX)
|
||||
- [MMHuman3D](https://github.com/open-mmlab/mmhuman3d)
|
||||
|
||||
## Citation
|
||||
```text
|
||||
# SMPLest-X
|
||||
@article{yin2025smplest,
|
||||
title={SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation},
|
||||
author={Yin, Wanqi and Cai, Zhongang and Wang, Ruisi and Zeng, Ailing and Wei, Chen and Sun, Qingping and Mei, Haiyi and Wang, Yanjun and Pang, Hui En and Zhang, Mingyuan and Zhang, Lei and Loy, Chen Change and Yamashita, Atsushi and Yang, Lei and Liu, Ziwei},
|
||||
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
|
||||
year={2026},
|
||||
volume={48},
|
||||
number={2},
|
||||
pages={1778-1794},
|
||||
doi={10.1109/TPAMI.2025.3618174}
|
||||
}
|
||||
|
||||
# SMPLer-X
|
||||
@inproceedings{cai2023smplerx,
|
||||
title={{SMPLer-X}: Scaling up expressive human pose and shape estimation},
|
||||
author={Cai, Zhongang and Yin, Wanqi and Zeng, Ailing and Wei, Chen and Sun, Qingping and Yanjun, Wang and Pang, Hui En and Mei, Haiyi and Zhang, Mingyuan and Zhang, Lei and Loy, Chen Change and Yang, Lei and Liu, Ziwei},
|
||||
booktitle={Advances in Neural Information Processing Systems},
|
||||
year={2023}
|
||||
}
|
||||
```
|
||||
|
||||
## Explore More [Motrix](https://github.com/MotrixLab) Projects
|
||||
|
||||
### Motion Capture
|
||||
- [SMPL-X] [TPAMI'25] [SMPLest-X](https://github.com/MotrixLab/SMPLest-X): An extended version of [SMPLer-X](https://github.com/MotrixLab/SMPLer-X) with stronger foundation models.
|
||||
- [SMPL-X] [NeurIPS'23] [SMPLer-X](https://github.com/MotrixLab/SMPLer-X): Scaling up EHPS towards a family of generalist foundation models.
|
||||
- [SMPL-X] [ECCV'24] [WHAC](https://github.com/MotrixLab/WHAC): World-grounded human pose and camera estimation from monocular videos.
|
||||
- [SMPL-X] [CVPR'24] [AiOS](https://github.com/MotrixLab/AiOS): An all-in-one-stage pipeline combining detection and 3D human reconstruction.
|
||||
- [SMPL-X] [NeurIPS'23] [RoboSMPLX](https://github.com/MotrixLab/RoboSMPLX): A framework to enhance the robustness of whole-body pose and shape estimation.
|
||||
- [SMPL-X] [ICML'25] [ADHMR](https://github.com/MotrixLab/ADHMR): A framework to align diffusion-based human mesh recovery methods via direct preference optimization.
|
||||
- [SMPL-X] [MKA](https://github.com/MotrixLab/MKA): Full-body 3D mesh reconstruction from single- or multi-view RGB videos.
|
||||
- [SMPL] [ICCV'23] [Zolly](https://github.com/MotrixLab/Zolly): 3D human mesh reconstruction from perspective-distorted images.
|
||||
- [SMPL] [IJCV'26] [PointHPS](https://github.com/MotrixLab/PointHPS): 3D HPS from point clouds captured in real-world settings.
|
||||
- [SMPL] [NeurIPS'22] [HMR-Benchmarks](https://github.com/MotrixLab/hmr-benchmarks): A comprehensive benchmark of HPS datasets, backbones, and training strategies.
|
||||
|
||||
### Motion Generation
|
||||
- [SMPL-X] [ICLR'26] [ViMoGen](https://github.com/MotrixLab/ViMoGen): A comprehensive framework that transfers knowledge from ViGen to MoGen across data, modeling, and evaluation.
|
||||
- [SMPL-X] [ECCV'24] [LMM](https://github.com/MotrixLab/LMM): Large Motion Model for Unified Multi-Modal Motion Generation.
|
||||
- [SMPL-X] [NeurIPS'23] [FineMoGen](https://github.com/MotrixLab/FineMoGen): Fine-Grained Spatio-Temporal Motion Generation and Editing.
|
||||
- [SMPL] [InfiniteDance](https://github.com/MotrixLab/InfiniteDance): A large-scale 3D dance dataset and an MLLM-based music-to-dance model designed for robust in-the-wild generalization.
|
||||
- [SMPL] [NeurIPS'23] [InsActor](https://github.com/MotrixLab/insactor): Generating physics-based human motions from language and waypoint conditions via diffusion policies.
|
||||
- [SMPL] [ICCV'23] [ReMoDiffuse](https://github.com/MotrixLab/ReMoDiffuse): Retrieval-Augmented Motion Diffusion Model.
|
||||
- [SMPL] [TPAMI'24] [MotionDiffuse](https://github.com/MotrixLab/MotionDiffuse): Text-Driven Human Motion Generation with Diffusion Model.
|
||||
|
||||
### Motion Dataset
|
||||
- [SMPL] [ECCV'22] [HuMMan](https://github.com/MotrixLab/humman_toolbox): Toolbox for HuMMan, a large-scale multi-modal 4D human dataset.
|
||||
- [SMPLX] [T-PAMI'24] [GTA-Human](https://github.com/MotrixLab/gta-human_toolbox): Toolbox for GTA-Human, a large-scale 3D human dataset generated with the GTA-V game engine.
|
||||
|
||||
Reference in New Issue
Block a user