Please check your email address for a confirmation link
Send confirmation email
Big Model Warehouse
.ModelCenter implements pre-trained language models (PLMs) based on BMTrain backend. It supports
Efficient, Low-Resource, Extendable model usage and distributed training
Share on WeChat
Easy To Use
Compared to Deepspeed and Megatron, ModelCenter have better and more flexible code-packaging and easy to configure Python environments, and the training code is uniform with PyTorch style.
More Efficient Memory Utilization
Our implementation reduces the memory footprint by several times, allowing more efficient use of the GPU's computational power with a larger batch size.
Efficient Distributed Training With Low Resources
With the support of BMTrain, ModelCenter can easily extend ZeRO3's optimization to any PLMs, and we optimize communication and time scheduling for faster distributed training.
Thanks to BMTrain, ModelCenter performs amazingly compared to other popular frameworks.
In line with the usage habits of Huggingface transformers, the threshold for getting started is lower, and the training speedup can be achieved with simple replacement.
Code after Replacement
General Model License
© 2023 OpenBMB. All Rights Reserved