GPU Memory Calculator for LLM Training - Calculate GPU memory requirements for training Large Language Models with support for multiple training engines including PyTorch DDP, DeepSpeed ZeRO, Megatron-LM, and FSDP.
nlp machine-learning deep-learning pytorch developer-tools gpu-computing model-training distributed-training memory-optimization transformer-models deepspeed megatron-lm tensor-parallelism large-language-models llm llm-training ai-infrastructure pytorch-fsdp gpu-memory-calculator zero-optimization
-
Updated
Jan 26, 2026 - Python