Why PyTorch Lightning?
Question
PyTorch is quite easy to use, however, we can also find researchers are using PyTorch Lightning. It must has some reason. Why?
Answer
Selling points are well summerized from the official webset
Quick summary of what PyTorch Lightning is doing from [4]
In summary, the LightningModule groups the core ingredients we need to build a deep learning system:
- The computations (init, forward).
- What happens in the training loop (training_step).
- What happens in the validation loop (validation_step).
- What happens in the testing loop (test_step).
- The optimizer(s) to use (configure_optimizers).
- The data to use (train, test, val dataloaders).
Difference between Keras and PyTorch Lightning [4]
Following is from [4].
“Keras provides a terrific high-level interface to Tensorflow. Now Keras users can try out PyTorch via a similar high-level interface called PyTorch Lightning.
However, Lightning differs from Keras in that it’s not so much a framework but more of a style-guide for PyTorch which gives users (researchers, students, production teams) ultimate flexibility to try crazy ideas, without having to learn yet another framework while automating away all the engineering details.
Lightning differs from Keras in that:
- It is not a framework but more of a style guide.
- Lightning does not hide away details of network and optimization design.
- Lightning automates away all the engineering like early stopping, multi-GPU training, multi-node training, etc…
- Lightning gives ultimate flexibility to researchers and production teams who might need to do things like negative sampling across multiple GPUs or set up their own distributed environments.”
Reference
[1]Lightning AI
[2] Why Should I Use PyTorch Lightning?
[3] Pytorch Lightning 完全攻略 — 知乎 (zhihu.com) (In Chinese)
[4] Converting From Keras To PyTorch Lightning | by William Falcon | Towards Data Science
