Bag of Tricks in Machine Learning

https://amitness.com/2020/05/data-augmentation-for-nlp/

https://yangkky.github.io/2019/07/08/distributed-pytorch-tutorial.html

Unsupervised Pre-Training of Image Features on Non-Curated Data

Preparation: CNN course by Andrew Ng

Convolutional Neural Networks | Coursera

ImageNet Leaderboard (read in reverse order)

State-of-the-art table for Image Classification on ImageNet

[D] Why is PyTorch as fast as (and sometimes faster than) TensorFlow? from MachineLearning

I will go back to work on deep learning after writing bash for 6 months. Here is my plan to pick up deep learning.

---

Foundation:

• Deep learning (Andrew Ng): https://www.coursera.org/specializations/deep-learning
• Book (Part 2): https://www.deeplearningbook.org/

CV or NLP:

• Convolutional Neural Networks for Visual Recognition (Spring 2017): https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&disable_polymer=true
• CS224N: Natural Language Processing with Deep Learning | Winter 2019: https://www.youtube.com/playlist?list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z&disable_polymer=true

PyTorch and Tensorflow:

• Fast AI (PyTorch): https://www.youtube.com/playlist?list=PLfYUBJiXbdtSIJb-Qd3pw0cqCbkGeS0xn&disable_polymer=true
• Tensorflow: https://www.coursera.org/specializations/tensorflow-in-practice

https://www.kdd.org/awards/view/2019-sigkdd-best-paper-award-winners

https://devblogs.nvidia.com/nlu-with-tensorrt-bert/

Research on training transformer language models at scale, including BERT:

https://github.com/NVIDIA/Megatron-LM