admin

I will go back to work on deep learning after writing bash for 6 months. Here is my plan to pick up deep learning.

Foundation:

• Deep learning (Andrew Ng): https://www.coursera.org/specializations/deep-learning
• Book (Part 2): https://www.deeplearningbook.org/

CV or NLP:

• Convolutional Neural Networks for Visual Recognition (Spring 2017): https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&disable_polymer=true
• CS224N: Natural Language Processing with Deep Learning | Winter 2019: https://www.youtube.com/playlist?list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z&disable_polymer=true

PyTorch and Tensorflow:

• Fast AI (PyTorch): https://www.youtube.com/playlist?list=PLfYUBJiXbdtSIJb-Qd3pw0cqCbkGeS0xn&disable_polymer=true
• Tensorflow: https://www.coursera.org/specializations/tensorflow-in-practice

https://www.kdd.org/awards/view/2019-sigkdd-best-paper-award-winners

https://devblogs.nvidia.com/nlu-with-tensorrt-bert/

Research on training transformer language models at scale, including BERT:

https://github.com/NVIDIA/Megatron-LM

https://arxiv.org/abs/1908.04284

Personal VAD: Speaker-Conditioned Voice Activity Detection

Shaojin Ding, Quan Wang, Shuo-yiin Chang, Li Wan, Ignacio Lopez Moreno

Comments: To be submitted to ICASSP 2020Subjects: Audio and Speech Processing (eess.AS); Machine Learning (cs.LG); Machine Learning (stat.ML)

http://www.acl2019.org/EN/winners-of-acl-2019-best-paper-awards.xhtml

Concern: https://arxiv.org/abs/1906.02243

Good examples:

• $40 to train imagenet: https://www.fast.ai/2018/08/10/fastai-diu-imagenet
• Fastest end2end ASR: https://github.com/facebookresearch/wav2letter
• Neural architecture search in 1.5 days on a single GPU: https://arxiv.org/abs/1806.09055