Say hello to spacy-pytorch-transformers!
— spaCy (@spacy_io) August 2, 2019
🛸 BERT, XLNet & GPT-2 in your spaCy pipeline
🤗 Based on @HuggingFace's pytorch-transformers
🎚️ Fine-tune pretrained models on your task
📦 Model packages for English & German
🚀 Token alignment, similarity & morehttps://t.co/3ftjt6pXwi
Affordable AI
Concern: https://arxiv.org/abs/1906.02243
Good examples:
- $40 to train imagenet: https://www.fast.ai/2018/08/10/fastai-diu-imagenet
- Fastest end2end ASR: https://github.com/facebookresearch/wav2letter
- Neural architecture search in 1.5 days on a single GPU: https://arxiv.org/abs/1806.09055
Attention mechanism
Attention for interpretability:
- Learning to Explain: https://arxiv.org/abs/1802.07814
- Attention is not Explanation: https://arxiv.org/abs/1902.10186
- Is Attention Interpretable?: https://arxiv.org/abs/1906.03731
Reviews:
Why I no longer play on Kaggle
Kaggle has two strong assumptions:
1. Amount of data you have is fixed, however: https://anand.typepad.com/datawocky/2008/03/more-data-usual.html
2. Processing time is not important, however: https://www.wired.com/2012/04/netflix-prize-costs/