August 4, 2019 - Bag of Tricks in Machine Learning

Say hello to spacy-pytorch-transformers!

🛸 BERT, XLNet & GPT-2 in your spaCy pipeline
🤗 Based on @HuggingFace's pytorch-transformers
🎚️ Fine-tune pretrained models on your task
📦 Model packages for English & German
🚀 Token alignment, similarity & morehttps://t.co/3ftjt6pXwi
— spaCy (@spacy_io) August 2, 2019

Advances in Conversational AI by FAIR

August 4, 2019August 4, 2019 by admin

https://ai.facebook.com/blog/advances-in-conversational-ai

Shujian 🍱Follow

Retweet on Twitter Shujian 🍱 Retweeted

Thang Luong @lmthang·

12 Nov

Another view of Noisy Student: semi-supervised learning is great even when labeled data is plentiful! 130M unlabeled images yields 1% gain over previous ImageNet SOTA that uses 3.5B weakly labeled examples! joint work /w @QizheXie, Ed Hovy, @quocleix https://t.co/VUmtRlK8xk https://t.co/gdqmbCCebz

Reply on Twitter 1194352183271026689 Retweet on Twitter 119435218327102668921 Like on Twitter 119435218327102668978 Twitter 1194352183271026689

Retweet on Twitter Shujian 🍱 Retweeted

Gary Marcus @GaryMarcus·

12 Nov

My first attempt at conversation w GPT-2. It evades questions, flunks basic arithmetic, and gets caught up in probabilities of blank spaces.

Conversations with bots like Eliza and GPT-2 can look decent for short periods, but fail if you insist they keep track of anything.

Reply on Twitter 1194072212841517056 Retweet on Twitter 119407221284151705641 Like on Twitter 1194072212841517056146 Twitter 1194072212841517056

Retweet on Twitter Shujian 🍱 Retweeted

Quoc Le @quocleix·

23h

Method is also super simple:

1) Train a classifier on ImageNet
2) Infer labels on a much larger unlabeled dataset
3) Train a larger classifier on the combined set
4) Iterate the process, adding noise

Reply on Twitter 1194387839779266562 Retweet on Twitter 119438783977926656229 Like on Twitter 1194387839779266562184 Twitter 1194387839779266562

Retweet on Twitter Shujian 🍱 Retweeted

Quoc Le @quocleix·

12 Nov

Want to improve accuracy and robustness of your model? Use unlabeled data!

Our new work uses self-training on unlabeled data to achieve 87.4% top-1 on ImageNet, 1% better than SOTA. Huge gains are seen on harder benchmarks (ImageNet-A, C and P).

Link: https://t.co/ZYDaef6sdp

Reply on Twitter 1194334947156193280 Retweet on Twitter 1194334947156193280401 Like on Twitter 11943349471561932801283 Twitter 1194334947156193280

Retweet on Twitter Shujian 🍱 Retweeted

Allyson Ettinger @AllysonEttinger·

12 Nov

Further evidence for what I found ("What BERT is not" https://t.co/9P62YDLD1F): these models don't handle negation, likely because LM pre-training isn't cut out for negation. (And they even continue the bird theme.) https://t.co/6kH8wQdsWG

Reply on Twitter 1194284161495896065 Retweet on Twitter 119428416149589606513 Like on Twitter 119428416149589606585 Twitter 1194284161495896065

ACL 2019 best papers

spaCy transformer

Advances in Conversational AI by FAIR

August 2019
M	T	W	T	F	S	S
« Jul				Oct »
			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31