Skip to content

Bag of Tricks in Machine Learning

Public Personal Notebook

  • Home
  • random
  • paper
  • list
  • nlp
  • reinforcementlearning
  • gan
  • cv

spaCy transformer

August 4, 2019 by admin

Say hello to spacy-pytorch-transformers!

🛸 BERT, XLNet & GPT-2 in your spaCy pipeline
🤗 Based on @HuggingFace's pytorch-transformers
🎚️ Fine-tune pretrained models on your task
📦 Model packages for English & German
🚀 Token alignment, similarity & morehttps://t.co/3ftjt6pXwi

— spaCy (@spacy_io) August 2, 2019
Categories Uncategorized
Post navigation
Advances in Conversational AI by FAIR
ACL 2019 best papers

Best practice to follow this website

  1. In Feedly, click “add content”
  2. Input the url “bagoftricks.ml” and follow

ShujianFollow

Shujian
Retweet on TwitterShujian Retweeted
luyu_gaoLuyu Gao@luyu_gao·
22 Sep

Excited to share our EMNLP 2021 paper, Condenser: a Pre-training Architecture for Dense Retrieval. Condenser unsupervisedly learns information aggregation into dense representation through language model pre-training. https://arxiv.org/pdf/2104.08253.pdf with @JamieCallan

Reply on Twitter 1440705155385606145Retweet on Twitter 144070515538560614510Like on Twitter 144070515538560614568Twitter 1440705155385606145
Retweet on TwitterShujian Retweeted
stanfordnlpStanford NLP Group@stanfordnlp·
19 Sep

The result that the multimodal models much more effectively use vision for language than language for vision is intriguing (and somewhat surprising). https://twitter.com/ebugliarello/status/1437456748240121862

Emanuele Bugliarello@ebugliarello

Is your Vision-and-Language model really a Vision-AND-Language model? 👀

“Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers”

📄 https://arxiv.org/abs/2109.04448

🗣️ #EMNLP2021

Reply on Twitter 1439676722417319937Retweet on Twitter 143967672241731993736Like on Twitter 1439676722417319937178Twitter 1439676722417319937
Retweet on TwitterShujian Retweeted
karpathyAndrej Karpathy@karpathy·
27 Aug

Badly tuned LR decay schedules are an excellent way to silently shoot yourself in the foot. Models can often look like they are converging but it's just LR getting too low too fast. FixedLR (+optional warmup) with 1 manual decay of 10X on plateau is a safe strong baseline.

Reply on Twitter 1431380525759885313Retweet on Twitter 1431380525759885313191Like on Twitter 14313805257598853132114Twitter 1431380525759885313
Retweet on TwitterShujian Retweeted
sirbayesKevin Patrick Murphy@sirbayes·
6 Sep

For those that think that all we need to "solve AGI" is to "train bigger models" (eg GPT4?), I recommend this paper on causal inference in / for NLP as an antidote to such naive thinking: https://arxiv.org/abs/2109.00725

Reply on Twitter 1435025390326583298Retweet on Twitter 143502539032658329884Like on Twitter 1435025390326583298594Twitter 1435025390326583298
Retweet on TwitterShujian Retweeted
jacobeisensteinJacob Eisenstein@jacobeisenstein·
4 Sep

Our NLP+causality survey paper is finally on arxiv and I'm looking forward to a relaxing three day weekend! Cause and effect? Read the paper and you might find out. But if you *do* find out, is it because you read the paper? Read the paper and you might https://twitter.com/roireichart/status/1433884845235650567

Roi Reichart@roireichart

Since Jan. 2021 we, an interdisciplinary team of researchers interested in causal inference (CI) and text, have been working on a paper that provides a unified view on the intersection of CI and NLP.

I am happy to share it is now published on arxiv:

https://arxiv.org/abs/2109.00725

Reply on Twitter 1433985701003993088Retweet on Twitter 143398570100399308811Like on Twitter 1433985701003993088101Twitter 1433985701003993088
Load More...

Categories

  • ml (2)
  • nlp (12)
  • papers (1)
  • random (8)
  • reinforcementlearning (1)
  • Uncategorized (57)

Tags

anomaly (1) automl (1) ctr (1) cv (2) data (1) distributedtraining (1) gan (1) kaggle (1) list (2) ml (2) NER (1) nlp (10) nn (1) paper (1) random (3) reinforcementlearning (1) sql (1)

Recent Posts

  • Version mismatch in embedding-based retrieval
  • NER with small strongly labeled and large weakly labeled data
  • A Survey of Data Augmentation Approaches for NLP
  • Data Augmentation in NLP
  • PyTorch nn.DistributedDataParallel

Archives

  • July 2021 (2)
  • June 2021 (1)
  • May 2020 (1)
  • October 2019 (1)
  • August 2019 (11)
  • July 2019 (8)
  • June 2019 (6)
  • May 2019 (1)
  • April 2019 (11)
  • March 2019 (4)
  • February 2019 (2)
  • January 2019 (32)

Recent Comments

    August 2019
    M T W T F S S
     1234
    567891011
    12131415161718
    19202122232425
    262728293031  
    « Jul   Oct »

    • 0
    • 2
    • 82,546
    • 40,385
    • 80
    • 0

    Recent Posts

    • Version mismatch in embedding-based retrieval
    • NER with small strongly labeled and large weakly labeled data
    • A Survey of Data Augmentation Approaches for NLP
    • Data Augmentation in NLP
    • PyTorch nn.DistributedDataParallel

    Categories

    • ml (2)
    • nlp (12)
    • papers (1)
    • random (8)
    • reinforcementlearning (1)
    • Uncategorized (57)

    Meta

    • Log in
    • Entries feed
    • Comments feed
    • WordPress.org
    © 2021 Bag of Tricks in Machine Learning • Built with GeneratePress