BERT for text classification

BERT can achieve high accuracy with small sample size (e.g. 1000):

To simply get features (embedding) from BERT, this Keras package is easy to start with:

For fine-tuning with GPUs, this PyTorch version is handy (gradient accumulation is implemented):

For people how have access to TPUs:

Combine different word embeddings

How to take advantage of different word embeddings in text classification task? Please check my Kaggle post:

A subset of this field is called meta-embedding. Here is a list of papers:

I found that just taking average of different embeddings is already powerful enough.

One thing to try is BERT:

Random seed optimization in GAN research?

Two quotes from this paper: Are GANs Created Equal? A Large-Scale Study

  • Even with everything else being fixed, varying the random seed may influence on the results
  • authors often report the best FID which opens the door for random seed optimization