Combine different word embeddings

How to take advantage of different word embeddings in text classification task? Please check my Kaggle post:

A subset of this field is called meta-embedding. Here is a list of papers:

I found that just taking average of different embeddings is already powerful enough.

One thing to try is BERT: