GPT-2 token embeddings visualized

2022 January 22

I visualized the token and position embeddings for GPT-2. The plot below uses t-SNE for the visualization. Click and drag to zoom in.