Hi! My research collaborators and I happened to be in possession of a significant amount of compute. We're inspired by the Eleuther's team Pythia repo and would like to make a similar model suite for interpretability/alignment research (models at varying levels of sizes, with ~150 weight checkpoints, training data, and relevant hyperparameters, like training schedule).

I'm curious where there is demand here to answer mechanistic interpretability/scaling research questions. Some options are:

  • ViT models
  • Other language models (e.g. opensource BLOOM)
  • Multimodal models (e.g. video models like Phenaki) 

Thank you in advance.

10

0
0

Reactions

0
0
Comments
No comments on this post yet.
Be the first to respond.
Curated and popular this week
Relevant opportunities