Hi! My research collaborators and I happened to be in possession of a significant amount of compute. We're inspired by the Eleuther's team Pythia repo and would like to make a similar model suite for interpretability/alignment research (models at varying levels of sizes, with ~150 weight checkpoints, training data, and relevant hyperparameters, like training schedule).
I'm curious where there is demand here to answer mechanistic interpretability/scaling research questions. Some options are:
- ViT models
- Other language models (e.g. opensource BLOOM)
- Multimodal models (e.g. video models like Phenaki)
Thank you in advance.