What happens when a small language model reuses its layers?
I trained five models on BabyLM to test one idea: whether repeating the same transformer layers can buy better predictions without buying more parameters.
Writing
2 essays on ideas, craft, and the world.
