I'm happy to see a bigger focus on expanding language coverage and less on human feedback. Human feedback has it's place but personally I've found all models that train heavily on human feedback become sycophantic and weak willed in a bad way. Personally I prefer hallucinations over blindly agreeing with everything said / telling me what I want to hear rather than what I need to hear.
The reason I am happy to see expansion of language coverage, is that linguistic research indicates that different cultures capture different parts of the world at different resolution. It depends on cultural values and priorities, but certain aspects of the world and the environment get a greater share of the cultural lexicon and so those aspects are better captured in that language.
Since LLM's much make sense of our world from language alone, it makes sense to me that the best way forward is to expand language coverage, since this gives the model better insight into more areas it cannot experience, and therefore deepen it's understanding of, itself.
This makes me think the first to make a really successful model will be the ones to pony up to get human feedback at scale from the smartest humans instead of what they do now.
I'm happy to see a bigger focus on expanding language coverage and less on human feedback. Human feedback has it's place but personally I've found all models that train heavily on human feedback become sycophantic and weak willed in a bad way. Personally I prefer hallucinations over blindly agreeing with everything said / telling me what I want to hear rather than what I need to hear.
The reason I am happy to see expansion of language coverage, is that linguistic research indicates that different cultures capture different parts of the world at different resolution. It depends on cultural values and priorities, but certain aspects of the world and the environment get a greater share of the cultural lexicon and so those aspects are better captured in that language.
Since LLM's much make sense of our world from language alone, it makes sense to me that the best way forward is to expand language coverage, since this gives the model better insight into more areas it cannot experience, and therefore deepen it's understanding of, itself.
This makes me think the first to make a really successful model will be the ones to pony up to get human feedback at scale from the smartest humans instead of what they do now.