Discovering Versatile Language Models: The Domain-General Lottery
Ever thought about how language models learn from multiple domains? Well, turns out, not all parameters in these models behave the same way. Some act differently depending on the domain, while others stay consistent. This can lead to both strengths and weaknesses. Imagine you're trying to build a l