The Google T5Gemma model has been released

Google has released a new language model called T5Gemma, which is a large language model based on an encoder-decoder architecture and comes with 32 derived versions. T5Gemma is mainly designed for text generation tasks, including summarization, machine translation, question answering, mathematical reasoning, and reading comprehension. The model is built on the Gemma 2 framework and employs a technique called “model adaptation” to convert a pre-trained decoder-only model into an encoder-decoder architecture. It also supports flexible “imbalanced” configurations, such as adjusting between a 9B-parameter encoder and a 2B-parameter decoder, enabling a trade-off between quality and efficiency. Its advantage lies in outperforming decoder-only models under the same inference computational cost, while allowing flexible scaling of the encoder and decoder sizes according to specific tasks.