Gemma

Gemma is a family of lightweight models developed by Google. Rubra offers the 2B variant with function calling, which is most appropriate for single instruction chats.

note

Model	Params	Context Length	GQA	Token Count	Knowledge Cutoff
Gemma-1.1 2B Instruct	2B	8192	No (MQA)	6T	Undisclosed

tip

We observe in the benchmarks and in our own testing that Gemma models do not perform well when the number of turns (chat messages) increases.

Gemma-1.1 2B Instruct

rubra-ai/gemma-1.1-2b-it
rubra-ai/gemma-1.1-2b-it-GGUF

Model	Function Calling	MMLU	GPQA	GSM-8K	MATH	MT-bench	MT-bench Pairwise Comparison
Model	Function Calling	MMLU	GPQA	GSM-8K	MATH	MT-bench	Win	Loss	Tie	Win Rate	Loss Rate	Adjusted Win Rate
Gemma-1.1 2B Instruct	-	37.84	22.99	6.29	6.14	5.82	33	56	71	0.20625	0.35000	0.428125
Rubra Enhanced Gemma-1.1 2B Instruct	45.00%	38.85	24.55	6.14	2.38	5.75	56	33	71	0.35000	0.20625	0.571875

Gemma-1.1 2B Instruct​

Gemma-1.1 2B Instruct