Phi 3
Phi-3 is a state of the art, lightweight model. It performs exceptionally well despite being half the size of Llama-3 8B. It is highly capable of being an on-device agentic LLM.
note
| Model | Params | Context Length | GQA | Token Count | Knowledge Cutoff |
|---|---|---|---|---|---|
| Phi-3 Mini 128k Instruct | 3.8B | 128,000 | Yes | 3.3T | October 2023 |
Phi-3 Mini 128k Instruct
| Model | Function Calling | MMLU | GPQA | GSM-8K | MATH | MT-bench | MT-bench Pairwise Comparison | |||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Win | Loss | Tie | Win Rate | Loss Rate | Adjusted Win Rate | |||||||
| Phi-3 Mini 128k Instruct | - | 69.36 | 27.01 | 83.7 | 32.92 | 8.02 | 21 | 72 | 67 | 0.13125 | 0.45000 | 0.340625 |
| Rubra Enhanced Phi-3 Mini 128k Instruct | 70.0% | 67.87 | 29.69 | 79.45 | 30.80 | 8.21 | 72 | 21 | 67 | 0.45000 | 0.13125 | 0.659375 |
- The above is based on the Phi-3 Mini that was updated by Microsoft in June 2024. The original Phi-3 mini came out in April and the Rubra enhanced model has been trained on both versions