For each AI model test in the benchmark, a final score is calculated using the average Time to First Token (TTFT) and Output Token Speed (OTS).
TTFT is measured in seconds, and OTS is measured in tokens per second.
The Final Benchmark result is calculated as such:
Final_score=√(TTFT_Score ×OTS_Score)
The TTFT_Score and OTS_Score are calculated using constants (C1 or C2) that depend on the AI model used. The average_TTFT and Average_OTS values are the arithmetic means of the TTFT and OTS for each prompt in an AI model test.
TTFT_Score=C1/(Average_TTFT)
OTS_Score=C2 ×Average_OTS
| AI Model | C1 | C2 |
| Phi-3.5-mini | 1250 | (5000/244) |
| Mistral-7B | 1640 | (5000/244) |
| Llama-3.1-8B | 1300 | (5000/244) |
| Llama-2-13B | 2435 | (5000/90) |
Table. Constant Values for C1 and C2