The Procyon AI Text Generation Benchmark lets you test and compare how a device performs local LLM inference tasks such as using an on-device AI assistant to help with simple office tasks.
This benchmark consists of four workloads using the Phi-3.5-mini, Mistral 7B, Llama-3.1-8B and Llama-2-13B LLM instruct models. The test will run all workloads for which the system meets the requirements, with each workload increasing in model size and completion time.
These tests measure the performance of using a local LLM to analyze locally stored documents in order to summarize or answer queries about their contents. This benchmark defaults to the recommended supported inference engine for currently installed hardware.