A benchmark version number is specific to a test. Benchmark version numbers change rarely and only when absolutely necessary to accommodate changes in third-party applications or bug fixes.

UL Procyon AI Computer Vision Benchmark v1.5.290

April 04, 2024

This update supports the release of the AI Computer Vision Benchmark for Apple Mac, with support for the Apple Core ML Inference Engine. Benchmark results are not affected.


New 

  • The ResNet50 AI model has been replaced with a PyTorch-based version, as the previous ONNX-based ResNet50 version could not be converted into Core ML. This does not meaningfully affect benchmark results.

UL Procyon AI Computer Vision Benchmark on macOS v1.0.58

April 04, 2024

This update supports the release of the AI Computer Vision Benchmark for Apple Mac, with support for the Apple Core ML Inference Engine. Benchmark results are not affected.


New

  • Added support for the Apple Core ML Inference Engine on devices running macOS.

UL Procyon AI Computer Vision Benchmark v1.4.289

April 04, 2024

This is a minor update. Benchmark results are not affected. 


Fixed

  • Updated Qualcomm SNPE to version 2.20
  • Updated Intel OpenVINO to version 2024.0
  • Updated Microsoft Windows ML to version 1.17.1

UL Procyon AI Inference Benchmark for Windows v1.3.276

January 17, 2024

This is a minor update. Benchmark results are not affected. 


Fixed

  • Updated the workload to display the correct version number

UL Procyon AI Inference Benchmark for Windows v1.2.273

January 4, 2024

This version updates the Intel OpenVINO, Qualcomm SNPE and Microsoft WindowsML runtimes. Updates to SNPE and OpenVINO result in performance improvements on supported hardware.
Results from the AI Inference Benchmark for Windows v1.2.273 or later are not comparable with results from earlier versions.


New

  • Added support for Qualcomm Snapdragon X Elite SoCs.

Updated

  • Updated Intel OpenVINO inference engine to v2023.2. This runtime update increases scores by up to 5% for most supported hardware, with some components seeing more significant score improvements.
  • Updated Qualcomm SNPE inference engine to v2.17. This runtime update increases scores by roughly 10% on supported hardware.
  • Updated Microsoft WindowsML inference engine to v1.16.3. This version requires DirectML v1.13.0 or later.

UL Procyon AI Inference Benchmark for Windows v1.2.267

November 13, 2023

This is a minor update. This version updates the Intel OpenVINO runtime, which results in performance improvements on supported hardware.


New

  • Added support for testing the inference performance of NPUs supported by the Intel OpenVINO AI Inference Engine version 2023.1.

Updated

  • Updated Intel OpenVINO AI Inference Engine to 2023.1. This runtime update increases scores by roughly 5% for most supported hardware, with some components seeing more significant score improvements.
  • Qualcomm SNPE workload changed to use the C API instead of C++ API, as Qualcomm is retiring the C++ API. Benchmark scores are not affected.

UL Procyon AI Inference Benchmark for Windows v1.1.145

July 27, 2023

This update includes updates to the runtimes used in the Procyon AI Inference benchmark for Windows. Updates to AI runtimes changes performance. Results from the AI Inference Benchmark for Windows from Procyon v2.6.847 or later are not comparable with results from earlier versions of Procyon.


AI Inference Benchmark for Windows runtimes

  • TensorRT updated to version 8.6.1 GA 
  • Windows ML updated to version 1.15.1 (DirectML to version 1.12.0)
  • OpenVINO updated to version 2023.0
  • OpenVINO models updated to IR11

New

  • New FP16 precision option for Windows ML, OpenVINO and TensorRT runtimes 
  • New GPU selector for Windows ML runtime 
  • New device selector for OpenVINO runtime

UL Procyon AI Inference Benchmark for Windows v1.0.133

March 22, 2023

  • Launch version