Benchmarks

In some regions, UL Procyon cannot automatically download the required AI models. In these cases, users will have to manually add the models themselves. 

Non-converted Pytorch Models

Stable Diffusion 1.5

Stable Diffusion XL

Converted Olive-optimized ONNX models

Stable Diffusion XL

Converted AMD-optimized ONNX models

Stable Diffusion 1.5

Stable Diffusion XL

Quantized OpenVINO models

Stable Diffusion 1.5

Quantized Qualcomm QNN models

Stable Diffusion 1.5

Installing the models

By default, the benchmark is installed in

%ProgramData%\UL\Procyon\chops\dlc\ai-imagegeneration-benchmark\

1. If it does not exist, create a subfolder named ‘models’ at:

   %ProgramData%\UL\Procyon\chops\dlc\ai-imagegeneration-benchmark\ 

2. In this ‘models’ folder, create the following subfolders based on the tests you are looking to run:
   1. For non-converted Pytorch models:
      Create a subfolder 'pytorch' and place each full Pytorch model in it with the model's HF ID in the folder structure; E.g.

      ...\ai-imagegeneration-benchmark\models\pytorch\nmkd\stable-diffusion-1.5-fp16\<each subfolder of the model>

      Please note: 
      The first run of benchmarks using these models can take significantly longer, as the models need to be converted.
   2. For converted Olive Optimized ONNX models for ONNX Runtime with DirectML:
      Create a subfolder ‘onnx_olive_optimized’ and place each full model in it with the model’s HF ID in the folder structure; E.g.
      

      ...\ai-imagegeneration-benchmark\models\onnx_olive_optimized\nmkd\stable-diffusion-1.5-fp16\<each subfolder of the model> 

   3. For converted AMD Optimized ONNX models for ONNX Runtime with DirectML:
      Create a subfolder ‘onnx_amd_optimized’ and place each full model in it with the model’s HF ID in the folder structure; E.g.
      

      ...\ai-imagegeneration-benchmark\models\onnx_amd_optimized\nmkd\stable-diffusion-1.5-fp16\<each subfolder of the model> 
      

   4. For quantized OVIR models for OpenVINO Runtime: 
      Create a directory ‘ovir\<HF ID>\unet’ and place each part of the int8 model in it:
      

      ...\ai-imagegeneration-benchmark\models\ovir\intel\sd-1.5-square-quantized\unet\<each required unet part>
      

   5. For quantized QNN models for QNN Runtime: 
      Create a directory ‘qnn\<HF ID>\unet’ and place each model in it:
      

      ...\ai-imagegeneration-benchmark\models\qnn\qualcomm\Stable-Diffusion-v1.5\<submodel>\<submodel>.bin keeping the original name of the files: 
      ...\text_encoder\TextEncoder_Quantized.bin 
      ...\unet\UNet_Quantized.bin 
      ...\vae_decoder\VAEDecoder_Quantized.bin 
      
      Follow the instructions in step (2.1) for the required pytorch model files

Note:

Not all models for all engines are required to always be present in the installation directory.

• For OpenVINO, only the OVIR models must exist. 
• For ONNX Runtime-DirectML, only the Olive-optimized ONNX models must exist. 
• For TensorRT, only the Engine created for the current settings (batch size, resolution) and hardware must exist. The Engine is generated from the CUDA-optimized ONNX models in case changes are made.