Installing required software

Make sure you are connected to the Internet.

Updating macOS

  1. Ensure you are running the latest version of macOS. This benchmark requires macOS 26 or later. 

Installing the Procyon AI Text Generation Benchmark application

  1. Install Procyon:
    Procyon-<version>-text-generation-benchmark-macos.pkg
  2. Follow the prompts to complete the installation. 

Note:  
the Procyon AI Text Generation Benchmark is installed to the ‘/Library/UL/Procyon/AITextGeneration’ folder. 

Registering your Procyon license 

  1. Open Terminal. 
  2. Go to installation folder:  
    cd /Library/UL/Procyon/AITextGeneration
  3. Register your license:
    ./UL_Procyon --register <license key>

Running the AI Text Generation Benchmark 

Once the benchmark is installed and permissions configured properly, it can be run as follows: 

  1. Open Terminal. 
  2. In Terminal, navigate to the installation folder with:
    cd /Library/UL/Procyon/AITextGeneration
  3. Run the AI Text Generation benchmark:
    ./UL_Procyon -d ai_textgeneration_<model>.def -o “<output path>/<output file>.procyon-result” --export-pdf <PDF File Name>.pdf
    Example:
    (Please note: "ProcyonTestUser/Procyonresults" is an example folder, attempting to export results to a folder that does not exist will mean no results file is created.)
    ./UL_Procyon -d ai_textgeneration_all.def 
    -o “/Users/ProcyonTestUser/Procyonresults/AITextGenerationResult.procyon-result” --export-pdf AITextGenerationResult.pdf 

Notes:

  1. Unlike the AI Text Generation Benchmark on Windows, we have already created individual definition files for each configuration of device type and precision. The complete list can be found below: 
    ai_textgeneration_all.def
ai_textgeneration_phi.def
ai_textgeneration_mistral.def
ai_textgeneration_llama2.def
ai_textgeneration_llama3.1.def
  2. If the output path has whitespace, it must be escaped (note quotation marks and backslash).  
    Example:
    for output folder “/Users/Steve/result files/“ :
    -o “/Users/<username>/result\ files/AITextGenerationResult.procyon-result” 

Viewing your results

Exporting with command line switch

Option 1:

  1. Use --export-csv, --export-pdf or --export-xml command line switch to export results as an XML or CSV file:  
    ./UL_Procyon -d ai_textgeneration_<model>.def -o <Export File Name>.zip --export-csv <CSV File Name>.csv --export-pdf <PDF File Name>.pdf --export-xml<XML File Name>.xml
    Example:
    ./UL_Procyon -d ai_textgeneration_all.def -o ZipAITextGenerationResult.zip --export-csv AITextGenerationResult.csv --export-pdf AITextGenerationResult.pdf --export-xml AITextGenerationResult.xml

Option 2:

  1. Use the --export-simple-CSV command line switch to export results as a single CSV file:  
    ./UL_Procyon -d ai_tetxgeneration_<model>.def -o <Export File Name>.zip -l 2 --export-simple-csv <CSV File Name>.csv
    Example:
    ./UL_Procyon -d ai_textgeneration_all.def -o ZipAITextGenerationResult.zip -l 2 --export-simple-csv AITextGenerationResult.csv

Option 3:

  1. Use the --online command line switch to upload results online:  
    ./UL_Procyon -d ai_textgeneration_<model>.def --online
    Example:
    ./UL_Procyon -d ai_textgeneration_all.def --online

Option 4:

  1. Use the --export-prompts command line switch to export all generated prompts as .txt files:  
    ./UL_Procyon -d ai_textgeneration_<model>.def --export-prompts "<directory path>"
    Example:
    ./UL_Procyon -d ai_textgeneration_all.def --export-prompts "Users/<username>/prompts/"

Extracting Result.xml from the .procyon-result file.

(This is essentially the same as using –export-xml command) 

  1. Unzip the .procyon-result file to extract the contents. Your benchmark results are available in the ‘Result.xml’ file.  

Once you have the CSV or XML file, you can process the data with your preferred tools.


Unregistering UL Procyon AI Text Generation Benchmark

The benchmark license can be unregistered as follows: 

  1. Open Terminal. 
  2. In Terminal, navigate to the installation folder with:  
    cd /Library/UL/Procyon/AITextGeneration
  3. Unregister your license:
    ./UL_Procyon --unregister


Uninstalling the Procyon AI Text Generation Benchmark 

The benchmark can be uninstalled as follows: 

  1. Open Terminal. 
  2. In Terminal, navigate to the installation folder with:
    cd /Library/UL/Procyon/AITextGeneration
  3. Run the uninstallation script:
    sudo ./uninstall.sh

Note:   

It is not recommended to uninstall just by removing the installation folder, as it will leave the Procyon app installed in the operating system registers. This can cause issues when installing Procyon again later.