ONNX Profiler

Profile onnx models anywhere, anytime

This tool allows you to profile ONNX models using ONNX Runtime. It generates dummy inputs based on the model's input shapes and performs profiling on both CPU and CUDA devices (if available). The profiling results are saved as JSON files.

Features

• Automatically derives input shapes and data types from the ONNX model.
• Profiles the model on CPU, CUDA (if supported) and DirectML(if supported).
• Generates profiling outputs in JSON format, saved alongside the model.
• Provides a simple API for profiling and describing ONNX models.

Installation

pip install git+https://github.com/Param-Uttarwar/onnx-profiler.git

Usage

# 1: Profiling a model

from onnxprofiler import profile

profile("model.onnx", num_runs=5)

Output

Saved as <device>_profile_<model_name>_<time>.json at the same location as the model.

• Use Perfetto UI to visualize the profiling results.

# 2: Get information about inputs and outputs of the model

from onnxprofiler import describe

describe("model.onnx")

Output

Prints output to the console.

Requirements

• Python 3.7+
• ONNX
• ONNX Runtime
• NumPy