This document provides an overview of converting an ONNX model to a symbol table that can be used as input for the DNN Compiler (DNNC). It describes what ONNX is, how ONNX models are structured and encoded, and the classes and objects used to represent the model and computational graph. It also outlines the process for parsing an ONNX model into a symbol table and proposes next steps, including writing test cases and researching how to process the symbol table using LLVM.

1. ONNX to DNNC Symbol Table:
Status Update
Colin Lu, AI Engineering Intern
Ravit Sharma, Software Engineering Intern
https://github.com/ai-techsystems/dnnCompiler

2. A Step Back: DNN Compiler
Overview

3. Project Goal
Use Symbol Table as Input
to LLVM
Obtain ONNX Model
Parse ONNX Model into
Symbol Table

4. What is ONNX?
- An open format that can represent models from different
frameworks
- Currently only ONNX operators (neural networks/DL) are supported
by DNNC, not ONNX-ML (classical ML)
- Contains the computation graph for a particular DL Model

5. ONNX vs ONNX-ML
ONNX operator set (neural networks)
- Convolutional layers, activation functions, etc.
- Utility and math functions
e.g. flatten tensor and log, respectively
ONNX-ML (classical ML)
- Classical models, like SVM and tree ensemble
(possibly for random forest, boosted trees, etc.)
- Data processing utilities
e.g. imputing missing values, one-hot encoding
The complete listing for ONNX and ONNX-ML operator sets can be found here:
- ONNX: https://github.com/onnx/onnx/blob/master/docs/Operators.md
- ONNX-ML: https://github.com/onnx/onnx/blob/master/docs/Operators-

6. Symbol Table Formatting
- Represented as text file containing model structure
- Should have standardized format, regardless of whether ONNX or
some other format (e.g. Caffe) is being used to generate the table.

7. Symbol Table Formatting
ONNX Format
Symbol Table
Format

8. Protobuf Encoding of .onnx
- Model information is stored in an .onnx file as a binary following
onnx.proto3 format
- Needs a .proto file in order to decode the binary. A .proto has
definitions for classes (e.g. models, graphs, nodes, etc.)
- Must be compiled in terminal using the protoc command, generating a source
file that can be imported

9. Protobuf Encoding of .onnx
- After .onnx binary is loaded in to an appropriate object in your
source file (onnx.ModelProto object in the python script)
- Relevant attributes are filled in and the object ready for use
Loading the ONNX Model

10. Step-by-step of ONNX model
loading
● Gather the .onnx (model stored as binary) and .proto (class definition
used to decode the .onnx)
● Compile the .proto into a (.py or other language) source file
○ Command: protoc -I=$SRC_DIR --python_out=$DST_DIR
$SRC_DIR/[filename].[proto/proto3]
○ For example: protoc -I=. --python_out=. ./onnx.proto3
● (Working in .py file now, not terminal) Import source file created by
protoc
○ Source file should be in ./onnx/proto3_pb2.py; add an empty __init__.py inside ./onnx to
allow python to recognize proto3_pb2.py as an importable library
○ In python: from onnx import proto3_pb2

11. Step-by-step of ONNX model
loading
● Create template variable to be filled in with data from the .onnx binary
○ In python: model = proto3_pb2.ModelProto()
● Load in the data from the binary
○ In python: with open(“model.onnx”, “rb”) as f:
model.ParseFromString(f.read())
At this point model can be accessed like a regular object, e.g. use model.graph.node to
get a list of nodes.

12. ONNX model structure
A .onnx file is organized in the following hierarchy:
All possible attributes of the model, graph, node objects, etc. are
described in the onnx.proto3 file.

13. ONNX Graph Paradigm
- Similarly to TensorFlow and some other frameworks, ONNX
represents a model as a computational graph (example image on
next page)
- Operations like addition, activation functions (e.g. ReLU),
convolutional layers, etc. are represented as nodes in the
computational graph

14. Sample Computation Graph
- Graph on right displays flow of
operations from input to output
- This information is represented in
.onnx file

15. Functions of ONNX Classes
● Model
○ Overarching class; the entire .onnx file describes a single model
○ Basically a wrapper for a graph object + metadata (such as version of
the ONNX representation, and the operator set version)
● Graph
○ Collection of elements that make up the computational graph
○ The graph object is what you might think of as the ML model, i.e. a
neural network that takes in inputs and computes outputs

16. Functions of ONNX Classes
● Node
○ Each node in the graph represents an operation, such as max pooling,
reshaping a tensor, adding a bias, etc.
○ In ONNX, is represented by the op_type, inputs, and outputs, along with any
attributes necessary to perform the operation (e.g. stride for a maxpool op)
● Input/Output
○ Primarily composed of some metadata (tensor name, docstrings) + Tensor
object, which contains the type (int32, float, etc.) and tensor dimensions

17. Pseudocode of Parser

18. Testing
● A set of test cases with expected outputs will be created to verify that there are no bugs or
missing components in current or future versions of the symbol table generator.
● Around 50 test cases is the target, including some common CNN architectures (ResNet,
GoogLeNet, etc.)

19. Next Steps Proposal
● Testing
○ Writing test cases to verify parser is working
○ Integration into github (CI)
● Next components in the compiler pipeline
○ Research on LLVM and processing the symbol table