Supported model framework

AI Model Deployer supports the following model frameworks:

TensorFlow: frozen format (.pb format), saved model (.pb format) and metagraph model (.meta format)
Keras: .h5 format
ONNX: .onnx format
PyTorch: .pth format

For different FW versions of TM NPU, the supported framework versions may be different. The GUI of AI Model Deployer can guide you through selecting the compatible model framework.

The framework supporting information is as follows:

Selected firmware version	Framework	Supported framework version
2.0	TensorFlow	1.x and 2.x (≤ 2.4)
	Keras	≤ 2.4
	ONNX	≤ 1.12
	PyTorch	≤ 1.13

Note:

Limitation for models to run on TM NPU

AI Model Deployer warns if the size of the trained model which needs to be converted is over 200 MB.

The size of the converted model for TM NPU should be less than 100 MB.

TensorFlow

The following TensorFlow models are tested with AI Model Deployer and can be converted to TM NPU compatible model format.

	Pre-trained Models	Model Parameters	TM NPU Firmware version	Type
Frozen model	Inception v2	Input shape: 1,224,224,3 Mean values: 127.5,127.5,127.5 Scale values: 127.5,127.5,127.5	2.0	Image classification
Frozen model	MobileNet v1	Input shape: 1,224,224,3 Mean values: 127.5,127.5,127.5 Scale values: 127.5,127.5,127.5 Input: input Output: MobilenetV1/Predictions/Reshape_1	2.0	Image classification
Frozen model	SSD MobileNet v1	Input shape: 1,300,300,3 Inputs: image_tensor Outputs: detection_scores,detection_boxes,num_detections TensorFlow custom operation configuration: ssd_v2_support.json Custom TensorFlow object detection pipeline configuration: pipeline.json	2.0	Object detection
Frozen model	SSD MobileNet v2	Input shape: 1,300,300,3 Inputs: image_tensor Outputs: detection_classes,detection_scores,detection_boxes,num_detections TensorFlow custom operation configuration: ssd_v2_support.json Custom TensorFlow object detection pipeline configuration: pipeline.json	2.0	Object detection
Saved model	SSD MobileNet v1	Input shape: 1,300,300,3 Inputs: image_tensor Outputs: detection_scores,detection_boxes,num_detections TensorFlow custom operation configuration: ssd_v2_support.json Custom TensorFlow object detection pipeline configuration: pipeline.json	2.0	Object detection
Saved model	SSD MobileNet v2	Input shape: 1,300,300,3 Inputs: image_tensor Outputs: detection_classes,detection_scores,detection_boxes,num_detections TensorFlow custom operation configuration: ssd_v2_support.json Custom TensorFlow object detection pipeline configuration: pipeline.json	2.0	Object detection
Meta graph model	SSD MobileNet v1	Input shape: 1,300,300,3 Inputs: image_tensor Outputs: detection_scores,detection_boxes,num_detections TensorFlow custom operation configuration: ssd_v2_support.json Custom TensorFlow object detection pipeline configuration: pipeline.json	2.0	Object detection
Meta graph model	SSD MobileNet v2	Input shape: 1,300,300,3 Inputs: image_tensor Outputs: detection_classes,detection_scores,detection_boxes,num_detections TensorFlow custom operation configuration: ssd_v2_support.json Custom TensorFlow object detection pipeline configuration: pipeline.json	2.0	Object detection

TensorFlow2

The following TensorFlow2 models are tested with AI Model Deployer and can be converted to TM NPU compatible model format.

	Pre-trained Models	Model Parameters	TM NPU Firmware version	Type
Saved model	MobileNet v1 Converted from Keras model	Input shape: 1,224,224,3 Mean values: 127.5,127.5,127.5 Scale values: 127.5,127.5,127.5	2.0	Image classification
Saved model	MobileNet v2 Converted from Keras model	Input shape: 1,224,224,3 Mean values: 127.5,127.5,127.5 Scale values: 127.5,127.5,127.5	2.0	Image classification
Saved model	MobileNet v3 small Downloaded from Keras applications and converted to TensorFlow 2 saved model	"input_shape": "(1,224,224,3)"	2.0	Image classification
Saved model	MobileNet v3 large Downloaded from Keras applications and converted to TensorFlow 2 saved model	"input_shape": "(1,224,224,3)"	2.0	Image classification

Keras

The following Keras models are tested with AI Model Deployer and can be converted to TM NPU compatible model format.

Pre-trained Models	Model Parameters	TM NPU Firmware version	Type
MobileNet v1	Input shape: 1,224,224,3 Mean values: 127.5,127.5,127.5 Scale values: 127.5,127.5,127.5	2.0	Image classification
MobileNet v2 Downloaded from Keras applications	Input shape: 1,224,224,3 Mean values: 127.5,127.5,127.5 Scale values: 127.5,127.5,127.5	2.0	Image classification
MobileNet v3 small Downloaded from Keras applications	Input shape: 1,224,224,3 Mean values: 127.5,127.5,127.5 Scale values: 127.5,127.5,127.5	2.0	Image classification
MobileNet v3 large Downloaded from Keras applications	Input shape: 1,224,224,3 Mean values: 127.5,127.5,127.5 Scale values: 127.5,127.5,127.5	2.0	Image classification
EfficientNet B0 Downloaded from Keras applications	Input shape: 1,224,224,3	2.0	Image classification

ONNX

The following ONNX models are tested with AI Model Deployer and can be converted to TM NPU compatible model format.

Pre-trained Models	Model Parameters	TM NPU Firmware version	Type
MobileNet V2 (Converted from PyTorch model)	Mean values: 123.675,116.28,103.53 Scale values: 58.395,57.12,57.375	2.0	Image classification
YOLO v2 tiny (Converted from Keras model)	Input shape: 1,3,416,416 Mean values: 123.675,116.28,103.53	2.0	Object detection

Pre-trained Models

Model Parameters

TM NPU Firmware version

Type

MobileNet V2

(Converted from PyTorch model)

Mean values: 123.675,116.28,103.53
Scale values: 58.395,57.12,57.375

2.0

Image classification

YOLO v2 tiny

(Converted from Keras model)

Input shape: 1,3,416,416
Mean values: 123.675,116.28,103.53

2.0

Object detection

PyTorch

The following PyTorch models are tested with AI Model Deployer and can be converted to TM NPU compatible model format.

Pre-trained Models	Model Parameters	TM NPU Firmware version	Type
MobileNet V2	Input shape: 1,3,224,224 Mean values: 123.675,116.28,103.53 Scale values: 58.395,57.12,57.375	2.0	Image classification