Basic Usage¶
Using your saved models¶
nnio provides three classes for loading models in different formats:
Loaded models can be simply called as functions on numpy arrays. Look at the example:
import nnio
# Create model and put it on TPU device
model = nnio.EdgeTPUModel(
model_path='path/to/model_quant_edgetpu.tflite',
device='TPU:0',
)
# Create preprocessor
preproc = nnio.Preprocessing(
resize=(224, 224),
dtype='uint8',
padding=True,
batch_dimension=True,
)
# Preprocess your numpy image
image = preproc(image_rgb)
# Make prediction
class_scores = model(image)
See also nnio.Preprocessing
documentation.
Description of the basic model classes¶
- class nnio.ONNXModel(model_path: str)¶
This class is used with saved onnx models.
Usage example:
# Create model model = nnio.ONNXModel('path/to/model.onnx') # Create preprocessor preproc = nnio.Preprocessing( resize=(300, 300), dtype='uint8', batch_dimension=True, channels_first=True, ) # Preprocess your numpy image image = preproc(image_rgb) # Make prediction class_scores = model(image)
Using this class requires onnxruntime to be installed. See Installation.
- __init__(model_path: str)¶
- Parameters
model_path – URL or path to the .onnx model
- forward(*inputs, return_info=False)¶
This method is called when the model is called.
- Parameters
*inputs – numpy arrays, Inputs to the model
return_info – bool, If True, will return inference time
- Returns
numpy array or list of numpy arrays.
- get_input_details()¶
- Returns
human-readable model input details.
- get_output_details()¶
- Returns
human-readable model output details.
- class nnio.EdgeTPUModel(model_path: str, device='CPU')¶
This class works with tflite models on CPU and with quantized tflite models on Google Coral Edge TPU.
Using this class requires some libraries to be installed. See Installation.
- __init__(model_path: str, device='CPU')¶
- Parameters
model_path – URL or path to the tflite model
device – str.
CPU
by default. SetTPU
orTPU:0
to use the first EdgeTPU device. SetTPU:1
to use the second EdgeTPU device etc.
- forward(*inputs, return_info=False)¶
This method is called when the model is called.
- Parameters
*inputs – numpy arrays, Inputs to the model
return_info – bool, If True, will return inference time
- Returns
numpy array or list of numpy arrays.
- get_input_details()¶
- Returns
human-readable model input details.
- get_output_details()¶
- Returns
human-readable model output details.
- property n_inputs¶
number of input tensors
- property n_outputs¶
number of output tensors
- class nnio.OpenVINOModel(model_bin: str, model_xml: str, device='CPU')¶
This class works with OpenVINO models on CPU, Intel GPU and Intel Movidius Myriad.
Using this class requires some libraries to be installed. See Installation.
- __init__(model_bin: str, model_xml: str, device='CPU')¶
- Parameters
model_bin – URL or path to the openvino binary model file
model_xml – URL or path to the openvino xml model file
device – str. Choose Intel device:
CPU
,GPU
,MYRIAD
If there are multiple devices in your system, you can use indeces:MYRIAD:0
but it is not recommended since Intel automatically chooses a free device.
- forward(inputs, return_info=False)¶
- Parameters
inputs – numpy array, input to the model
return_info – bool, If True, will return inference time
- Returns
numpy array or list of numpy arrays.