As with many new complex technologies, machine learning is following an adoption life cycle
whereby ‘early adopters’ have utilized this technology for the last few years. So to speak, they
are ahead of the curve. But starting in 2021, the adoption shift will go toward the ‘early
majority’ (Figure 1). Until now, NXP’s eIQ® machine learning (ML) software development environment
has successfully supported the early adopters, but as we cross ‘The Chasm’, machine learning
support must become more comprehensive and easier-to-use. I’ll explain.
What Is an Early Adopter?
Early adopters for machine learning go back more than just a few years–there have been people (and
companies) working on this technology for decades. However, the move to machine learning at the
edge is a relatively recent activity and the requirements are different. The tools are different.
System resources are more constrained. The applications are more responsive. The early adopters
have had to figure out the entire process–from model training to deployment of the inference
engine that runs the model, as well as all other aspects of the system integration (i.e. the video
pipeline from capture to output of the inference).
In the development process, once a model is trained, optimized and quantized, the next phase
consists of deploying that model on the device, allowing it to perform its inferencing function.
NXP provided eIQ to support this process, combining all the software pieces, application examples,
user guides, etc. to allow the advanced developer to deploy a variety of open source inference
engines on our devices. Today, eIQ has inferencing support for TensorFlow Lite, Arm NN, ONNX
runtime and Glow neural network compiler. To use eIQ today, the developer follows a process we
call ‘bring your own model’, or BYOM. Build your trained models using public or private
cloud-based tools, then simply bring your models into the eIQ software environment and plug it
into the appropriate inference engine.
Machine Learning for All
The 2019 Embedded Markets Study by EE Times indicated that 55% of developers said their current or
future work requires machine learning (I believe this number has gone up since then). To cross the
chasm and enable machine learning for the majority of developers, machine learning support must
become more comprehensive and easier-to-use. More comprehensive support means that NXP must
provide an end-to-end workflow, allowing developers to bring in their training data, select the
right model for their application, perform model training, optimization and quantization and
finally perform on-target profiling and subsequently move to production.
Easier-to-use is a matter of perspective, but for majority adoption this means that NXP must
provide a simplified yet optimized user interface. Theoretically, this would enable a machine
learning development environment that could essentially hide the details, and with the click of a
few options, import the user’s training data and deploy the model on the target device.
Crossing the Chasm with Au-Zone Technologies
To make this a reality and cross that adopter chasm, NXP has made an investment in Canada-based
Au-Zone Technologies, establishing an exclusive, strategic partnership to expand eIQ with
easy-to-use ML tools and expand its offering of silicon-optimized inference engines for Edge ML.
Au-Zone’s DeepView™ ML Tool Suite will augment eIQ with an intuitive, graphical user interface
(GUI) and workflow, enabling developers of all experience levels (e.g. embedded developer, data
scientist, ML expert) to import datasets and/or models, rapidly train and deploy NN models and ML
workloads across the NXP Edge processing portfolio (Figure 2).
The DeepView tool suite includes a dataset workspace that will provide developers with a means to
capture and annotate images for model training and validation. Start by drawing a box around an
object(s) of interest and describe the item (search the Internet for the paid and free versions of
tools to perform this) and from there you can start building machine learning datasets and train
models. The workspace also supports dataset augmentation allowing developers to quickly adjust the
image parameters to improve model training by reducing over-fitting and increasing robustness to
dynamic real-world environments. For example, dataset augmentation functions include image
rotation, blurring, color conversions, etc. Remember, the NN model only ‘sees’ numbers, so any
variation on an input image will look different. The more variations you can provide on the
original training data, the more accurate your model will be.
The DeepViewML trainer allows developers to select hyperparameters and click train. In most cases,
you won’t be training your model from scratch, so this DeepViewML trainer supports transfer
learning (i.e. modifying the last layers of the model). A model optimizer is also part of the
DeepView ML tool suite; it will help you fine-tune your model for the desired target platform and
inference engine. The optimizer provides automatic graph-level optimizations such as pruning,
fusing and folding layers to reduce complexity and improve performance with no loss of precision.
Furthermore, optimizations that have accuracy tradeoffs are configurable by the developer. These
include quantization, layer replacement and weight rounding. You’ll be able to validate lossy
optimizations and compare them using the validator tool to fully understand the impact (i.e. size
versus accuracy).
Through this strategic partnership, we’ll also be adding Au-Zone’s DeepView run-time inference
engine to the eIQ family; this will complement the open source inference technologies already
deployed. A key feature of this DeepView engine is that it will optimize the system memory usage
and data movement uniquely for each NXP SoC architecture. Users will then have an additional
choice of inference engines - the profiling tool will quickly allow users to do performance and
memory size comparisons on the various inference runtime options.
If you haven’t already joined the
machine learning
movement, hopefully you’ll be crossing the chasm soon and taking advantage of the comprehensive
eIQ machine learning
(ML) software development environment, complete with the DeepViewML Tool Suite and DeepView
run-time inference engine, available in Q1, 2021.
