eIQ® Inference with TensorFlow™ Lite | NXP Semiconductors

Automotive

Industrial

Consumer

Technologies

eIQ® Inference with TensorFlow™ Lite

eIQTensorFlowLite

Integrated into NXP's Yocto development environment, eIQ software delivers TensorFlow Lite for NXP’s MPU platforms. Developed by Google to provide reduced implementations of TensorFlow (TF) models, TF Lite uses many techniques for achieving low latency such as pre-fused activations and quantized kernels that allow smaller and (potentially) faster models. Furthermore, like TensorFlow, TF Lite utilizes the Eigen library to accelerate matrix and vector arithmetic.

TF Lite defines a model file format, based on FlatBuffers. Unlike TF’s protocol buffers, FlatBuffers have a smaller memory footprint allowing better use of cache lines, leading to faster execution on NXP devices. TF Lite supports a subset of TF neural network operations, and also supports recurrent neural networks (RNNs) and long short-term memory (LSTM) network architectures.

Diagram

eIQ® TensorFlow Lite

eIQ<sup>&reg;</sup> TensorFlow Lite
Get diagram PDF

Features

  • Delivered as middleware in NXP Yocto BSP releases
  • NXP eIQ software support available for i.MX applications processors
  • Provides the ability to run inferencing on Arm® Cortex®-M, Cortex-A, Verisilicon GPUs and NPU
  • Faster and smaller than TensorFlow — enables inference at the edge with lower latency and smaller binary size
  • Uses open source libraries to accelerate matrix and vector arithmetic (Eigen and GEMMLOWP)
  • NXP optimized implementation of GEMMLOWP uses Arm® Cortex®-M7 SIMD instructions yielding 2-3x performance increase compared to out-of-the-box implementation

Supported Devices

  • i.MX8: i.MX 8 Family – Arm® Cortex®-A53, Cortex-A72, Virtualization, Vision, 3D Graphics, 4K Video
  • i.MX8M: i.MX 8M Family - Arm® Cortex®-A53, Cortex-M4, Audio, Voice, Video
  • i.MX8MMINI: i.MX 8M Mini - Arm® Cortex®-A53, Cortex-M4, Audio, Voice, Video
  • i.MX8MNANO: i.MX 8M Nano Family - Arm® Cortex®-A53, Cortex-M7
  • i.MX8X: i.MX 8X Family – Arm® Cortex®-A35, 3D Graphics, 4K Video, DSP, Error Correcting Code on DDR

Downloads

Quick reference to our software types.
Filters

2 downloads

  • Examples and Quick Start Software

    MCUXpresso SDK Builder

    EXTERNALRev 1Jan 28, 20211 KBMCUXPRESSO-SDK-EIQ
    Download
  • Software Development Resources

    eIQ Machine Learning Software for i.MX Linux 4.14.y

    EXTERNALRev 0Jun 24, 20191 KBEIQ-MLSOFTWARE
    Download

Note: For better experience, software downloads are recommended on desktop.

Y true 0 SSPeIQTensorFlowLiteen 6 Application Note Application Note t789 5 Fact Sheet Fact Sheet t523 1 en_US en_US en Application Note Application Note 5 1 1.0 Y English https://docs.nxp.com/bundle/AN14411/page/topics/introduction.html This document describes the deployment of the TensorFlow Lite inference engine and related delegates for on-chip Machine learning (ML) accelerators (NPU and GPU) available on i.MX 8MP and i.MX 95 application processors 1727191378039701171828 SSP 351.6 KB None None documents None 1727191378039701171828 /docs/en/application-note/AN14411.pdf 351558 /docs/en/application-note/AN14411.pdf AN14411 documents N N Y 2024-09-24 Enabling eIQ Core NPU Delegates for i.MX Android Applications https://docs.nxp.com/bundle/AN14411/page/topics/introduction.html /docs/en/application-note/AN14411.pdf Application Note N 645036621402383989 /bundle/AN14411/page/topics/introduction.html /docs/en/application-note/AN14411.pdf 2024-09-30 351.6 KB pdf N en Sep 23, 2024 645036621402383989 Application Note Y N Enabling eIQ Core NPU Delegates for i.MX Android Applications 2 0 English 1662723227268733735895 SSP 217.0 KB None None documents None 1662723227268733735895 /docs/en/application-note/AN13699.pdf 217036 /docs/en/application-note/AN13699.pdf AN13699 documents N N 2022-09-09 Enabling TensorFlow Operators in TensorFlow Lite Runtime for i.MX 8 Linux Platform /docs/en/application-note/AN13699.pdf /docs/en/application-note/AN13699.pdf Application Note N 645036621402383989 2022-12-07 pdf N en Aug 16, 2022 645036621402383989 Application Note Y N Enabling TensorFlow Operators in TensorFlow Lite Runtime for i.MX 8 Linux Platform 3 1 English AN12766SW.zip /docs/en/application-note-software/AN12766SW.zip /docs/en/application-note-software/AN12766SW.zip application note, AN12766, eIQ i.MXRT1060 i.MXRT1050 TensorFlow Anomaly detection 1585907866779711774116 SSP 906.9 KB None None documents None 1585907866779711774116 /docs/en/application-note/AN12766.pdf 906940 /docs/en/application-note/AN12766.pdf AN12766 documents N N 2020-04-03 Anomaly Detection with eIQ using K-Means clustering in Tensor Flow Lite /docs/en/application-note/AN12766.pdf /docs/en/application-note/AN12766.pdf Application Note N 645036621402383989 2022-12-07 pdf N en Nov 12, 2020 645036621402383989 Application Note Y N Anomaly Detection with eIQ using K-Means clustering in Tensor Flow Lite 4 1 English AN12964SW.zip /secured/assets/documents/en/application-note-software/AN12964SW.zip /webapp/Download?colCode=AN12964SW&docLang=en The purpose of this application note is to clarify the impact of the i.MX 8M Plus NPU warmup time on overall performance. When comparing NPU with CPU performance on the i.MX 8M Plus, the perception is that inference time is much longer on the NPU. This is due to the fact that the ML accelerator spends more time performing overall initialization steps. This initialization phase is known as warmup and is necessary only once at the beginning of the application. After this step inference is executed in a truly accelerated manner as expected for a dedicated NPU. 1597994156516715220287 SSP 239.7 KB Registration without Disclaimer None documents Extended 1597994156516715220287 /secured/assets/documents/en/application-note/AN12964.pdf 239681 /secured/assets/documents/en/application-note/AN12964.pdf AN12964 documents Y N 2020-08-20 i.MX 8M Plus NPU Warmup Time /webapp/Download?colCode=AN12964 /secured/assets/documents/en/application-note/AN12964.pdf Application Note N 645036621402383989 2023-06-18 pdf Y en Oct 7, 2020 645036621402383989 Application Note Y N i.MX 8M Plus NPU Warmup Time 5 0 English AN12892SW.zip /docs/en/application-note-software/AN12892SW.zip /docs/en/application-note-software/AN12892SW.zip application note, AN12892, eIQ i.MXRT1060 MNIST TensorFlow TensorFlow Lite 1592398141396719445049 SSP 731.0 KB None None documents None 1592398141396719445049 /docs/en/application-note/AN12892.pdf 730959 /docs/en/application-note/AN12892.pdf AN12892 documents N N 2020-06-16 Transfer Learning and the Importance of Datasets /docs/en/application-note/AN12892.pdf /docs/en/application-note/AN12892.pdf Application Note N 645036621402383989 2022-12-07 pdf N en Jun 17, 2020 645036621402383989 Application Note Y N Transfer Learning and the Importance of Datasets Fact Sheet Fact Sheet 1 6 4 Japanese Machine learning software for NXP i.MX and MCUs – libraries, example applications, inference engines, HALs 1562948465206707057398ja SSP 653.2 KB None None documents None 1562948465206707057398 /docs/ja/fact-sheet/EIQ-FS.pdf 653150 /docs/ja/fact-sheet/EIQ-FS.pdf EIQ-FS documents N N 2019-07-12 eIQ Software Fact Sheet /docs/ja/fact-sheet/EIQ-FS.pdf /docs/ja/fact-sheet/EIQ-FS.pdf Fact Sheet N 736675474163315314 2022-12-07 ja Jul 11, 2023 736675474163315314 Fact Sheet Y N eIQ 機械学習ソフトウェア開発環境 4 English Machine learning software for NXP i.MX and MCUs – libraries, example applications, inference engines, HALs 1562948465206707057398 SSP 653.2 KB None None documents None 1562948465206707057398 /docs/en/fact-sheet/EIQ-FS.pdf 653150 /docs/en/fact-sheet/EIQ-FS.pdf EIQ-FS documents N N 2019-07-12 eIQ Software Fact Sheet /docs/en/fact-sheet/EIQ-FS.pdf /docs/en/fact-sheet/EIQ-FS.pdf Fact Sheet N 736675474163315314 2022-12-07 pdf N en Jan 21, 2022 736675474163315314 Fact Sheet Y N eIQ Software Fact Sheet false 0 eIQTensorFlowLite downloads en true 1 Y SSP Y Y Application Note 5 2024-09-24 /bundle/AN14411/page/topics/introduction.html Sep 23, 2024 Application Note This document describes the deployment of the TensorFlow Lite inference engine and related delegates for on-chip Machine learning (ML) accelerators (NPU and GPU) available on i.MX 8MP and i.MX 95 application processors /docs/en/application-note/AN14411.pdf Y documents Y 2024-09-30 https://docs.nxp.com/bundle/AN14411/page/topics/introduction.html 645036621402383989 en None /docs/en/application-note/AN14411.pdf 351.6 KB pdf N 351.6 KB AN14411 1727191378039701171828 https://docs.nxp.com/bundle/AN14411/page/topics/introduction.html /docs/en/application-note/AN14411.pdf 1727191378039701171828 SSP 1 None English 351558 None 645036621402383989 N Enabling eIQ Core NPU Delegates for i.MX Android Applications /docs/en/application-note/AN14411.pdf documents Application Note N Y 1.0 N Enabling eIQ Core NPU Delegates for i.MX Android Applications N 2022-09-09 Aug 16, 2022 Application Note /docs/en/application-note/AN13699.pdf documents 2022-12-07 /docs/en/application-note/AN13699.pdf 645036621402383989 en None pdf N 217.0 KB AN13699 1662723227268733735895 /docs/en/application-note/AN13699.pdf 1662723227268733735895 SSP 2 None English 217036 None 645036621402383989 N Enabling TensorFlow Operators in TensorFlow Lite Runtime for i.MX 8 Linux Platform /docs/en/application-note/AN13699.pdf documents Application Note N Y 0 N Enabling TensorFlow Operators in TensorFlow Lite Runtime for i.MX 8 Linux Platform N 2020-04-03 Nov 12, 2020 Application Note application note, AN12766, eIQ i.MXRT1060 i.MXRT1050 TensorFlow Anomaly detection /docs/en/application-note/AN12766.pdf documents 2022-12-07 /docs/en/application-note/AN12766.pdf 645036621402383989 en None pdf N 906.9 KB AN12766 1585907866779711774116 /docs/en/application-note/AN12766.pdf 1585907866779711774116 SSP 3 None English 906940 None 645036621402383989 N Anomaly Detection with eIQ using K-Means clustering in Tensor Flow Lite /docs/en/application-note/AN12766.pdf documents Application Note N Y 1 N Anomaly Detection with eIQ using K-Means clustering in Tensor Flow Lite N 2020-08-20 Oct 7, 2020 Application Note The purpose of this application note is to clarify the impact of the i.MX 8M Plus NPU warmup time on overall performance. When comparing NPU with CPU performance on the i.MX 8M Plus, the perception is that inference time is much longer on the NPU. This is due to the fact that the ML accelerator spends more time performing overall initialization steps. This initialization phase is known as warmup and is necessary only once at the beginning of the application. After this step inference is executed in a truly accelerated manner as expected for a dedicated NPU. /secured/assets/documents/en/application-note/AN12964.pdf documents 2023-06-18 /webapp/Download?colCode=AN12964 645036621402383989 en Extended pdf N 239.7 KB AN12964 1597994156516715220287 /secured/assets/documents/en/application-note/AN12964.pdf 1597994156516715220287 SSP 4 Registration without Disclaimer English 239681 None 645036621402383989 Y i.MX 8M Plus NPU Warmup Time /secured/assets/documents/en/application-note/AN12964.pdf documents Application Note N Y 1 Y i.MX 8M Plus NPU Warmup Time N 2020-06-16 Jun 17, 2020 Application Note application note, AN12892, eIQ i.MXRT1060 MNIST TensorFlow TensorFlow Lite /docs/en/application-note/AN12892.pdf documents 2022-12-07 /docs/en/application-note/AN12892.pdf 645036621402383989 en None pdf N 731.0 KB AN12892 1592398141396719445049 /docs/en/application-note/AN12892.pdf 1592398141396719445049 SSP 5 None English 730959 None 645036621402383989 N Transfer Learning and the Importance of Datasets /docs/en/application-note/AN12892.pdf documents Application Note N Y 0 N Transfer Learning and the Importance of Datasets N Fact Sheet 1 2019-07-12 Jan 21, 2022 Fact Sheet Machine learning software for NXP i.MX and MCUs – libraries, example applications, inference engines, HALs /docs/en/fact-sheet/EIQ-FS.pdf documents 2022-12-07 /docs/en/fact-sheet/EIQ-FS.pdf 736675474163315314 en None pdf N 653.2 KB EIQ-FS 1562948465206707057398 /docs/en/fact-sheet/EIQ-FS.pdf 1562948465206707057398 SSP 6 None English 653150 None 736675474163315314 N eIQ Software Fact Sheet /docs/en/fact-sheet/EIQ-FS.pdf documents Fact Sheet N Y 4 N eIQ Software Fact Sheet N true Y Softwares

Documentation

Quick reference to our documentation types.

Filters

6 documents

Compact List

Application Note (5)
Fact Sheet (1)

Hardware

Quick reference to our board types.
Filters

1-5 of 6 hardware offerings

Show All

Related Software

Quick reference to our software types.

Filters

4 software files

To find a complete list of our partners that support this software, please see our Partner Marketplace.

Training

Filters

1 training