Camera system design is complex. An appropriate lens and image sensor combination must be selected for the target application and the image signal processor (ISP) needs to be calibrated and tuned. This process involves capturing hundreds of pictures of various specialized target images under various lighting conditions. These RAW images are then processed by a tuning tool to generate a base-line ISP calibration from which the fine (read: subjective) tuning can commence. Does the target image have a white that looks a little too blue? Or does the black look more like a dark gray? Do you need to adjust the gamma correction, the white balance or both to fix that? These are some of the many questions an experienced image engineer can answer.
Learn more about the complexities of digital imaging systems and image sensor calibration. Read our technical article Understanding the Challenges of Image Signal Processing Calibration.
An integrated ISP provides several significant advantages over a discrete ISP. When receiving image data in a RAW format we can achieve bandwidth savings over full color RGB or YUV. A 4K image (3840 x 2160). Using the 8-bit RAW format can achieve 60 frames per second (FPS) on the i.MX 8M Plus applications processor. If a discrete ISP was used in your system design, and the output uses the RGB888 format, the i.MX 8M Plus processor is bandwidth limited to 30 FPS. Using the RAW format allows system designers to either use less MIPI lanes or accommodate larger frame sizes or frame rates.
On top of that, a discrete ISP adds cost. A discrete ISP ranges in price between $6-12 (USD), and an FPGA based solution can run upwards of $20 (USD). That’s not counting the extra PCB area and additional power supply requirements – neither of which are free. Additionally, the i.MX 8M Plus processor has two integrated ISPs.
Another benefit to an integrated ISP on the i.MX 8M Plus processor is the NXP Product Longevity program. The parts in this program (including the i.MX 8M Plus processor) are guaranteed to be in production for 15 years. For example, the i.MX 8M Plus processor, launched in 2021, will be available until at least 2036. This program also includes support from NXP engineers for the remaining lifetime of these parts.
NXP would love to introduce you to a handful of trusted partners who have demonstrated the capabilities required to help bring our customers design ideas to life. Below is the current list of partners that not only have extensive digital image system design experience, but they also have proven calibration experience with the i.MX 8M Plus processor’s ISP.
FRAMOS can help with these complex ISP and IQ issues. FRAMOS’ partnership with NXP provides them with the in-depth technical support required to master an image sensor's special features. This enables FRAMOS’ engineers and developers to take advantage of innovative sensor features that are not “standard” or that are only available in a very constrained implementation. FRAMOS has the tools and know-how to get the best image quality out of the image signal processor. Contact FRAMOS to discuss RAW IQ characterization, system level processed IQ characterization, IQ optimization, custom IQ tests, reports and tuning.
InnoWave has unmatched expertise in the world of vision, in-depth expertise in the areas of firmware development for video processing and embedded systems. They provide complete solutions which include software drivers, 3A libraries (auto-exposure, auto-white balance and auto-focus), calibration and tuning tools. They develop and port custom imaging algorithms and deliver the most optimal solutions for various use-cases. Camera characterization (shading, distortion) and calibration for both intrinsic and extrinsic parameters ISP tuning for various camera parameters, such as color and white balance, accuracy, sharpness, noise, tonal response, dynamic range, optical distortion, etc. For more information, visit Innowave.
eInfochips, an Arrow company, is a leading global provider of product engineering and semiconductor design services. With over 500+ products developed and 40M deployments in 140 countries, eInfochips continues to fuel technological innovations in multiple verticals. eInfochips has enabled companies to develop customized evaluation kits, reference designs and next-generation, fully featured products on NXP platforms. The core strength of eInfochips lies in NXP technologies. They have enabled community boards targeted for embedded machine learning, HMI, industrial, automotive, health monitoring, building control, dash cams and smart homes. eInfochips services also support custom hardware firmware, BSP, HW acceleration, algorithms, video optimization, end-to-end product design and manufacturing. For more information, visit eInfochips.
Basler built the 4K BCON camera for the launch of the i.MX 8M Plus processor. They’re a leading international manufacturer of high-quality imaging components for computer vision applications. In addition to classic area scan and line scan cameras, lenses, frame grabbers, light modules, 3D products and software, the company offers embedded vision solutions for the i.MX 8M Plus processor, including consulting services, customer-specific software development and customized products. Basler's products are used in a variety of markets and applications like factory automation, medical, logistics, retail and robotics. They are characterized by high reliability, an excellent price to performance ratio and long-term availability. For more information, visit Basler.
IMDT is a leading designer and manufacturer of a variety of System-On-Modules (SOM) featuring i.MX applications processors. IMDT’s SOM solutions are ideal for cost sensitive, low-power and high-performance embedded products. IMDT provides hardware and software design services specifically related to its SOM products as well as custom boards. All IMDT products are manufactured and tested in the company’s own ISO9001:2015 certified facilities. Products are CE certified and RoHS compliant. For more information, visit IMDT.
Leopard Imaging is a global leader providing high-definition embedded cameras and AI-based imaging solutions. They focus on core technologies that improve image processing in automotive vehicles, aerospace vehicles, drones, IoT and robotics. Leopard Imaging works closely with the most established sensor and AI platform companies in producing advanced camera solutions for global customers. With original equipment manufacturer (OEM) and original design manufacturer (ODM) services, Leopard Imaging provides one-stop-shop customization in camera design, manufacturing, calibration, imaging tuning, driver development, system integration and related services.
PHYTEC offers products and services that enable fast and easy integration of embedded vision systems. Phytec supplies matched components like board-level cameras, controller modules, single board computers and customized hardware into the customers' serial products. For evaluation purposes Embedded Vision Kits are available. Phytec also offers design service and mass production service for fully customized embedded vision systems. The Linux-BSPs of the phyCORE System on Modules include software drivers for the phyCAM industrial graded board level cameras. In addition, the BSPs already include imaging middleware such as OpenCV, Gstreamer and Halcon, along with ISP pre-calibrations for selected Phytec cameras. Individual ISP calibrations are offered for customer projects based on Phytec products. For more information, visit PHYTEC.
Yosun Imaging Solution Center (a.k.a. ISC) was established in 2018 and focuses on research, development, integration and promotion of CMOS Image Sensors Solutions in many diverse applications for automotive, AIoT, security, consumer, computing, mobile, industrial and machine vision. They collaborate with ISP and SoC platforms and software algorithm partners to develop reference designs in response to market requirements which are beneficial to demonstrating image quality and features — allowing their customers a reduced development and design process. ISC has Imaging Labs in Shanghai, Shenzhen and Taipei and possess a wealth of professional experience in fine-tuning image quality. With a deep understanding of CMOS Image Sensor technology, they excel in optimizing parameters like auto white balance, auto exposure and auto focus to achieve the highest image fidelity through precise calibration, noise reduction and color accuracy.
Image Quality Labs Labs has more than 20 years of operational experience in selecting, integrating, assessing, tuning and optimizing camera modules into multiple existing and emerging markets. Image Quality Labs offers companies an end-to-end imaging solution. They offer software development of camera device drivers and applications, custom imaging hardware design, image quality testing and image quality tuning and optimizations. For more information, visit Image Quality Labs.
TechNexion is a NXP Gold Partner with decades of experience working with NXP i.MX applications processors. They provide a complete family of fully calibrated MIPI CSI-2 camera sensors ranging from 1MP to 20MP, rolling and global shutters and a wide selection of lens options with cables and adapters to connect directly with NXP official evaluation kits (EVK). This enables rapid prototyping and an easing of the path to integrating embedded vision and machine learning for NXP i.MX processor customers. TechNexion MIPI CSI-2 camera sensors come with software support for the official NXP i.MX 8M, i.MX 8M Mini, i.MX 8M Nano and i.MX 8M Plus processor evaluation boards as well as the latest i.MX 93 processor EVK. Their VizionViewer™ and VizionSDK allow for desktop tuning of their embedded camera module parameters for easy fine-tuning and adjustment post-calibration. For more information, visit TechNexion.
John has over 20 years of semiconductor experience ranging from analog circuit design to system-in-package architecture to software and IP integration and debug. He is currently focused on NXP’s various HW and SW IP to support computer vision and AI. He graduated with a BSEE from The University of Texas at Austin.