What is Embedded Vision?

Smart Imaging for the Vision of Things

What is Embedded Vision

A growing number of “intelligent” devices and machines are placing new demands on embedded image functionalities and systems. Whether it’s driverless cars, intelligent vacuum cleaners or robots interacting as colleagues in industrial production – only native components of fully integrated vision systems make them effective. Embedded vision, as an integrated technology in devices and machines, ensures that all industries and almost any application will benefit from imaging.

The Internet of Things (IoT) creates the swarm intelligence of holistic systems by connecting all devices among one another to interact accordingly. Embedded Vision technologies provide the eyes and brain power (AI) for autonomous decision making without any human interaction to empower the Vision of Things (VoT) to act intelligently within the Internet of Things.

Embedded vision gives visual intelligence to machines and teaches them how to see and think. Embedded vision in a device defines the integration of image processing with digital processing and intelligent algorithms to evaluate an acquired image, and the associated video data. In this way, a seeing and thinking machine can react to events and processes, and interact with the environment. Integrated image processing is the basis for safe, man-machine collaboration, and the networked activities of robots and machines in the automated Smart Factory.

What differentiates Embedded Vision from Classic Vision?

Regular vision systems are mainly built with a camera that is connected to a host PC with a known data interface. The system is mostly separated into the machine that runs and the controlling process that does the inspection. The processing of the video stream and images are mostly outsourced and often needs user interaction for validation and/or decision making.

A surveillance application may recognize people, but a security officer needs to validate any abnormal occurrence to determine if it is a threat that needs an immediate response. In comparison, a security based Embedded Vision application would be able to assess the thread, determine that the threat is a person of interest and alert the authorities to react accordingly. In this case, the vision technology inside of a device, a complete system with intelligent on-board processing, can provide an appropriate response without any human interaction.

Embedded Vision is not only part of the device, it is the ‘smart eye’. In its entirety Embedded Vision minimizes or removes human interactions within the imaging pipeline and allows machines to make their own decisions by capturing, analyzing and interpreting the data all-in-one.

From a developer’s standpoint, classic vision systems were mostly made to support numerous verticals with multitude of possible tasks to be programmed. This broad variety is the main reason for the large processing space requirements needed off-board. Embedded Vision tends to be more laser-focused in its applications, it is designed for a specific task. This “purpose-build” approach opens new possibilities and frees processing space to be used for neural intelligence algorithms. From the vision manufacturer perspective, he does not have to provide a one-fits-all product and cover all possible use cases but can specialize and focus his development on the “how” of a specific system which will be customized later by the OEM developer to satisfy his unique requirements.

Classic Vision
Separated system for capturing image streams and processing, outsourced analysis for decisions, made to support many possible tasks.

 

Embedded Vision
Fully integrated all-in-one system capturing image streams, on-board processing and interpreting data, autonomous decision making, resulting in an action. Made for a specific task.<

Advantages of Embedded Vision

An important factor in the expansion of integrated systems continues to be progressive miniaturization and the simultaneous increase in performance. In today’s world, even a drone can visualize without requiring a significant increase in size or weight – this improvement is of enormous importance to its energy consumption, maneuverability, and range. Now, intelligent “sensing” applications can be embedded into stationary, and both mobile and portable devices:

From a developer’s standpoint, classic vision systems were mostly made to support numerous verticals with multitude of possible tasks to be programmed. This broad variety is the main reason for the large processing space requirements needed off-board. Embedded Vision tends to be more laser-focused in its applications, it is designed for a specific task. This “purpose-build” approach opens new possibilities and frees processing space to be used for neural intelligence algorithms. From the vision manufacturer perspective, he does not have to provide a one-fits-all product and cover all possible use cases but can specialize and focus his development on the “how” of a specific system which will be customized later by the OEM developer to satisfy his unique requirements.

Less required space

Less Weight

Lower Power Consumption

Lower Costs per Unit

Embedded Vision Applications

Embedded vision, as an integrated technology in both smaller devices and industrial machines, ensures that all industries and many kinds of applications will derive a benefit from integrated image processing. Some industrial examples are seeing robots, self-positioning lasers, or obstacle-avoiding drones. In the consumer sector, driverless cars, intelligent pens, or coffee machines outline the great potential of embedded vision.

Industrial applications
Consumer applications
Drones for process monitoring and optimization
Home robotics; e.g., robotic vacuum cleaners, and robotic lawn mowers
Human-machine collaboration
Home security
Quality and process control
Entertainment (virtual reality)
Mixed reality (MR), e.g. for assembly tasks
Wearables; e.g., helmets and clothing

Fully Embedded Vision products for OEMs, Industrial Customers and Users

It is all about efficiency. Embedded Vision brings vision technology to its simplest formula “capture, process, respond”. The building blocks of a true embedded vision systems are:

Embedded Vision requires more analysis and processing of the image data, so an embedded vision product typically includes back-end processing that is done on either an ISP or GPU. Intelligent algorithms running on these devices allow the machine to analyze the incoming video data, process and interpret them to make decisions and react accordingly. Embedded vision products provide not only data, but results based on this data.

“Embedded Vision does not have to be small but it has to be smart.” – Darren Bessette

Typical embedded vision components come in small form factors, but this does not mean that they can only be used in small or low-cost devices. Self-driving cars are the exception to using embedded vision in small, low cost devices. A better way to think of embedded vision is to think of smart imaging that does not need human interaction to process and react to the imaging/video stream. Embedded Vision is enabling machines to see and think, powering the Vision of Things (VoT) of the future.
 

Embedded Vision for Smart Factory and IoT

Today, the afore mentioned features of embedded vision, and the wide range of available design options, make embedded vision systems the basis for intelligent automation in the Smart Factory, and modern production. The potential of seeing, thinking, and interactive machines comes into full play both in “Internet of Things” (IoT) environments, and networked systems.

Embedded Vision does not only take pass / fail or yes / no decisions based on some criteria, it provides a broader form of intelligence leveraging neural networks that massage and analyze the image data and information. Embedded Vision systems moves from pre-defined actions based on specific inputs to specific reactions to spontaneous situations, with real-time decision making and resulting activities. This is similar to how smart cameras work but allows for adaptation and expansion with evolving responses as more scenarios are encountered and evaluated. Also, the “smarts” are being increasingly integrated into every kind of device. This creates new IoT devices that are more aware and better able to process inputs from their surroundings, further propagating how Embedded Vision is enabling more VoT devices.

Most industrial and consumer products are internet-aware today, exchanging data with one another using local networks and the cloud. With the addition ofvision, these devices will be controllable via eye tracking, face or gesture recognition. As an example, a refrigerator with embedded vision inside would be able to recognize what food has been consumed and then automatically add it to the family’s online shopping list. Using intelligent embedded vision, security applications can count people, create heat maps, or identify persons of interest and share the visual and analytics data within networks. When it comes to self-driving cars, embedded vision steers the vehicle within its lane on the road and avoids obstacles that may appear without any warning. This example highlights the importance of Embedded Vision in this application to not only to see but to also understand the scene and react accordingly. From a technical perspective, a smart embedded vision system not only recognizes defects or abnormalities based on pre-defined criteria, it can determine an appropriate response to correct or avoid them. Embedded Vision provides a more comprehensive view of the world by recognizing, understanding and identifying the environment without further external interaction.

The integration of different vision systems with a variety of video and vision inputs and outputs, and different imaging pipelines, is a challenge to new embedded architectures, and image-processing analyses in real time. Triggers and control pulses can be controlled synchronously within the overall system; even in dynamic systems under changing conditions. Herein lies the advantage of embedded vision, and the opportunity to have new applications see and think with the help of image processing. Manufacturers need consistently precise results for quality control, robotic control, and more efficient production – the ever-increasing integration of embedded vision, and its further development help to ensure its successful implementation.

FRAMOS enables machines to see and think. Whether it’s about selecting individual components or complex system compositions for embedded vision, our engineers support with both technical expertise and services. Ranging from sensors to systems, the FRAMOS engineering services helps in choosing the right components, customizing solutions or completely developing specific solutions for embedded vision systems.

Start your Embedded Vision Project

Sensors

The Foundation of Imaging Systems

Sensors

The Foundation of Imaging Systems

Modules

Embedded Vision Modules for Seamless Integration

Modules

Embedded Vision Modules for seamless integration

3D

3D Vision for Autonomous Applications

3D

3D Vision for Autonomous Applications

Optics

The Best Optics for Image Processing

Optics

The Best Optics for Image Processing

Microdisplays

MOLED´s for AR/VR & Viewfinder

Microdisplays

MOLED´s for AR/VR & Viewfinder

Do you have any questions?