A growing number of “intelligent” devices and machines are placing new demands on embedded image functionalities and systems. Whether it’s driverless cars, intelligent vacuum cleaners or robots interacting as colleagues in industrial production – only native components of fully integrated vision systems make them effective. Embedded vision, as an integrated technology in devices and machines, ensures that all industries and almost any application will benefit from imaging.
The Internet of Things (IoT) creates the swarm intelligence of holistic systems by connecting all devices among one another to interact accordingly. Embedded Vision technologies provide the eyes and brain power (AI) for autonomous decision making without any human interaction to empower the Vision of Things (VoT) to act intelligently within the Internet of Things.
Embedded vision gives visual intelligence to machines and teaches them how to see and think. Embedded vision in a device defines the integration of image processing with digital processing and intelligent algorithms to evaluate an acquired image, and the associated video data. In this way, a seeing and thinking machine can react to events and processes, and interact with the environment. Integrated image processing is the basis for safe, man-machine collaboration, and the networked activities of robots and machines in the automated Smart Factory.
What differentiates Embedded Vision from Classic Vision?
Regular vision systems are mainly built with a camera that is connected to a host PC with a known data interface. The system is mostly separated into the machine that runs and the controlling process that does the inspection. The processing of the video stream and images are mostly outsourced and often needs user interaction for validation and/or decision making.
A surveillance application may recognize people, but a security officer needs to validate any abnormal occurrence to determine if it is a threat that needs an immediate response. In comparison, a security based Embedded Vision application would be able to assess the thread, determine that the threat is a person of interest and alert the authorities to react accordingly. In this case, the vision technology inside of a device, a complete system with intelligent on-board processing, can provide an appropriate response without any human interaction.
Embedded Vision is not only part of the device, it is the ‘smart eye’. In its entirety Embedded Vision minimizes or removes human interactions within the imaging pipeline and allows machines to make their own decisions by capturing, analyzing and interpreting the data all-in-one.
From a developer’s standpoint, classic vision systems were mostly made to support numerous verticals with multitude of possible tasks to be programmed. This broad variety is the main reason for the large processing space requirements needed off-board. Embedded Vision tends to be more laser-focused in its applications, it is designed for a specific task. This “purpose-build” approach opens new possibilities and frees processing space to be used for neural intelligence algorithms. From the vision manufacturer perspective, he does not have to provide a one-fits-all product and cover all possible use cases but can specialize and focus his development on the “how” of a specific system which will be customized later by the OEM developer to satisfy his unique requirements.
Classic Vision
Separated system for capturing image streams and processing, outsourced analysis for decisions, made to support many possible tasks.
Embedded Vision
Fully integrated all-in-one system capturing image streams, on-board processing and interpreting data, autonomous decision making, resulting in an action. Made for a specific task.<
Advantages of Embedded Vision
An important factor in the expansion of integrated systems continues to be progressive miniaturization and the simultaneous increase in performance. In today’s world, even a drone can visualize without requiring a significant increase in size or weight – this improvement is of enormous importance to its energy consumption, maneuverability, and range. Now, intelligent “sensing” applications can be embedded into stationary, and both mobile and portable devices:
From a developer’s standpoint, classic vision systems were mostly made to support numerous verticals with multitude of possible tasks to be programmed. This broad variety is the main reason for the large processing space requirements needed off-board. Embedded Vision tends to be more laser-focused in its applications, it is designed for a specific task. This “purpose-build” approach opens new possibilities and frees processing space to be used for neural intelligence algorithms. From the vision manufacturer perspective, he does not have to provide a one-fits-all product and cover all possible use cases but can specialize and focus his development on the “how” of a specific system which will be customized later by the OEM developer to satisfy his unique requirements.
Less required space
Less Weight
Lower Power Consumption
Lower Costs per Unit
Embedded Vision Applications
Embedded vision, as an integrated technology in both smaller devices and industrial machines, ensures that all industries and many kinds of applications will derive a benefit from integrated image processing. Some industrial examples are seeing robots, self-positioning lasers, or obstacle-avoiding drones. In the consumer sector, driverless cars, intelligent pens, or coffee machines outline the great potential of embedded vision.
Industrial applications | Consumer applications |
|---|---|
Drones for process monitoring and optimization | Home robotics; e.g., robotic vacuum cleaners, and robotic lawn mowers
|
Human-machine collaboration | Home security
|
Quality and process control | Entertainment (virtual reality)
|
Mixed reality (MR), e.g. for assembly tasks | Wearables; e.g., helmets and clothing
|
Fully Embedded Vision products for OEMs, Industrial Customers and Users
It is all about efficiency. Embedded Vision brings vision technology to its simplest formula “capture, process, respond”. The building blocks of a true embedded vision systems are:
- Sensor or Sensor Module
- Control unit - receiving the images and direct them to the processing unit
- Processing unit - either local or cloud based providing the full image pipeline
- Purpose build algorithms and neural networks - intelligent image processing
Embedded Vision requires more analysis and processing of the image data, so an embedded vision product typically includes back-end processing that is done on either an ISP or GPU. Intelligent algorithms running on these devices allow the machine to analyze the incoming video data, process and interpret them to make decisions and react accordingly. Embedded vision products provide not only data, but results based on this data.
“Embedded Vision does not have to be small but it has to be smart.” – Darren Bessette
Embedded Vision for Smart Factory and IoT
Today, the afore mentioned features of embedded vision, and the wide range of available design options, make embedded vision systems the basis for intelligent automation in the Smart Factory, and modern production. The potential of seeing, thinking, and interactive machines comes into full play both in “Internet of Things” (IoT) environments, and networked systems.
Embedded Vision does not only take pass / fail or yes / no decisions based on some criteria, it provides a broader form of intelligence leveraging neural networks that massage and analyze the image data and information. Embedded Vision systems moves from pre-defined actions based on specific inputs to specific reactions to spontaneous situations, with real-time decision making and resulting activities. This is similar to how smart cameras work but allows for adaptation and expansion with evolving responses as more scenarios are encountered and evaluated. Also, the “smarts” are being increasingly integrated into every kind of device. This creates new IoT devices that are more aware and better able to process inputs from their surroundings, further propagating how Embedded Vision is enabling more VoT devices.
Most industrial and consumer products are internet-aware today, exchanging data with one another using local networks and the cloud. With the addition ofvision, these devices will be controllable via eye tracking, face or gesture recognition. As an example, a refrigerator with embedded vision inside would be able to recognize what food has been consumed and then automatically add it to the family’s online shopping list. Using intelligent embedded vision, security applications can count people, create heat maps, or identify persons of interest and share the visual and analytics data within networks. When it comes to self-driving cars, embedded vision steers the vehicle within its lane on the road and avoids obstacles that may appear without any warning. This example highlights the importance of Embedded Vision in this application to not only to see but to also understand the scene and react accordingly. From a technical perspective, a smart embedded vision system not only recognizes defects or abnormalities based on pre-defined criteria, it can determine an appropriate response to correct or avoid them. Embedded Vision provides a more comprehensive view of the world by recognizing, understanding and identifying the environment without further external interaction.
The integration of different vision systems with a variety of video and vision inputs and outputs, and different imaging pipelines, is a challenge to new embedded architectures, and image-processing analyses in real time. Triggers and control pulses can be controlled synchronously within the overall system; even in dynamic systems under changing conditions. Herein lies the advantage of embedded vision, and the opportunity to have new applications see and think with the help of image processing. Manufacturers need consistently precise results for quality control, robotic control, and more efficient production – the ever-increasing integration of embedded vision, and its further development help to ensure its successful implementation.
FRAMOS enables machines to see and think. Whether it’s about selecting individual components or complex system compositions for embedded vision, our engineers support with both technical expertise and services. Ranging from sensors to systems, the FRAMOS engineering services helps in choosing the right components, customizing solutions or completely developing specific solutions for embedded vision systems.