In the past two decades, 3D vision has become common place in industrial applications. Innovations in robotics, embedded systems, autonomous vehicles, and industrial automation depend on 3D vision. However, it must meet different demands from capturing fast-moving objects, lighting conditions, an industrial environment, and long distances. All these demands fulfill the FRAMOS D400e series.
In order to answer all doubts, we talked with Sam Leboe, FRAMOS Product Manager of the D400e series, who revealed all the details to us. Sam specializes in 3D and embedded cameras. With 8 years of experience in the machine vision and IoT industries. They have worked on various camera products utilizing advanced sensors, embedded processors, and high-speed interfaces. Sam’s understanding of customer needs and new technologies help Framos’ customers solve problems in the robotics and automation industries.
- Can you explain what depth sensing is and why it is important?
Depth sensing is the perception and measurement of distance in a 3D environment. This is critical in things such a Robotics where you have physical objects interacting with each other – a robot needs something to tell it how far away so that it can respond accordingly in situations such as trying to avoid an obstacle or pick something up from a bin.
- Where is depth sensing technology used?
Depth sensing is used everywhere, as humans we rely on both our eyes to gauge depth in our daily lives. Another common application people would be familiar with is the Time-of-Flight sensor integrated into recent mobile phones to assist with facial recognition and Augmented Reality experiences. It is also heavily featured in automated robotics, when the system needs to deal with a dynamic environment such as driving on the road or picking objects off pallets.
- What are types of depth sensing technologies?
There are many ways to do depth sensing, some of the most common include: Stereo vision, Time-of-Flight sensing, Structured light sensing, Lidar (light detection and ranging) and many more that leverage advanced optical techniques.
- Why was D400e series built?
The D400e series was designed to fulfill the needs of robotics system builders and integrators who want to add 3D measurement capabilities. It leverages Framos’ experience in GigE Vision camera development and our partnership with Intel Realsense to extend ways their technology can be integrated.
- What are the key features of D400e series?
The D400e series uses Intel’s Realsense stereo solution. It includes two image sensors to capture images for generating the depth map, a projector to improve the depth accuracy in scenes that lack natural texture, and an RGB image sensor to capture color information at the same time. It also uses Intel’s D4 Vision procesor to do the depth calculation onboard the device. Around this we’ve built a rugged aluminum enclosure with an IP66 rating for dust and water ingress protection. It uses a gigabit ethernet interface to enable cable runs of up to 100 meters using standard networking hardware familiar to most vision system integrators. For power, we support both Power over Ethernet (PoE) or external power via an M8 cable connection depending on what works best for the system’s design.
- What are the challenges in industrial environments?
One of the key requirements for these environments is reliability, both in terms of device uptime and the consistency of the measurement data over time. Automation is often a big capital investment for the companies implementing them, and uptime is a big factor in their return-on-investment expectations. Reliability challenges come from a variety of sources such as the complexity of automation systems, electromagnetic interference (EMI) and the dynamic nature of the applications.
- Can you explain depth technology in D400e series?
The D400e cameras are what is commonly referred to as “Active stereo” depth sensors. Stereo depth sensing is the calculation of disparity between two images taken from different perspectives a known distance apart. Because the baseline distance between the two imagers is known and the camera is factory calibrated, the disparity can be used to determine the distance of objects and features in the images. The active component of the camera refers to the integrated random dot projector that uses non-visible near-infrared wavelengths. This random dot pattern is visible to the stereo cameras, and the pattern helps improve the performance of the stereo matching engine when a scene has a lack of natural texture.
- What is the difference between D415e, D435e and D455e cameras?
The key difference between the cameras come down to the different CMOS image sensors used, the baselines for the stereo sensor pair, and the fields of view. All three cameras generate depth maps with resolutions up to 1280 x 720. The Stereo pair of the D435e and D455e use global shutter image sensors to minimize the effects of things like motion blur on the depth accuracy while the D415e use more cost effective rolling shutter sensors. The RGB sensors in the D415e and D435e use rolling shutter sensors, while the D455e uses global shutter for the overall highest performance in applications with fast moving objects. Field of view and minimum/maximum measurement distances are different for all models, a helpful table is available on our website.
- Recently FRAMOS launched new D400e-f cameras, can you tell us more about them?
What’s unique about these new cameras compared to the existing D400e cameras is the addition of filters for both sensors in the stereo pair. This change can help improve the accuracy of the depth measurements in some challenging scenarios for the stereo algorithm such as saturated parts of the image due to strong reflections, dense repeated vertical patterns, or scenes lacking natural texture where the standard IR pattern projector is insufficient.
- What are the key features of the new cameras?
With the addition of the IR-pass filters on the stereo sensors, only near-infrared light is passed while visible light is absorbed by the filter. This increases the relative intensity of the projected IR pattern in some scenes. These high contrast features allow depth to be more accurately calculated for scenes that would otherwise have insufficient texture for an accurate measurement.
- Can you tell us more about D400e data streams?
The cameras are very flexible and allow for a variety of different data streams. Most often, users will want to stream the depth images that have been calculated onboard the sensor. This is sometimes the only stream needed for applications where color information isn’t needed. Images from the RGB camera can be streamed at the same time for color information. Additionally, users can selectively stream from the left and/or right sensor of the stereo pair, this can be useful when troubleshooting or comparing the depth output of the camera’s onboard processing to other algorithms that can be run on a PC.
- What is a part of D400e series starter kit?
Our starter kit includes everything someone needs to get the camera up and running on their PC or Laptop. In addition to the camera, we include an M12 to RJ45 cable to connect to the device and a power adapter with a M8 connector to supply power for setups where you don’t have PoE available such as connecting the camera directly to a laptop for benchtop testing.
- To whom would you recommend D400e series cameras?
I would recommend this camera to any robotics system integrators who need to add reliable and robust depth sensing. It’s a great device that requires very little setup to get quality depth data from, and the inclusion of a color camera can help simplify how many different cameras are being integrated. This helps reduce the cost of the overall system to end customers.
