The interaction through gesture control in E-learning environments, virtual teams, and in remote conferences, is based on 3D imaging and increases the focus and attention of participants though innovative applications.
Knowledge is more easily conveyed, exchanged, and retained when a high level of natural user interaction is employed. E-learning providers are developing novel interactive applications based on this principle. In addition, teams working in different locations benefit from the improved clarity and perspective in virtual meetings; for example, a user can turn objects 360° by simply pointing at them. 3D image processing permits intuitive gesture control for the precise interaction between the user and the software. Currently, the depth perception required for this purpose is simple to integrate into mobile devices with small 3D modules for immersive user interactions, and an intuitive user experience. This article shows how only 3D enables this interaction and shows the advantages of modern 3D technology for developer and companies seeking for cutting-edge ways of collaboration through virtual worlds.
For example:
Several second graders sit around their classroom table. A large screen in the middle of the room captures the pupils’ attention with colorful dinosaurs. While the teacher explains how these extinct reptiles lived, the animals wander through a virtual prehistoric landscape on the screen. The children point their fingers, and the system responds by displaying more information. The turn of a hand rotates a dinosaur by 90° to show it from behind. A tilt of the hand forward shifts the view to the dinosaur’s feet; spreading the thumb and index finger apart zooms in for a closer view. “Wow! Look at those claws,” exclaims a pupil.
Similarly, a scenario could play out in the development department of a global production company, with distributed teams collaborating on virtual models of their latest products. E-learning or shared-working solution providers attach great importance to drawing their users into the virtual environment. Highly versatile haptic or touchscreen solutions with cameras are presently available for this purpose. They rely on simple operation and interactive elements like video streaming, or photo and document uploads. However, up to now, only pure game manufacturers like Nintendo® were able to offer direct user interaction via gesture control. This mode involves capturing users’ movements with a camera and converting them into actions on the screen. Now that 3D modules are available on the market, it is easy to transfer the interaction techniques from gaming to e-learning schemes, virtual conferences, or global team solutions. The latest 3D technology opens up new possibilities for creating virtual learning environments, or worlds, in which users can contribute in innovative ways.
Picture 1:
2D versus 3D: Precise Interaction in Real Time Brings Movement to the Screen
Precise 3D data is vital for correct, accurate control of virtual environments, and for real-time interaction. Accurate image and gesture recognition with 2D technology is highly dependent on the lighting; usually it is restricted to a specific device or a certain position, and requires adaptation to the user or software training. Additionally, gestures must be relearned and programmed for each device and application. 3D vision is possible with commercially available PC or laptop cameras, but only with much programming effort. In addition, this method involves many assumptions, and the rendering of precise control in virtual worlds is nearly impossible together with greatly increased latencies. A 2D camera can estimate the size, shape, and proportions of people or objects. However, it cannot determine whether a child’s hand is becoming larger because it is approaching the screen, or in reality it is the hand of an adult. The distance required for this purpose is derived from predefined models. However, the estimates of 2D distances are only approximations. The greatest variances in size and inaccurate estimates, particularly occur for children. Consequently, algorithms based on 2D images have difficulty categorizing and correctly distinguishing gestures. This difficulty is particularly true for both Asian women’s hands, and European men’s hands in in front of the screen. Capturing gestures in the directions of right, left, up and down is easy; but, frontal or oblique movements are difficult – the precise measurement of depth as the third dimension is difficult to achieve with 2D technology.
Picture 2: Virtual Worlds for Medical Applications; Surgeons Perform Brain Surgery Using Augmented Reality
3D Increases Safety, Security and User Acceptance of the “New Learning”
3D technology is crucial for the recognition, location, and the orientation of gestures in real time. It does not matter whether the task involves the control of virtual dinosaurs in an interactive learning environment, or three-dimensional CAD models in virtual development teams. 3D technology uses position, distance, and angular measurement (both linear and rotational), to provide extensive information about user interactions and the commands issued by their gestures. The position of the hands, inclination of the elbows, changes in distance, and other variables that transfer the activity to the screen contribute to rotating objects in virtual space. The precise coordinates in the X, Y and Z planes reduce computing power, hardware resources, and the programming effort. This precision accelerates and refines the entire cognition and processing procedure; distance-based scaling errors do not affect the application. Complete virtual interaction is only practical with 3D data. In addition, the implementation of 3D imaging is a safety and security factor. A 2D camera can be deceived by the presentation of a photograph of a person or object; a 3D camera detects the lack of depth information. When modern 3D modules – like the Intel® RealSense™ line – are seamlessly integrated into the frames of laptops, tablets or touchscreens, users cannot distinguish them from conventional devices. Gesture control becomes more natural, because users interact with it in the same manner as with real objects. Therefore, they consider it unobtrusive, which is a deciding factor for user acceptance. The bonus of gesture control as an “entertainment factor” results in more productive e-learning or meeting sessions. 3D technology supports the “new learning” with practical relevance; for example, through virtual reality (VR) or augmented reality (AR). 3D technology simulates real environments, and expands them with virtual elements and digital information. They control these environments interactively to make learning and working more interesting, relevant, and appealing, with a more lasting result. Readily integrated 3D technologies help to implement new learning possibilities and help to expand new ways of working together.
3D modules for simple development and integration, as well as rapid market maturity
Developers of educational or “new work” applications can select ready-to-use 3D starter kits, which offer mature technology at affordable prices and are easy to integrate into customer-specific devices. This technology, in systems like the Intel® RealSense™ line, captures information and interprets it simultaneously. It is based on sensor technology with an integral processor that supplies both 2D and 3D data. Then, depending on the customer’s requirement and application, the data is sent to a GPU, CPU, or an ASIC processor. Subsequently, the intelligent gesture recognition algorithms can be implemented on these platforms. This facility allows engineers to produce 3D applications without long development times, and provides users with an interactive and intuitive experience.
