Developing embedded systems faster
Whether for the car or the drone: Developing image processing software for embedded systems takes a lot of time and is therefore very expensive. Now the TULIPP platform makes it possible to develop energy-efficient embedded image processing systems more quickly and less expensively, with a drastic reduction in time-to-market. The Fraunhofer Institute for Optronics, System Technologies and Image Exploitation IOSB is a member of the EU consortium which simplified the process.
At first glance drones, driver assistance systems and mobile medical diagnostic equip-ment don’t appear to have much in common. But in reality they do: they all make in-creasing use of image processing components, for example for detecting obstacles and pedestrians. Image processing can also be used with mobile x-ray equipment to ensure adequate image quality at reduced radiation levels, thus considerably reducing radioac-tive exposure.
In contrast to a workstation computer, where dimensions and energy consumption are not particularly critical factors, applications like these call for small, lightweight, energy-efficient image processing components that are nevertheless real-time capable. Hard-ware platforms based on conventional computer architectures and processors can’t properly meet these requirements. This is why embedded systems using field-program-mable gate arrays (FPGAs) are often used.
Field-programmable gate arrays are logic components whose circuit structure can be freely configured using a special type of programing, usually involving the low-level language VHDL. There’s a problem, however: The majority of image processing applica-tions are written in higher-level programming languages such as C/C++, and their mi-gration to the embedded systems is highly complicated. Not only does VHDL differ greatly from other programming languages, but the code must also be adapted to the specific hardware. This means even existing VHDL programs can’t be transferred to other hardware. Software developers have to start virtually from scratch with every new system.
Starter kit for rapidly implementing especially energy-efficient embedded systems
A consortium of eight partners from six countries, including the Fraunhofer Institute for Optronics, System Technologies and Image Exploitation IOSB in Karlsruhe, has now considerably simplified this procedure in the TULIPP project. “The result is a develop-ment platform consisting of design guidelines, a configurable hardware platform and a real-time-capable operating system that supports multicore processors, as well as a pro-graming tool chain,” says Dr.-Ing. Igor Tchouchenkov, group manager at Fraunhofer IOSB. “A starter kit put on the market by one of our partners in TULIPP provides addi-tional support. The starter kit makes developing such applications much faster and easier. Porting C++ programs to FPGA, which frequently means several months of work for the developer, can be handled within only a few weeks using the TULIPP starter kit.”
This means the developer first has to consider, based on the software programmed in C++, which code elements should be distributed to which hardware components and which program steps could be optimized or parallelized. The formulated design guide-lines provide help with this task. Then the starter kit comes into play. It contains the configurable hardware to which the necessary sensors and output devices can be con-nected, the multiprocessor-capable real-time operating system, and what is called the STHEM toolchain. The applications in the toolchain make it possible to optimize the C++ program in such a way that it can be ported to the FPGA as easily and quickly as possible. “One special focus of the toolchain is on energy optimization: after all, the aim is to design image processing systems that can be powered by a small battery whenever possible,” says Tchouchenkov. “The toolchain makes it possible to individu-ally display and optimize energy consumption for each code function.”
A hundred times faster than a high-end PC
The consortium worked through three specific use cases in order to develop and test the TULIPP platform: The Fraunhofer IOSB research team addressed stereo camera based obstacle detection for drones, while other project partners worked on pedestrian de-tection in the vicinity of a car and on live enhancement of X-ray images taken by mo-bile C-Arms during surgical operations. In TULIPP they ported the corresponding image processing software from C++ to FPGA.
The results are impressive: The processing, which originally took several seconds to ana-lyze a single image on a high-end PC, can now run on the drone in real time, i.e. now approximately 30 images are analyzed per second. It was also possible to increase the speed of pedestrian detection by a factor of 100: Now the system can analyze 14 im-ages per second compared to one image every seven seconds. And upstream filters and improved image exploitation have made it possible to reduce radiation during X-ray ex-aminations to one fourth of the previous level. At the same time energy consumption could be significantly reduced for all three applications.