KIsSME — using artificial intelligence to recognize critical traffic situations

Already, assistance systems support people in such situations and make driving safer, more efficient and more comfortable. In the partially and fully automated driving of the future, this will happen to an even greater extent. In the distant future, vehicles will transport people and goods from A to B completely autonomously and make reliable and safe decisions in real time. But we are not there yet. Many systems and solutions need to be developed and tested on the path to this goal.

During test drives for environmental detection, sensors, not people, record all the information generated in traffic. They keep an eye on the surroundings as well as the condition of the automated vehicle. Huge amounts of data of several gigabytes per second are generated, unfiltered. Processing, storing and analyzing this information is immensely cost-intensive, time-consuming and hardly or not at all manageable with conventional methods. In addition, this is often not very efficient for test drives, because every test drive is carried out under a certain premise: For example, the automated system can be tested at a junction, in moving traffic or in coordination with other vehicles. Each situation requires a different focus of environmental detection.

If the data recorded by sensors during test drives could be filtered to suit the specific situation, this would contribute to the efficiency of test drives and the faster development of automated vehicle systems. A team from Bosch Research led by Dr. Daniel Schmidt, research engineer in the field of evaluating automated driving functions and Bosch’s KIsSME coordinator, has achieved just that. It developed a software prototype for use in test drives with automated vehicles as part of the “KIsSME” joint project funded by the Federal Ministry for Economic Affairs and Climate Change. The acronym stands for: “Artificial intelligence for selective near-real-time recording of scenario and maneuver data in the testing of highly automated vehicles”. What is the prototype capable of? “Based on artificial intelligence, the software identifies critical traffic situations according to parameters determined in advance of the test drive,” says Daniel Schmidt.

Only the data relevant to the test case is saved. Among other things, this data will be used to train maneuvering algorithms so that automated vehicles can react even better to complex traffic situations. The use of artificial intelligence (AI) was central to the entire project. “We used a mix of AI methods, from machine learning to variational autoencoders,” says Daniel Schmidt. The AI methods thus range from simple and well-known methods to complex neural networks, which, like in the case of the variational autoencoder, process different information simultaneously and reduce it to the essential elements in order to answer the actual question. The composition of these measures supports the data collection process in many ways. AI came into play in many areas: from image recognition of the vehicle camera to the assessment of questionable driving maneuvers by other road users and the reliable prediction of future driving maneuvers by other road users.

A fundamental aspect of the KIsSME software is that it is applicable on a modular basis. That means: Depending on which automated system is to be tested in the vehicle, only the relevant data is collected and stored for evaluation. “Whether a system should have the function of driving so safely and comfortably in traffic circles that the vehicle does not fly out of the bend and the occupants are always comfortable, or whether a vehicle should overtake a truck on the highway quickly and yet safely, are two completely different approaches,” explains Daniel Schmidt. However, the camera, ultrasonic, LIDAR and radar sensors used during test drives all aggregate data continuously.

The KIsSME software can be used to define in advance which test case is to be tried out, i.e. which sensor data is to be explicitly collected. The software can be compared to a foil that is stretched around a lamp: punched out at the points through which the light should fall to create a pattern that is projected onto the wall or ceiling. Thanks to the defined parameters for the traffic circle situation or the overtaking maneuver on the freeway, the software only allows the relevant information to pass through to the digital storage location. This creates the relevant data set with which the systems for automated driving can be further developed and improved in a targeted manner. This saves time and money, but there is also an ecological aspect, because the less data is collected and stored, the less server power is required and therefore the less energy is consumed.

In order to validate the functions of highly automated vehicles, maneuver-based tests and statistical evidence based on enormous amounts of real-time data have been necessary up to now. With the software developed in the KIsSME project, the necessary data volumes can be reduced and only the significant sensor data relevant to critical driving situations can be recorded during the test drives. In the KIsSME project, Bosch Research combined its strengths in research on researching autonomous driving and AI with the expertise of various partners: the companies RA Consulting GmbH, Mindmotiv GmbH and LiangDao GmbH and scientific partners from the Fraunhofer Institute for High-Speed Dynamics (EMI), the Research Center for Information Technology (FZI) and the Karlsruhe Institute of Technology (KIT). The consortium was led by AVL Deutschland GmbH. The KIsSME software is still a prototype, but it is already being used in test drives. With it, the Bosch Research project team is laying the foundations for the Bosch divisions to further develop safe assistance systems and solutions for automated vehicles efficiently and sustainably.