AI-based Smart Test Facility Automation with Safe Active Learning
Bosch Research Blog | Posted by Daniel Neyer, Volker Imhof, Christoph Zimmer and Matthias Kraenzler, 2023-10-26
Artificial intelligence empowered computational models are increasingly important within the electric drive domain, e.g., virtual temperature- and torque estimation, embedded digital twins. In order to develop such dedicated AI models, sufficient high-quality measurement data needs to be acquired. This can be very cost and labor intensive. Our novel, AI-based “Smart Test Facility Automation with Safe Active Learning” allows for smart and automated test bench operation with the benefit of smart and automated test bench operation:
- Automatically learning and adhering to safety constraints allows for unsupervised test bench operation e.g., central for over-night and weekend runs
- Smart selection and generation of test bench trajectories, increases informativeness in the multi-dimensional input space and further reduces measurement campaign duration.
A collaboration between different units within Bosch Research implemented a Safe Active Learning framework communicating with Bosch test benches to significantly reduce calibration- and measurement effort for a variety of test systems — reducing cost and speeding up innovation.
Loading the video requires your consent. If you agree by clicking on the Play icon, the video will load and data will be transmitted to Google as well as information will be accessed and stored by Google on your device. Google may be able to link these data or information with existing data.
Why are AI models of high importance for e-drives?
The fast-paced development of new and innovative AI-based concepts to increase efficiency, performance and reliability of Bosch e-drive systems and components lead to an increased demand in test bench measurement campaigns for training, calibration and evaluation.
With enlarged usage of various embedded AI models within Bosch control units the developers are looking at new ways to make the calibration and training process automated and more efficient. In order to bring embedded AI models into application, the developers face the significant challenge of acquiring genuine and sufficient measurement data from a test bench for the development of “application/production-ready” models, since the performance of AI models is inherently dependent on the quality of the data provided for training.
What is challenging for data generation?
The development and training of AI models based on test bench measurements involves significantly more resources compared to simulations, which leads to the strong need of automated trajectory generation and AI training to accelerate the development cycle from an initial AI-model definition via acquiring measurement training data to the deployment on the target hardware.
This causes new and very demanding requirements such as the automated and efficient operation of the test bench over long periods of time to acquire high-quality training data for the subsequent training of AI models such as digital twins.
The execution of a measurement campaign is currently a predominantly manual process, which involves considerable effort, experience, and constant monitoring which results in long dedicated sessions that causes high costs. Trajectories need to be manually planned, executed and analyzed carefully depending on the characteristics of the AI model. Moreover, it is crucial to ensure that the entire multidimensional operation area of the electric motor has been covered, which is a challenging task.
Another significant issue is the potential violation of safety limits, which leads to an immediate termination of the ongoing measurement cycles and subsequent cooling down times, hence delays of the entire measurement campaign.
How can we improve the process of data generation?
To address this challenge Bosch Research experts together with Bosch Research domain experts of the Strategic Portfolio “Electrified Mobility and Systems” have developed an AI-based method that enhances the capabilities of the Bosch test benches by introducing a Safe Active Learning framework that automatically explores the whole multi-dimensional operating area of an electric motor with optimized trajectories while simultaneously training an AI model during runtime.
“Safe Active Learning for Time-Series Modeling with Gaussian Processes” [i] — a mathematical AI based algorithm — was initially published at the prestigious Neural Information Processing Systems Conference in 2018.
The Safe Active Learning method is a sequential design based on three steps:
- Automatically plan a new trajectory that is highly informative and safe
- Automatically execute the trajectory on a test bench
- Use measurements to update the AI model for next trajectory generation
Step 1: The core of the algorithm is a constrained optimization problem. The aim is to explore the multi-dimensional operating area of an unknown system (e.g., electric motor) on a test bench without violating safety critical constraints. Therefore, the objective of the optimization problem is to generate new exploratory trajectories in so far unknown areas. The constraints include information of the pre-defined operating areas and additional thresholds to be avoided (e.g., temperature limits, vibration limits, …). The new and informative trajectory is sent from the Active Learning Client to the test bench via generic interfaces.
Step 2 is based on the automation of the test bench to operate the input trajectories generated by the Safe Active Learning framework and returning the measured data via the same interfaces.
In Step 3 the Active Learning Client, uses the information of the latest measurement send by the test bench to update the internal probabilistic model, to calculate the next informative trajectory and to improve the AI model for the safety constraints.
What is the benefit for Bosch?
“Developing production-ready models in hours, not weeks”
Safe Active Learning enhances the test bench capabilities and avoids shutdowns by automatically planning and adapting new trajectories based on predefined safety constraints, thereby allowing for a safe automated day, night and even weekend operation. This significantly reduces the required measurement campaign time and optimizes the utilization of the test benches at the individual business units within Bosch.
First evaluations of test benches enhanced with Safe Active Learning capabilities show a significant reduction in required test bench time compared to the current conventional manual process of obtaining measurement data for subsequent model calibration or AI training. This guarantees a fast time-to-market for future AI models within the Bosch product portfolio.
What are the next steps?
The Smart Test Facility Automation based on Safe Active Learning enhances the capabilities of different test benches of various business departments. This innovative technology will contribute to the task of bringing AI into new domains and applications in multiple domains within Bosch.
What was your personal highlight?
Volker:
I experienced two special moments in this project. One was when we first got the system running: Generating a safe trajectory, sending it via the Interfaces to the test bench, seeing the system starting to turn, measuring the trajectory and receiving it back at the Active Learning Client. The second moment was when I wanted to show my colleagues that the Safe Active Learning is running but I could not enter the control room of the test bench because it already was running unattended.
Christoph:
As an interdisciplinary mathematician, I find it really fascinating to see a mathematical algorithm – initially developed in an academic publication – getting industrialized at a Bosch test bench and create business impact for e-mobility. Success of a great interdisciplinary team!
Matthias:
I really appreciate that we brought together a team with people of quite diverse backgrounds and skills, where everyone was interested and open to benefit from each other's knowledge. I think this shows the big advantage of the great competencies within Bosch. It was also great to see that we were able to bring a new methodology from basic AI research to life on our e-mobility test benches. And to be honest - the first evening when we turned off the lights in the test bench and went home while the AI was autonomously controlling the test bench was somehow a magical moment.
Daniel:
I enjoyed working with all the experts in this interdisciplinary team that has the clear focus on solving challenges by introducing new and innovative AI methods that cover the whole range from probabilistic models to the application and integration on the test bench.
Our Bosch Research Blog authors
Author: Daniel Neyer
Daniel is working as a Bosch Research engineer in the department “Advanced Vehicle Systems” on establishing new AI technologies within the electric powertrain domain. He joined Bosch in 2017 working on the electrification and automation of Off-Highway Machines in the construction sector. He currently leads the “Applied AI in eMobility” research activity with the aim to utilize the potential of modern hybrid AI models to solve current challenges within the powertrain of electric vehicles.
Author: Volker Imhof
Volker is working as a Bosch Research engineer in the department “Artificial Intelligence Research” in the group “Probabilistic Modeling and Validation”. In 1989 he joined Bosch Research as a computer scientist. He worked at three separate locations at Bosch on lots of projects with various business units. In his current work he uses gaussian process regression as probabilistic models. This model type is key to the modular "Safe Active Learning" Client that he implemented.
Author: Christoph Zimmer
Christoph is a Lead Research Scientist and Technical Lead Active Learning at Bosch Research.
As an interdisciplinary mathematician he designs mathematical algorithms that create impact in the domain. In his early career he worked together with systems biologists at Heidelberg and epidemiologists at Yale before joining the Bosch Center for Artificial Intelligence in 2017. Here, he combined his previous expertise in mathematical modeling, parameter estimation and experimental design with machine learning techniques to create innovation for Bosch.
Author: Matthias Kraenzler
Matthias is working as a Bosch Research engineer in the department for Advanced Vehicle Systems. He joined Bosch Research as a Ph.D. student in 2016. His research focuses on AI algorithms and databased methods with lead applications in the field of the electric powertrain. His responsibilities include the integration of databased methods like active learning to the test bench. Before joining the Smart powertrain team, he developed new powertrain concepts for electrified air mobility within Bosch Research.
