Current sensor data fusion architectures: Visteon’s approach

Sensor fusion is a critical requirement in creating an autonomous vehicle’s “brain,” ensuring it can make intelligent, accurate and timely decisions based on behaviors of other traffic participants. Sensor fusion takes the inputs of different sensors and sensor types and uses the combined information to perceive the environment more accurately. This results in better and safer decisions than independent systems can achieve.

An increasingly complex range of on-board smart sensors like cameras, radar, ultrasonic, infrared, LiDAR and connected sensors for V2X communications, Wi-Fi, 5G, GPS and telematics provides both real-time and rich data that the connected car must be able to process to create a picture of the environment around the car at any moment in time.

To achieve fully autonomous driving – SAE Level 4/5 – it is essential to make judicious use of this sensor data, which is only possible with multi-sensor data fusion. By fusing sensor data, the vehicle forms a more accurate and reliable view of its environment and will have intelligent situational awareness. Sensor data fusion acts as a human brain, providing the car with a complete and accurate picture of its surroundings. 

The techniques that surround multi-sensor data fusion make up a very big and complex topic. Data processing techniques that associate, aggregate and integrate data from different sources help the system to build knowledge about certain events and environments, which is not possible using individual sensors separately. 

There are many ways to fuse the sensor data, and multi-sensor data fusion can be classified in two categories: Homogeneous and heterogeneous sensor data fusion. In homogeneous sensor fusion, sensor data of the same type are fused together, whereas in heterogeneous sensor fusion, data from different sensors are fused together based on the time of arrival of data (synchronization of data based on time stamps).

Multi-sensor data fusion can be performed at four different processing levels, according to the stage at which the fusion takes place: signal, object, feature, or decision level.

Currently all automotive players, be they automakers or Tier 1 suppliers, are extensively using feature-level fusion and decision-level fusion for multi-sensor data fusion. The benefit of this approach is the simplification of the sensor data fusion system as the smart sensors provide the list of features the system needs to make decisions. Almost all Level 2 autonomous systems are now making use of this approach.

But to achieve a Level 4/5 autonomous system, feature level fusion is not sufficient due to its limited sense of environment and loss of contextual information. The application of artificial intelligence (AI) will also have limitations when working on individual sensor data to obtain a feature list because of inherent flaws in this fusion approach.

Visteon is currently working on a signal-based multi-sensor data fusion approach. In this centralized architecture, sensor data from different sources are fused using AI techniques at a raw signal level. The major advantage of this approach is the availability of complete environmental information to the system which, with the help of machine learning techniques, will make informed decisions right from the start of a journey and, subsequently, be able to safely direct the car.

The safe and successful introduction of autonomous driving depends on a robust solution to sensor data fusion – creating a “brain” for the car’s automated body that is able to process huge data sets, apply AI and rapidly transmit information to and from its environment. Visteon’s approach is based on integrated and centralized multi-sensor fusion technology that is built on fault-tolerant hardware and incorporates AI for the most accurate levels of object detection and classification. 

As a lead software engineer, Anshul is involved in the development of SmartCore™ and autonomous driving domain controller platforms. He is focused on self-driving car technologies and the effect of the Internet of Things on the auto industry. Anshul is based in Karlsruhe, Germany. 

The Hidden Value in Internships

By Joslyn R. Holmes, Human Resources Generalist

Gaining job experience through an internship should be seen as more than just a resume builder. It can often be the first step in your career. That is, if you choose your internship and invest in it wisely.

Internships can be competitive and for good reason. An internship allows you to become more familiar with the day-to-day knowledge, skills and requirements of your career focus and area. Particularly if this is your first internship, you’ll gain an understanding of the practical application of your education outside of the classroom.

One critical and sometimes overlooked benefit of an internship is the networking opportunities it creates. Being able to access, talk with, collaborate and work side-by-side with colleagues gives you a great opportunity to learn not only about the job itself, but also where your skills and career can take you.

If you feel strongly that your internship is a good fit for you – and your career path – internships are a natural way to get your foot in the door for future employment.

We’re told that Visteon’s internship program is valued because we focus on providing our interns hands-on career experience and growth. Engineering and other technical interns are given the opportunity to work on projects that are being developed for application in vehicles, such as infotainment and instrument clusters.

Part of the benefit of working on production projects is the chance to learn about initiatives and tools that may not be covered by your college curriculum. Different software, materials, applications and processes are all part of the knowledge interns can gain.

Real-world projects give Visteon interns visibility to more areas than some full-time employees – many times our interns have the opportunity to meet with our customers (automotive manufacturers) and get immediate feedback on their work. Our mentor program also provides access to senior managers to give greater opportunity for professional development outside of day-to-day responsibilities.

While it’s true that an internship can boost your resume – it does so much more. The benefits of an internship can be lifelong as you can establish relationships and develop good work habits that can continue throughout your career. Visteon takes pride in its internship program, and we work hard to ensure our interns are prepared to contribute to the growing field of vehicle electronics.

We welcome our 2017 summer interns. You are joining Visteon at an exciting time.  We hope that this is a valuable learning experience for you, and we are excited to incorporate your fresh ideas that will meet the changing needs of the automotive industry.

Learn more about Visteon’s internship program and tell us about yourself by visiting http://visteon.com/careers/.

Joslyn R. Holmes is a Human Resources professional proficient in multiple functional areas of Human Resources including: Talent Acquisition, Employee Relations and Training and Development. She holds a Master’s Degree and a Bachelor’s degree, both from Central Michigan University, in Human Resources Management. Joslyn brings over 10 years of in depth Human Resources knowledge to Visteon. She is based in Michigan at Grace Lake Corporate Center where she has responsibilities in Talent Acquisition and is currently the Intern Project Manager.  

Artificial Intelligence Emerges from Data Rooms to Help Drive Autonomous Cars

By Vijay Nadkarni

A century ago, many large businesses ran their operations with rooms full of skilled clerks rapidly entering figures into a comptometer, a type of mechanical calculator considered very efficient for the times. After many decades, the comptometer proved too limiting for a rapidly advancing marketplace and was replaced by teams of data entry clerks feeding powerful mainframes. Technology continued to accelerate to the point where the calculating and data entry power of an entire corps of workers could be managed by a laptop. Today, apps have generated brilliant machinery and pocket-sized communications devices, thanks to innovative programmers and billions of lines of code.

In considering the challenge of autonomous vehicles, however, yet another level of technology will be required. Conventional programming and computational approaches to problem-solving will be far outpaced by the speed and complexity needed for automated driving.

The programming approaches for autonomous driving that are currently getting the lion’s share of attention – high-speed cameras, LIDAR and ultrasonic sensors – are unable to incorporate all potential driving scenarios while staying up-to-the-minute with traffic conditions, weather, construction zones and other driving issues. There is an approach, however, that will allow cars and trucks to learn and respond quickly and accurately to their constantly changing surroundings. That approach involves artificial intelligence (AI).

Artificial intelligence allows the vehicle to analyze in real time the massive amounts of data – gigabytes per second – received by its cameras, LIDAR and other sensors to avoid objects and plan the vehicle’s path. Applying AI in an optimal manner involves using neural networks for object classification and reinforcement learning for path planning.

Consumers already are bringing AI into their vehicles via their smartphones. Voice-based search engines and in-car navigation depend on a level of AI from off-board servers, and more infotainment systems are integrating connected features from outside servers that use AI in the background.

To manage an autonomous vehicle, engineers will need to transform AI from its typical location in a room full of servers for computer and Internet access to a self-contained location in the vehicle that, for the most part, does not need to depend on outside data connections. They also will have to solve the challenge of AI’s huge demand for power, resulting in heat generation that must be dissipated, contributing to higher fuel consumption.

Another area that needs to be addressed is which type of microprocessor will prove most efficient. Should it be a central processing unit (CPU), a graphics processing unit (GPU) or an application-specific integrated circuit (ASIC)? Each has benefits and drawbacks in terms of power, performance and cost. This issue is complicated by the need to decide between a CPU, where all information is received and processed, and a decentralized system with several smaller processors. Both have value, depending on how the vehicle manufacturer wishes to establish the architecture.

To meet this demand, Visteon is developing a scalable autonomous driving solution applying AI – specifically neural networks and machine learning. This approach can support either centralized or decentralized processing and can greatly improve the accuracy of detecting and classifying objects in a vehicle’s path. This approach holds much promise for moving autonomous driving from a few real-world examples among a roomful of innovators to an everyday reality on our roads and highways.

Vijay Nadkarni is the global head of artificial intelligence and deep learning technology for Visteon's product lines, including autonomous driving/ADAS and infotainment. He is based in Santa Clara where he has management oversight of Visteon's Silicon Valley technology center. Vijay is a hands-on technology veteran whose current focus is machine learning, Cloud computing and mobile apps. Prior to joining Visteon, he founded Chalkzen, which developed a novel Cloud platform for vehicular safety.