Through this, automobile manufacturers will be able to ensure that autonomous systems are fully prepared, with the capacity to appropriately respond to the broad array of potential circumstances that may arise. Though simulations can provide a large proportion of the data required, they must be supplemented by data that comes from extensive road-based testing. With the data volumes involved escalating, the execution of such work becomes an ever more difficult prospect - as this article discusses.
A huge amount of virtual simulation will be applied to autonomous function testing, but relying solely on this will not be enough. There will be scenarios that were not actually considered when the simulator was constructed, and it is vital that these are identified if potential risks of accidents occurring are to be adequately safeguarded against. Such ‘unknown unknowns’ will only be found through conducting many hours of real-world testing.
After completion of initial test activities, where data coming from a small number of vehicles on a track has been compiled, testing then moves to open roads. Here a much larger fleet of vehicles will be utilised (this could easily be over a hundred vehicles in total). These vehicles will be deployed to various locations around the globe, to get driving data relating to different types of environment and road conditions.
As well as the technical challenges associated with accurately acquiring all this data, there are also various logistical obstacles to overcome. These will relate to how such data is stored and subsequently shared, ahead of in-depth analysis being done. The size of the problem is only set to increase, as the quantities of test data needing to be captured is currently ramping up at an exponential rate.
The advanced driver assistance system (ADAS) technology incorporated into vehicles previously only took data from a relatively limited collection of external sources. These would generally be a couple of cameras and some additional sensor devices. As vehicle designs head towards the higher levels of autonomous functionality (3, 4 and 5), the number of sources where data is being generated is not only expanding substantially, but also the levels of data streaming in from each source is orders of magnitude greater too.
Rather than just having a few cameras installed, the next generation of vehicles may feature as many as 15-20 cameras. This is not exclusively about luxury cars, either, as it will be applicable to mid-range models too. They are also likely to include multiple radars, inertial measurement units (IMUs), ultrasound, plus LiDAR and time-of-flight (ToF) 3D imaging hardware.
Data management
For arrangements where there are two or three high-resolution cameras and a single radar, the data being captured would probably come to less than 100MB/s. This would mean that all the data from eight-hour-long test-driving runs would still fit onto a 2TB USB memory stick - pretty convenient for test drivers.
Having completed a test run during the day, they could take the memory stick and back-up all the captured data on their laptop. After that, they just had to mail the memory stick to the vehicle manufacturer’s development centre, so that it could be examined.
This approach may have worked well in the past, but it simply will not scale in line with future data volume expectations. The automotive industry has to accept that manual backing up of laptops and sending memory sticks in the post is no longer feasible. A radical overhauling of data management strategies is therefore required, and this needs to be supported by specialist service providers.
The numbers game
To illustrate the points just raised, it is worth looking at the actual numbers involved. Instead of having to cope with 100MB/s data acquisition rates, the higher autonomy levels of vehicles that are already emerging will mandate 5-8GB/s capabilities. Consequently, an 8-hour test run will see somewhere in the region of 150TB to 230TB of data being produced. This goes far beyond the hardware requirements that came with the old approach of sending back memory sticks.
For example, the solid-state drive (SSD) built into the average laptop is unlikely to support anything above 3GB/s read/write speeds. In terms of actual capacity, even the high-end ones will only reach figures of 12TB. Furthermore, the fast Internet connections in urban environments will not offer much more than 100MB/s of upload speed, and a lot of the testing work will be undertaken in remote rural locations, where Internet connectivity performance is markedly lower.
If an automotive OEM has 100 road test vehicles in operation with each capturing data at a rate of 5GB/s, then the total amount of new data needing to be managed on a daily basis will be in the region of 14PB. How to successfully handle the transferring of such colossal quantities, given the computer hardware and connectivity constraints, will clearly be problematic logistically speaking.
Data logging considerations
In order to capture multiple GB/s of data from their vehicles, automotive OEMs need to procure equipment that is capable of delivering elevated performance parameters. In addition to attaining high-speed acquisition benchmarks, the scope to attend to a wide selection of sensor interfaces (like GMSL, FPD-Link, GigE and USB) and in-vehicle networking protocols (Automotive Ethernet, FlexRay, CAN/FD, LIN, etc.) will also be called for.
It is, likewise, essential that all of the different data streams are timestamped with pinpoint accuracy, so that they are fully synchronised with one another. Otherwise, data mismatches could occur that might confuse the artificial intelligence (AI) algorithms of the vehicle’s computing system, resulting in an ill-judged response which may lead to an outcome where fatalities are witnessed.
There must also be flexibility when it comes to assigning a clock master, so that any other instrumentation within the system can be properly aligned.
The data logger has to be versatile enough that it may be employed in all manner of different test configurations. Every automotive OEM will obviously follow its own structural layouts, but even within a single organisation there may be several development projects under way, and each of these could have specific nuances (with respect to sensing modalities, number of sensor devices being used, etc.). Hence, a platform arrangement, where different modules can be accommodated in accordance with requirements, will be invaluable.
To minimise storage volume, data compression will be needed. Different types of data will require particular compression methodologies (MDF4, KITTI, TDMS, PCAP, etc.). For this reason, use of programmable logic within the data logging equipment will be advisable - so that there is inherent system agility and compression is always dealt with in the most appropriate way.
Capable of attaining 6GB/s throughput, NI has developed the Data Record AD data logger, which is proving highly effective in the road testing of vehicles.
Based on PXI technology, the relevant hardware modules and software can be chosen based on which sensors are required in the test set-up. These can be swapped in/out, as test requirements change over time.
Sophisticated FPGA technology also allows for lossless data compression IP to be leveraged. CAN, LIN, Flexray, 1000BASE-T1 Automotive Ethernet, and 100/1000BASE-T Ethernet are among the interfaces that are supported.
Above: The numerous elements comprised within an autonomous vehicle validation workflow - with inclusion of road testing being paramount
The data storage aspect
Test drivers evidently need a repository in which all of the acquired road test data can be compiled and then expediently passed back to where it can be properly examined.
By complementing Data Record AD acquisition hardware with technical partner Seagate’s Lyve Mobile SSD-based data storage units, the technical and organisational challenges that come with modern highly data intensive road testing can be alleviated.
Lyve Mobile units offer storage capacities of up to 120TB. To mitigate potential security breaches, they have 256-bit AES hardware encryption and key management capabilities. Through the NI/Seagate data acquisition and data storage combination, it is possible to eliminate the logjams that having to rely on available communication network infrastructure present.
After carrying out a test run, drivers need only swap the full storage unit for an empty one. They can then arrange for full ones to be picked up by Seagate’s logistic services, for subsequent analysis by engineers back in the automobile manufacturer’s development centre facility. The compiled data can be kept at an on-prem data centre. Alternatively, Seagate’s own data centre resources can be made use of (which thereby avoids large-scale investment in server equipment).
Conclusion
To ensure that autonomous vehicles attain the absolute maximum degrees of safety, it is critical that enough data is acquired and analysed to deal with all prospective situations. To be able to fully validate the autonomous capabilities of their latest vehicles, OEMs need to rely on a mixture of simulation-generated and road test data.
To support the progression of autonomous driving technology, NI and its technology partners, which includes the likes of Seagate, VSI Labs and Konrad Technologies, have worked together on the deployment of a dedicated fleet of test vehicles.
This fleet spans across Europe, North America and Asia, delivering a fully effective end-to-end connected workflow.
Author details: Daniel Riedelbauch, Chief Solutions Marketing Manager, ADAS & AD, NI
