Driver assistance, in particular, is undergoing rapid innovation as system designers aim to maximise driver assist features within their price, power and space constraints. As the name suggests, driver assistance comprises a class of safety features that provide the driver with useful information in advance, in order to avoid potential accidents. Driver assistance includes features such as reversing cameras, surround view cameras, lane departure warning systems and pedestrian detection.
The value of the rear view camera during reversing manoeuvres is huge. Such devices are simple imaging systems with a camera sensor fitted at the rear of the automobile and a lcd in the driver cabin. More sophisticated systems may offer enhanced night vision. Surround view camera systems are an extension of the rear view camera solution, but are used all the time during vehicle operation and can often replace traditional side mirrors. The primary value of surround view systems is to provide constant coverage of what may otherwise be blind spots. In addition, the removal of external side mirrors makes the vehicle more aerodynamic and therefore more fuel efficient.
The presence of such hardware sensors can be viewed as a first and necessary step to eventually deliver more value added features that can be realised by analysing the data generated by these sensors. For example, the video image data generated by the backup and surround view systems can be passed through real time image processing algorithms to detect pedestrians and common road side objects, such as fire hydrants.
While these systems are currently being designed primarily to provide driver assistance, from a technology perspective, there is nothing to prevent these systems being expanded to create an intelligent proactive control system with the capability to bring the vehicle to a hard stop to prevent a collision. Similarly, lane departure warning systems use image data to detect the surroundings. This data is then processed in real time to detect if the car is within the limits of its own lane. If the car inadvertently sways too close to the lane's boundaries, an audible alarm is activated to alert the driver.
A common theme among several of the driver assistance systems discussed above is the use of video as the main input source. Incoming video streams need to be processed in real time to create actionable inputs to the driver. This requires high performance digital signal processing, along with flexible custom logic. Lattice's recently announced AEC-Q100 qualified LA-LatticeECP3 FPGAs are a suitable choice for such systems. With up to 1.3Mbit of embedded ram, up to 64 18x18 multipliers, four 3.2Gbit/s serdes channels and up to 35,000 look up tables (LUTs), the LA-LatticeECP3 family is suited for automotive applications that require low power, high speed and small form factors at the right price. Lattice's LA-LatticeECP3 automotive qualified devices enable affordable innovation for automotive solutions.
Sensor extender and mirror replacement
In automotive applications, it is typical for the image sensor to be located remotely, not only from the image signal processor but also the screen on which the image is to be displayed. This is not only true for the rear view back up camera, but also surround view camera applications. LA-LatticeECP3's programmable I/Os and serdes provide automotive OEMs with the flexibility to choose from multiple sensor, display and connection protocols.
Table 1 captures some of the common connection, sensor and display protocols that are supported by the device family. Figure 1, meanwhile, shows Lattice's reference design for extending the image sensor from the image signal processor (ISP) chip.
The Lattice solution has been proven to deliver 720p60 or 1080p30 video over distances of up to 8m with well matched pairs. At the receiving end, the LA-LatticeECP3 device maybe used to deserialise the incoming data, implement the complete image signal processing algorithm and drive the display – all in one device. The reference design uses a standard CAT5E cable to provide the sensor extension solution. In addition, the Lattice reference design solution can remotely deliver power to the image sensor.
After the raw data from the image sensor is brought to the desired location, the data needs to be processed before it can be used. This is often referred to as 'image signal processing'. Lattice has a suite of ISP pipeline IP cores, such as defective pixel correction, de-bayer, colour correction matrix and Gamma correction that allows designers to implement an end to end camera solution, all in one chip.
Figure 2 illustrates the ISP cores that have been ported to the LA-LatticeECP3 family of devices. In addition, the Lattice HDR-60 video camera development kit comes preloaded with a plug and play evaluation ISP pipeline. The kit enables rapid evaluation and prototyping of high definition HDR video camera solutions.
After market video black box recorder
Video black box recorders are class of after market automotive device that has recently emerged. These devices allow activity both inside and outside of the vehicle to be recorded. Some of these units will also support the connection of multiple camera sensors, providing a full 360° panoramic recording capability. LA-LatticeECP3 fpgas allow the entire camera ISP, compression and memory interface solution to be realised using a single chip. Figure 3 shows an example of how a video black box recorder system could be implemented on an LA-LatticeECP3 fpga. The range of standards supported by LA-LatticeECP3's I/O buffers offers the system designer the flexibility to interface with a wide range of sensors, memory, and display panels.
Summary
The proliferation of electronics in automobiles will only increase. Just as safety systems such as airbags have become mandatory features in most vehicles in most countries, the next class of safety features – such as those discussed in this article – are also set to become mainstream. Lattice fpgas are ideal for OEMs who are looking for the flexibility, quick time to market and low power. Given the long product life of an automobile, fpgas provide OEMs the flexibility to upgrade their system and respond to emerging market needs, quickly and cost effectively.
Subra Chandramouli is senior marketing manager with Lattice Semiconductor.
FPGAs enable innovative and cost effective automotive camera solutions
4 mins read
The increased use of semiconductor products in automobiles has been more pronounced in recent years. For example, it is now fairly common to find tyre pressure monitoring systems, even in entry level compact cars, a feature that used to be available only in some high end luxury cars until few years ago. The primary applications driving increased semiconductors use are driver assistance, instrument cluster, infotainment, telematics and in-vehicle networking.
