Mercedes retraces historic first road trip with autonomous S-Class [w/video]
Thu, 19 Sep 2013
To prove to her husband, Karl Benz, that the future of transportation was in automobiles, Bertha Benz drove a little over 60 miles from Mannheim to Pforzheim, Germany, in the Benz Patent-Motorwagen – her husband's creation and widely considered the world's first automobile – 125 years ago. Much more recently, Mercedes-Benz engineers followed the same route in a specially equipped autonomous S-Class, becoming the first people to be driven by a car, which dealt with pedestrians, traffic lights, roundabouts and all sorts of potential road hazards, on rural and urban routes.
The S500 Intelligent Drive research vehicle, however, is not as special as one might think. The sensors, radar, cameras and maps services used to make it autonomous are production-based, mainly used for active and passive safety features and adaptive cruise control. In the autonomous research vehicle's case, there are more of them and they're more strategically placed around the car for comprehensive coverage of its surroundings. All of the systems would be for naught if it weren't for the complex algorithm developed to read their feedback and drive the car.
For those wondering which organizations Mercedes worked with to develop the algorithms and maps required to pull the self-driving experience together, it wasn't Massachusetts Institute of Technology (MIT) and Google, but their European counterparts, Karlsruhe Institute of Technology (KIT) and Nokia, with its HERE maps services tailored to the research vehicle's needs.
Watch the video below of the autonomous Mercedes making the road trip, and be sure to read the press release below, as well, which contains much more detailed information on the autonomous drive system. The gallery we included also has some diagrams to help explain how the Intelligent Drive system works.
Pioneering achievement: Autonomous long-distance drive in rural and urban traffic
Mercedes-Benz S-Class INTELLIGENT DRIVE drives autonomously in the tracks of Bertha Benz
Stuttgart. When it sent its S 500 INTELLIGENT DRIVE research vehicle along a historic route in August 2013, Mercedes-Benz became the first motor manufacturer to demonstrate the feasibility of autonomous driving on both interurban and urban routes. The route in question, covering the 100 kilometres or so from Mannheim to Pforzheim, retraced that taken by motoring pioneer Bertha Benz exactly 125 years ago when she boldly set off on the very first long-distance drive. In the heavy traffic of the 21st century the self-driving S-Class had to deal autonomously with a number of highly complex situations – traffic lights, roundabouts, pedestrians, cyclists and trams. It should be noted that this trailblazing success was not achieved using extremely expensive special technology, but with the aid of near production-standard technology, very similar to that already found in the new E and S-Class. The project thus marks a milestone along the way that leads from the self-propelled (automobile) to the self-driving (autonomous) vehicle.
In August 1888, Bertha Benz set off on her famous first long-distance automobile journey from Mannheim to Pforzheim. In doing so, the wife of Carl Benz demonstrated the suitability of the Benz patent motor car for everyday use and thus paved the way for the worldwide success of the automobile. Precisely 125 years later, in August 2013, Mercedes-Benz recorded a no less spectacular pioneering achievement following the same route. Developed on the basis of the new Mercedes-Benz S-Class, the S 500 INTELLIGENT DRIVE research vehicle autonomously covered the approximately 100 kilometres between Mannheim and Pforzheim. Yet, unlike Bertha Benz all those years ago, it did not have the road "all to itself", but had to negotiate dense traffic and complex traffic situations.
"This S-Class spells out where we're headed with "Intelligent Drive" and what tremendous potential there is in currently available technology," says Dr. Dieter Zetsche, Chief Executive Officer of Daimler AG and Head of Mercedes-Benz Cars. "Of course, it would have been a lot easier to take the autobahn for the autonomous drive from Mannheim to Pforzheim. But there was a special motivation for us to carry out this autonomous drive along this very route 125 years after Bertha Benz. After all, we wouldn't be Mercedes-Benz unless we set ourselves challenging goals and then went on to achieve them."
Autonomous driving with production-based sensors
The Mercedes-Benz S 500 INTELLIGENT DRIVE research vehicle was equipped with production-based sensors for the project. Based on a further development of the sensor technologies already in use in the new S-Class, the developers taught the technology platform to know where it is, what it sees and how to react autonomously. With the aid of its highly automated "Route Pilot", the vehicle is able to negotiate its own way through dense urban and rural traffic.
"For us, autonomous vehicles are an important step on the way to accident-free driving," says Zetsche. "They will bring greater comfort and safety for all road users. That's because autonomous vehicles also react when the driver is inattentive or fails to spot something. On top of that, they relieve the driver of tedious or difficult tasks while at the wheel."
"With our successful test drives following in the tracks of Bertha Benz, we have demonstrated that highly automated driving is possible without the luxury of specially closed-off sections of road and relatively straightforward traffic situations," says Professor Thomas Weber, member of the Board of Management of Daimler AG with responsibility for Group Research and Head of Mercedes-Benz Cars Development. "In line with the goal of the project, we have gained important insights into the direction in which we need to further develop our current systems in order to enable autonomous driving not just on motorways, but also in other traffic scenarios. Even we ourselves were quite surprised at just how far we got using our present-day sensor technology. But now we also know how much time and effort is needed to teach the vehicle how to react correctly in a host of traffic situations – because every part of the route was different," adds Weber. This experience will now be incorporated into the engineering of future vehicle generations to be equipped with such innovative, further-developed functions. The Head of Daimler's Research and Development stresses: "With the new S-Class, we are the first to drive autonomously during traffic jams. We also want to be the first to bring other autonomous functions in series production vehicles. You can expect that we will reach this goal within this decade."
Several levels of autonomous driving
The main advantages of autonomous driving are plain to see: it allows motorists to reach their destination quickly, safely and in a more relaxed frame of mind. Above all on routine journeys, in traffic jams, on crowded motorways with speed restrictions and at accident blackspots, an autonomous vehicle is capable of assisting the driver and relieving them of tedious routine tasks. However, the intention is not to deprive the driver of the experience and pleasure of doing the driving for themselves. "Our autonomous systems offer to assist and unburden the driver. Those who want to drive themselves are free to do so, and that won't change in future either," stresses Daimler development chief Weber. "It's clear, however, that autonomous driving will not come overnight, but will be realised in stages. With this drive, we've now taken another important step into the future."
A distinction is made between three levels of autonomous driving. These have been defined by a VDA working group in collaboration with the German Federal Highway Research Institute (BASt): partially, highly and fully automated.
- With partially automated driving, the driver must constantly monitor the automatic functions and must not pursue any non-driving-related activity.
- In the case of highly automated driving, the driver need not permanently monitor the system. In this case, non-driving-related activities are conceivable on a limited scale. The system recognises its limitations by itself and passes the driving function back to the driver with sufficient time to spare.
- With fully automated driving, the system is capable of autonomously coping with every situation; the driver need not monitor the system and can pursue non-driving-related activities. Equally, driverless driving is possible at this level.
Partially automated driving is already available to drivers of new Mercedes-Benz E and S-Class models: the new DISTRONIC PLUS with Steering Assist and Stop&Go Pilot is capable of steering the vehicle mainly autonomously through traffic jams. This system thus forms the core of "Mercedes Benz Intelligent Drive", the intelligent networking of all safety and comfort systems on the way to accident-free and, ultimately, autonomous driving.
The now successfully conducted autonomous test drives along the Bertha Benz route allowed the Daimler researchers to gather important information on the challenges that remain to be addressed on the way to highly and fully automated driving and what, for example, still needs to be done to enable a car to navigate safely in highly complex situations involving traffic lights, roundabouts, pedestrians and trams.
Initial road tests using technology platforms based on the E and S-Class
Unnoticed by the public, yet authorised by appropriate official exemptions and certificates from the TÜV (German Technical Inspection Authority), testing of the "Route Pilot" on the Bertha Benz route began in early 2012 with a total of three technology platforms based on the E and S-Class, which are equipped with all available active and passive safety systems.
These test vehicles employed only those sensor technologies that are already today used in similar form in Mercedes-Benz standard-production vehicles. This is because those technologies are already affordable and suitable for everyday use, which facilitates a possible transfer to subsequent standard-production models. However, improvements were made to the number and arrangement of the sensors in order to achieve comprehensive coverage of the vehicle's surroundings in every direction, and to obtain additional information on the area around the vehicle.
Based on these sensor data and determination of the vehicle's own position with reference to information from a digital map, an autonomously driving vehicle analyses the available free area for driving and plans its own route. The required algorithms were developed by the Mercedes-Benz research team in collaboration with the Institute for Measuring and Control Technology at the Karlsruhe Institute of Technology (KIT).
The specific technical modifications compared with the standard-production version of a Mercedes-Benz S-Class are as follows:
- The base width (distance between the eyes) of the stereo camera was increased to allow more-distant objects to be detected not only by the radar, but also by the camera.
- Two additional long-range radars were installed at the sides of the front bumpers to provide early detection of vehicles coming from the left or right at junctions. A further long-range radar monitors the traffic to the rear.
- Four short-range radars at the corners of the vehicle provide improved detection of the nearer surroundings and other road users.
- Traffic lights are monitored by a colour camera behind the windscreen with a 90-degree opening angle.
- Another camera looks towards the back through the rear window to locate the vehicle with reference to known environment features. These environment features were previously entered on a digital map. By comparing what has just been seen by the camera with what is stored on the map, the vehicle is able to locate its position with significantly greater accuracy than would be possible with GPS alone.
For the trip along the Bertha Benz route, Mercedes-Benz collaborated with KIT and HERE, a division of Nokia specialised in the production of digital maps and location-specific services, to produce a 3D digital map of the route between Mannheim and Pforzheim that was specifically adapted to the requirements of an autonomous vehicle. In addition to the road layout, this map – which must meet special requirements with regard to accuracy – includes information on the number and direction of traffic lanes and traffic signs as well as the positions of traffic lights. Digital maps of this kind are a key prerequisite for autonomous driving. Mercedes-Benz and HERE will therefore continue their collaboration in future with regard to the development of "intelligent" 3D digital maps for autonomous vehicles.
Route Pilot reacts to diverse traffic situations
The Route Pilot in the research vehicle is required to cope with a host of different challenges both on country roads and in urban traffic: roundabouts, obstructions in built-up areas with oncoming traffic, cyclists on the road, turn-off manoeuvres, variously parked vehicles, red traffic lights, "right before left" priority junctions, crossing pedestrians and trams.
The autonomously driving S-Class was monitored during the tests by specially trained safety drivers who, whenever the system made an incorrect decision, were able to intervene immediately and take over control of the vehicle. As real traffic is unpredictable – which means that no driving situation is exactly the same as an earlier one – a record was made each time it became necessary for the safety driver to take over control of the vehicle. This information was then evaluated by the development team, thus making it possible to extend the vehicle's repertoire of manoeuvres. This advances the development of the technology platform, enabling it to cope with more and more traffic situations.
The test drives along the 100-kilometre-long route deliver important information for further development of the technology and the product. "For example, it became apparent that the recognition of traffic light phases under different lighting conditions and the correct pairing of individual traffic lights with traffic lanes represents a major challenge," explains Prof. Ralf Herrtwich, head of driver assistance and suspension systems at Daimler Group Research and Advance Development, a role in which he initiated the autonomous driving project. "However, it is not our intention that the vehicle should master every situation on its own. If, for example, the road is blocked by a refuse collection vehicle, we certainly don't want the vehicle to automatically overtake it, especially as the vision of the vehicle's sensors is restricted in such a case. In such a situation, the vehicle passes control back to the driver."
For the company, the success of the autonomous road tests lies above all in having identified those areas on which the development team needs to concentrate in future. "We now know where we can make further improvements and refinements to the vehicle's repertoire of programmed manoeuvres, i.e. the situation-dependent control commands for steering, engine and brakes, such as how to autonomously negotiate a roundabout." A further challenge is to correctly locate the vehicle on the road, in order to determine, for example, precisely where a vehicle should stop at a junction while at the same time having a view of cross-traffic.
A particular challenge for autonomous vehicles is the way in which they communicate and interact with other road users. Coming to an agreement with an oncoming vehicle on who should proceed first around an obstruction is something that requires a very great deal of situational analysis. "Where a human driver might boldly move forward into a gap, our autonomous vehicle tends to adopt a more cautious approach," says Herrtwich. "This sometimes results in comical situations, such as when, having stopped at a zebra crossing, the vehicle gets waved through by the pedestrian – yet our car stoically continues to wait, because we failed to anticipate such politeness when we programmed the system."
To enable the developers to reconstruct the decisions made by the autonomous research vehicle in individual driving situations, the car makes recordings of all its sensor data. Images from the stereo camera alone generate 300 gigabytes of data every hour. Also in later standard operation, some of these data will continue to be stored. That's because if, for example, an autonomous vehicle is involved in an accident, this information will make it possible to establish what happened.
Challenges on the path to autonomous driving
Before the goal of highly and fully autonomous driving is achieved, the obstacles to be overcome will not be just of a technical nature. Many of the things that are already technically feasible are still not universally permitted.
For instance, international UN/ECE Regulation R 79 (steering systems) allows only corrective steering functions, but not automatic steering at speeds above 10 km/h. Under the Vienna Road Traffic Convention, which is relevant for EU law, the driver must be in constant control of their vehicle and be capable of intervening at all times. As autonomous vehicles were still out of the question at the time this convention was adopted, clarification is needed with regard to what this means for highly or fully automated vehicles. In some US states such as Nevada, there has already been such clarification, at least as far as the trial operation of autonomous vehicles is concerned. Another prerequisite for the transition from partially to highly automated systems is their acceptance in society. Just as when the automobile was originally invented, it will first of all be necessary to build up confidence in the technical capabilities of the systems. This is borne out by a recent study carried out by the Customer Research Centre at Mercedes-Benz involving around 100 test persons aged between 18 and 60. The initial scepticism of the study participants was almost entirely dispelled following an autonomous drive in the driving simulator. Even among those participants who were negatively disposed to begin with, there was a significant increase in acceptance after the drive in the simulator.
One way of ensuring that map data and route information is always kept up to date is to use "Car-to-X Communication". This could enable future vehicles to help each other to generate real-time maps, because, theoretically, every car is capable of recording the route it has driven and entering it in a database. Information on a red traffic light could be relayed from a waiting car to other road users. Alternatively, the traffic light itself could send a signal to nearby vehicles. Mercedes-Benz has been working for several years on communication between vehicles and between vehicles and their environment. This year, it is set to become the first manufacturer to bring "Car-to-X functions" onto the market.
PROMETHEUS - pioneering achievement on the way to autonomous driving
Mercedes-Benz's success on the Bertha Benz route is the latest result of years of research in the field of autonomous driving. An earlier milestone was the Daimler-Benz-initiated research project EUREKA PROMETHEUS ("Programme for European Traffic with Highest Efficiency and Unprecedented Safety"), which ran from 1986 and whose test vehicles made headlines when, in 1994, in normal traffic, they covered around 1000 kilometres, mainly autonomously, on a multi-lane motorway in the Paris region and then, in 1995, drove from Munich to Copenhagen. Consequently, almost 20 years ago, Mercedes-Benz demonstrated that automated driving on motorways, including lane-changing, overtaking and keeping a safe distance, is technically feasible.
One of the outcomes of the Prometheus project was DISTRONIC adaptive cruise control, which went into production in the S-Class in 1998. Based on DISTRONIC, Mercedes-Benz has developed a succession of assistance systems capable of detecting hazardous situations, warning the driver and, ever more frequently, also automatically intervening. The project also resulted in Speed Limit Assist, which went into series production in 2005. Continuous further advances in environment detection using stereo cameras, also first tested as part of PROMETHEUS, created the foundation for the "6D Vision" stereo camera, which has now been launched in the new E- and S-Class. Patented by Daimler, this technology makes it possible to anticipate the real-time movements of other nearby road users.
At a technical level, Prometheus and the Mercedes-Benz S 500 INTELLIGENT DRIVE are worlds apart. "Progress has been due above all to modern-day hardware and software, which have been the focus of targeted optimisation over the years," explains Mercedes-Benz development chief Weber. "Technical components in those days were much too big and much too expensive for standard use in automobiles. Also, they were not powerful or reliable enough. The situation today is a quite different one. Our modern systems can be installed in compact control units that, while exceptionally powerful, are still affordable. Because that's the only way in which the maximum possible number of customers can benefit from autonomous vehicle functions – and that's our ultimate goal."
Mercedes-Benz assistance systems with partially automated driving functions in standard-production vehicles
- DISTRONIC/DISTRONIC PLUS adaptive cruise control (1998/2005)Launched in 1998 and further developed in 2005 with improved radar sensors, adaptive cruise control automatically maintains a safe distance from the vehicle in front. It is capable of autonomous braking and acceleration.
- PRE-SAFE Brake (2006)Automatically brakes the vehicle if there is a risk of a rear-end collision (autonomous partial/emergency braking).
- Active Blind Spot Assist (2010)Detects whether there is a vehicle in the driver's blind spot and, by means of one-sided application of the brakes, can reduce the risk of collision from a change of lane.
- Active Lane Keeping Assist (2010)Networked with ESP. If the driver unintentionally crosses a continuous or discontinuous lane marking, Active Lane Keeping Assist can brake the wheels on the opposite side, thereby returning the vehicle to the original lane.
- Active Parking Assist (2010)Uses electromechanical direct steering to navigate the vehicle into a parking space.
- DISTRONIC Plus with Steering Assist and Stop&Go Pilot (2013)Helps the driver not only to maintain a desired distance from the vehicle in front, but also to stay in the centre of the lane. This makes it possible to autonomously follow vehicles in traffic tailbacks.
- BAS PLUS Brake Assist with Cross-Traffic Assist (2013)Capable of detecting cross-traffic and pedestrians and boosting the braking power applied by the driver.
By Damon Lowney