Mystery vans likely making 3-D road maps for Apple’s self-driving car

By

apple van
Mysterious unmarked vans roaming the Bay Area have been linked to Apple, and are likely generating detailed 3D maps for robot cars.
Photo: Business Insider/Stephen Smith

Some new data-gathering vehicles are roaming the streets of San Francisco. They’re unmarked, but are suspected to be Apple’s. They are laden with sensors, but what kind of data are they gathering, and what for?

Experts contacted by Cult of Mac say the mystery vans are next-generation mapping vehicles capable of capturing VR-style, 360-degree street photos. Plus, the vans use Lidar to create extraordinarily precise “point clouds,” a prerequisite for self-driving cars. Mesh those two databases together and you’ve laid the groundwork for an autonomous vehicle’s navigation system.

During a drive across the Golden Gate Bridge last weekend, a Business Insider reporter spotted an unmarked Ford van. In April, an identical van was spotted near Apple office buildings in Sunnyvale, Calif. by a Tech Radar reporter. (Note: The vehicles look the same but the plates are different.)

The vans look similar to Apple’s mapping minivans but are a different make (Ford versus Chrysler) and have a different configuration of sensors. And unlike the mapping vehicles, the new ones are unmarked.

It’s an “open secret” in Silicon Valley that Apple is working on a car. It’s likely to be electric like Tesla’s, and may be autonomous. Apple’s so-called Project Titan automotive initiative appears to be quite advanced, employing up to 600 staffers and moving beyond the prototype stage and into the early stages of production. Apple hasn’t confirmed anything, of course, but CEO Tim Cook recently offered a juicy non-denial when asked directly.

While the newly spotted Ford vans are almost certainly not prototype autonomous vehicles, they do appear to be gathering data that will be essential to an autonomous driving project.

It’s not an autonomous vehicle

The wheel encoder and GPS keep track of the vehicle's movements and provide "ground truth" to the maps being generated by other sensors.
The wheel encoder and GPS keep track of the vehicle’s movements and provide “ground truth” to the maps being generated by other sensors.
Photo: Business Insider/Stephen Smith

Some have speculated the Ford vans could be a prototype self-driving vehicle, given that Lidar on the roof and other sensors resemble the self-driving vehicles from Google and others.

However, Ryan Eustice, associate professor at the University of Michigan’s Perceptual Robotics Lab, said the combination of sensors on the vans points to a mapping vehicle. The vehicle’s cameras, Velodyne Lidar sensors, wheel encoder and GPS all point to mapping.

Paul Godsmark, chief technology officer of the Canadian Automated Vehicles Centre of Excellence, said the same thing. And self-driving car expert Brad Templeton, who runs the Robocar blog, also agreed. “It’s a mapping/street-scanning car, not a self-driving one.”

Ultra-precise Lidar maps are a prerequisite for autonomous vehicles

The four Velodyne Lidar sensors mounted on each corner are likely generating a "point cloud' -- an ultra precise 3D scan of the road -- that'll be used to navigate self-driving cars.
The four Velodyne Lidar sensors mounted on each corner are likely generating a “point cloud, an ultra-precise 3-D scan of the road that will be used to navigate self-driving cars.
Photo: Business Insider/Stephen Smith

Even though the vans are almost undoubtedly mere mapping vehicles, the configuration of rooftop equipment is different from Apple’s Maps minivans. Apple’s mapping minivans have four cameras on the corners and two Lidar sensors at the front and back. The new vehicles have four Lidar sensors on the corners and the cameras have been moved from the corners to the sides. Additional cameras pointing upward have been added.

According to Godsmark, the dominance of the Lidar units over the cameras is telling. “There is a good chance that the purpose of the mapping is related to autonomous vehicles in some way,” he said. “The way the Lidar are angled at each corner means that they provide an accurate view of the ground up close to the vehicle.”

Lidar (LIght Detection And Ranging) is a form of laser scanning. It’s similar to radar (RAdio Detection And Ranging) but uses light instead of radio waves to detect objects and their distances. Made by Velodyne, the Lidar sensors on each corner of the Ford van measure precisely the road and its surroundings, including curbs, drainage channels, even potholes — to within millimeters.

The Lidar is likely being used to create “point clouds” — ultra-precise 3-D scans of the road and its surroundings — that are assembled into “prior maps.” A prior map is a rich, 3-D map of the road that’s used by an autonomous vehicle to know precisely where it is at any time. Using onboard Lidar, an autonomous vehicle compares what it’s detecting in real time to a prior map loaded into memory. Point cloud prior maps are so accurate, autonomous vehicles know their road position with millimeter precision. Such data is “key for an autonomous vehicle to navigate safely,” said Godsmark.

This visualization from Lidar data shows what a 3D point cloud prior map might look like.
This visualization from Lidar data shows what a 3-D point cloud prior map might look like.
Photo: Alex Kushleyev

360-degree cameras for virtual reality street view

The array of cameras are capturing a 360-degree, hemispherical view of the road. Like virtual reality, the viewer will be able to look all around them and even upwards.
The array of cameras captures a 360-degree, hemispherical view of the road. As in virtual reality, the viewer will be able to look all around them and even upward.
Photo: Business Insider/Stephen Smith

The cameras, on the other hand, appear to be capturing an immersive, virtual-reality-style view of the road. The vans have an array of eight or more cameras positioned to capture a 360-degree hemispherical view of the road.

The vans have six cameras pointing outward — one at the front, one at the back, and two on the sides; and two more cameras angled toward the sky.

The camera mounted on front is likely a fisheye.
The camera mounted on front is likely a fisheye.
Photo: Business Insider/Stephen Smith

According to Godsmark, each camera likely has 60-to-120-degree field of view, and are aligned at 60-degree angles to each other. When stitched together, the cameras will provide “a 360 degree/hemispherical type virtual reality view of the world,” he said.

Godsmark noted that the vans couldn’t use a single virtual reality unit — like the ones used to capture VR movies — as the Lidar units on the van’s corners would obstruct the view. “This arrangement is an efficient way of providing a 360-degree view around the car and allowing the Lidar to achieve 360-degree coverage too.”

This is the camera mounted on the side of the vehicle. It has a pair of cameras aimed sideways (pointed across each other) and a third camera aimed upwards.
This is the camera mounted on the side of the vehicle. It has a pair of cameras aimed sideways (pointed across each other) and a third camera aimed upward.
Photo: Business Insider/Stephen Smith

The combination of Lidar data and 360-degree street views resemble mapping cars from other autonomous driving companies. For example, the test car from Uber’s Advanced Technologies Center in Pittsburgh combines Lidar and cameras; as does a fleet of vehicles from Bosch/TomTom that are currently mapping freeways in Germany.

The Bosch/TomTom vehicles are making multilayered maps for automated driving. They combine Lidar with 360-degree hemispherical cameras, which are clustered atop a pole on top of the vehicles. The multilayered maps meld ultra-precise 3D road information with visual cues about things like lane markers, traffic signs and speed limits.

A fleet of vehicles from Bosch/TomTom are creating multilayered maps for autonomous driving.
A fleet of vehicles from Bosch/TomTom are creating multilayered maps for autonomous driving.
Photo: Bosch/TomTom

Whose vehicle is it?

Super-secretive Apple won’t say if it’s the company behind the recently spotted Ford vans, and the vehicle could belong to any of the numerous Silicon Valley companies exploring self-driving cars.

However, when the Apple Maps minivans sparked feverish speculation, Apple began marking the vans with “Apple Maps” and “maps.apple.com” stickers. The company also posted a web page about the vehicles, with information about their purpose and a list of where they are operating. (The list has’t been updated in quite a while.)

Godsmark said he had no idea “whatsoever” what organization is operating the vehicles.

“I do find the lack of license plate interesting,” he said, “which reminded me of this old article of yours: ‘Why Steve Jobs’ Mercedes never had a license plate.'”