Amazon’s iPhone app uses computer vision to identify and replace screws and other parts

By

Amazon's new Part Finder feature for iPhone uses computer vision to identify and buy hundreds of different types of fasteners.
Amazon's new Part Finder feature for iPhone uses computer vision to identify and buy hundreds of different types of fasteners.
Photo: Sheila Sund via Flickr

Until recently, if you needed to replace a screw, nut or bolt, you’d take a trip to the hardware store. With Amazon’s latest update to its iPhone app, there’s no need.

Part Finder is a new feature of Amazon’s camera search that can identify hundreds of types of screws, nuts, bolts, washers and other fasteners. Once ID’d, they can be bought straight from the app.

This means thousands of parts that you can identify and replace without leaving the house. So Part Finder could mean another score in Amazon’s war against the brick-and-mortar world order.

To use the feature, open the Amazon app and then select the camera. From there, a screw icon indicates the Part Finder feature. Find a flat white surface — if a pristine desktop isn’t available, a sheet of paper should work fine. Next, place the part in question next to a penny (finally, they’re useful again). Align the part precisely with the reticule that appears, and in seconds you should get an ID, with options for details like head or driver type.

To use Part Finder, you need a flat white surface, a penny, and some patience.
To use Part Finder, you need a flat white surface, a penny, and some patience.
Photo: Screenshot

Point-and-shoot

The computer vision capabilities of the iPhone allow for precise measurements of a part’s width and depth. Considering the range of possible fastener types, and similarities in dimensions, the ability to precisely pick the right one is pretty impressive. Well, it would be if it worked reliably.

We tried a half dozen times to identify a screw, and couldn’t seem to get the right result. That might be due to inadequate lightning, or stark shadows. It could also be that we were searching for a particularly rare screw. Perhaps people with better cameras and trying it in better conditions will have better luck. Amazon will get it right eventually — machine vision is constantly improving by leaps and bounds.

It’s actually been a feature within the app for weeks, but Amazon didn’t acknowledge it until yesterday. Other visual product search features like barcode scanning, image-based product searches, and AR view have been around for a while. They all use the iPhone camera as a buying aid. This particular feature seems to be driven by technology developed by Partpic, an Atlanta-based startup that Amazon acquired in 2016.

As cameras and machine vision continue to advance, these tools will get better and better, meaning tasks and questions that used to require expert help are going the way of the shopping mall.