The 2020 iPhone SE is the first Apple smartphone to offer Portrait Mode photos created entirely with software techniques rather than hardware. That’s not true of any previous iOS device, not even the one you think it is.
Lux Optics, maker of Halide, examined how Apple’s new budget phone adds the bokeh effect to 2D pictures.
To be clear, Apple calls pictures with the subject in focus but the background blurry Portrait images. It’s also called the bokeh effect by professionals.
iPhone SE does 3D bokeh images entirely with software
The iPhone SE uses Single Image Monocular Depth Estimation to create Portrait mode images, according to developer Ben Sandofsky in a blog post for Lux Optics. Which is a really fancy way of saying the phone is simply guessing about which objects are near and which are far.
That’s why Apple limits the bokeh effect to only humans. The iOS handset uses software to determine which objects are in the foreground, and that’s easier if it only has to work with people because it knows what shape they have.
There are apps that will do the Portrait mode effect on pets, including Lux Optics’s own Halide. But, as Sandofsky points out in his blog post, sometimes that generates odd effects.
And weirdly blurred images are just one of the quirky side effects of Single Image Monocular Depth Estimation. You can also use it to take a 3D Portrait Mode image of a picture shown on a monitor.
Not the end of dual-lens cameras
The iPhone 11 series uses two camera lenses to create the bokeh effect, while the iPhone SE does it with only one. But Sandofsky doesn’t think single-lens cameras will replace multi-lens ones.
The iPhone SE has a limited version of Portrait Mode because it has to guess about depth information. A dual-lens camera has actually depth info through its stereoscopic vision. “We evolved two eyes for a reason,” points out Sandofsky.
iPhone XR did this differently
Some people reading this have been wondering why the 2020 iPhone SE is being called Apple’s first device that uses software to make 3D images out of 2D ones when the iPhone XR also had a single lens but could apply the bokeh effect.
But, as Sandofsky pointed out in a previous Lux Optics blog post, the XR used its camera’s focus pixels to get a rough estimation of depth. So the handset employed both hardware and software to make Bokeh images.
The new iPhone SE has the same camera as the iPhone 8, and it can’t pull this trick off. So it only uses software.
