Let’s start from the beginning: what the Face ID system looks like and how it works:
In terms of hardware, this system consists of a pair of cameras – IR and RGB, and two types of IR emitters – regular illumination and a specialized dot projector.
Let’s strip down an iPhone X, remove everything unnecessary, and we see the front camera block in its bare form. The IR illumination exists separately, but all other components are in our hands — they are securely fixed in this metal frame.
Here we see the IR camera and the IR dot projector. The operation of the TrueDepth and FaceID systems relies on this infrared pair. And the main subject of today’s story is precisely the dot projector.
A Brief Overview of How FaceID Works
The IR dot projector does exactly what its name suggests — it spits out tens of thousands of infrared dots into the surrounding world on command. And the IR camera, which issues this command, immediately photographs these dots.
Knowing the optical characteristics of the projector, camera, and the distance between them, the ISP in the iPhone processor can estimate how far away each dot is. I don’t fully understand the mathematics of the process myself, and reverse-engineering it would be difficult — but capturing a series of images with different dot patterns allows for a very accurate guess of the position of each individual dot, and thus creating a complete depth map. Without LIDAR and without ToF.
It is the depth map that allows FaceID to not be fooled by photographs printed on a printer. A photograph is flat — but a face has relief, and for FaceID, the topographic map of the mug is more important than its coloring.
However, the system also looks at the coloring of the face. The IR camera is hardware-synchronized with the RGB camera, and both cameras capture the face simultaneously. And the iPhone can also blink not only with the IR projector, but also with IR illumination — and capture the entire face in the IR spectrum.
The depth map itself is quite rough, and analyzing the texture of the face in IR and RGB images using a neural network allows both to refine the depth and to better understand details like facial expressions — both in daylight and in the dark.
If this system seems familiar to you, it’s probably because it’s directly ripped from the Kinect for Xbox 360. Only that Kinect was a huge contraption, and here it has been shrunk to the size of a notch in a smartphone screen.
This was done by PrimeSense, the company that developed the technologies at the heart of Kinect 1 — and was later acquired by Apple for $350 million. In its entirety — with all patents, developments, employees, and other innards.
Dissecting the Guts of the Projector
Let’s delve deeper into the reverse engineering: we take out the dot projector from the iPhone X camera block and disassemble it into its component parts. It consists of an FPC flex cable, an emitting assembly, and an optical assembly.
The flex cable is completely passive, and therefore of little interest. It is soldered to the emitting assembly and outputs signals to the FPC connector, which connects to the iPhone X motherboard. The connector has a contact pitch of 0.35mm, is custom (Apple are bastards), and it looks like it’s made by JAE.
Let’s look at the main components of the optics:
And let’s see what’s inside the emitter:
And here the role of the MOSFET and the mysterious chip piqued my interest. Why? Because it’s unclear what they are even doing there.
The first obvious option is that the mysterious chip is memory for the serial number and calibration data. The chip has a typical I2C interface for memory, and there is definitely memory inside. Projectors have serial numbers, by which you can determine the production date, among other things — and if the projector is replaced entirely, the iPhone will see a serial number mismatch and refuse to work with the replacement. But the most common I2C EEPROM comes in a tiny WLCSP-4 package — and it can even be write-protected if you really want to. Therefore, the chip cannot just be simple memory. It’s definitely doing something else.
The second obvious option is that the mysterious chip is a laser driver, and the MOSFET is its switch. And yes, the MOSFET is indeed controlled by the chip. However, the chip also can’t be something as critically important as a laser driver.
Firstly, the MOSFET is in the common cathode break of the laser assembly — and 4 separate anodes go directly to the flex cable and further into the depths of the iPhone’s layered board. And secondly, in the process of collecting data for reverse engineering, I came across various instructions from Chinese repair technicians.
They didn’t directly clarify the essence of the issue, but many of these instructions said: to repair a “broken” projector, you need to disassemble it, remove the MOSFET, and replace it with a jumper between the drain and source. The projector will eventually work with a jumper inside, and FaceID functionality will be restored. And if the projector works fine with a jumper instead of a MOSFET, what was that MOSFET doing there?
And it dawned on me: that’s what the repair was all about. The MOSFET is controlled by the chip — so, at the chip’s will, it can break the laser power circuit, and thus break the projector. And this repair eliminates this break.
What’s in a Name?
Once it became clear that the mysterious chip in tandem with the MOSFET is hindering the normal operation of the projector, the question arises — why is it doing this? Why put a chip in the projector that kills the projector?
For answers, I delved into the firmware of the ISP block in the iPhone processor – it is precisely what communicates with the camera sensors and the projector via I2C.
First, I downloaded the iOS 15 firmware image for iPhone X, a fresh one. Firmware images for iPhones are essentially zip files. Inside, I found the ISP firmware I was looking for — in the form of a file \Firmware\isp_bni\adc-nike-d22.im4p. From the compressed im4p file, I extracted a binary, in Mach-O format with AArch64 code inside. Mach-O, unlike a typical “firmware image for an unknown microcontroller” — is a documented executable file format, similar to PE or ELF. No guessing about the file structure, processor architecture, or the address where the code needs to be loaded. Just drop the file into Ghidra, and everything falls into place by itself. Nice.
Then instinct took over, and I decided to rummage through older firmwares. And in the iOS 13 firmware image, I found the adc-nike-d22 file. Even the size was almost the same. But in the new firmware there was more code — and in the old one there was less code, but there were symbols. All function names were in place. Always check older versions!
There is a lot of information in the ISP firmware, including how the iPhone communicates via I2C with various chips — with camera sensors, with camera PMUs, with flash and autofocus control chips. From there, thanks to the symbols, it was possible to extract the “names” of various components of the system — and some of them correlate with materials from other parts of the firmware, as well as from other reverse engineers and repair technicians. For example, the IR camera sensor is the STMicroelectronics VD56G0 “Savage”. The entire TrueDepth system in the code is called “Pearl”, and its main modules are given the names of characters from “Romeo and Juliet”. The IR projector is called “Romeo”, the IR camera is “Juliet”, and the IR illumination is called “Rosaline”. The laser driver, which lives on the iPhone motherboard and powers both the lasers inside “Romeo” and the laser inside the “Rosaline” illuminator, is called “Rigel“.
The mysterious chip that interests us? It also has a name. In the code, it is called “MamaBear”, abbreviated “MB”, and it seems that its functionality is quite simple. It lives on the I2C bus. It stores OTP data, including serial number and various calibrations. It turns the MOSFET on and off on command. And it also measures… capacitance? Not temperature; it’s not connected to an NTC thermistor at all, but capacitance. But capacitance of what?
The Tragic Demise of Romeo
The answer to this question is again helped by Chinese schematics. In the schematic from JCID, it is visible that in the “Romeo” module there are three contacts for connecting the emitting assembly with the optical assembly. One is ground, and the other two go directly to the “MamaBear” chip. These contacts pass through a special adapter on the side of the optical assembly and end up on its very top — on the diffraction optical element.
The diffraction beam splitter is uncontrolled and does not react to current. But it has capacitance. And with the help of those three lines, this capacitance can be measured. But why?
The thing is, this diffraction splitter plays a very important role. The pattern of dots used by the projector is set by the location of tiny laser “pits” on the VCSEL crystal. And then this pattern is multiplied by the diffraction element, which makes hundreds of beams of light out of one beam.
So, what will happen if this diffraction element is torn off?
The beams will not be split. Instead of hundreds of laser beams, there will be one beam — but a hundred times more powerful. And it’s still a laser. An infrared laser is more dangerous than a red one because a person does not see it — and therefore will not instinctively look away even from a dangerously powerful light source. And there is a non-zero chance that the characteristic dot pattern will be burned into the user’s retina in this case.
This is why the kill-chip is needed. After switching on, it constantly monitors the capacitance of the diffraction element — and if the element is broken off or damaged, the capacitance goes beyond the permissible limits, and the chip immediately cuts off the MOSFET and breaks the power supply to the VCSEL. And since the element is located at the very top of the optical assembly, it is almost impossible to damage the rest of the assembly with a blow without breaking it and disrupting the contact.
After an emergency laser shutdown, the chip burns a flag into its OTP, which marks the projector as defective — which means that the broken power supply will remain broken forever. No commands from the ISP will have power over it anymore. The MOSFET will always be closed, and the projector will never work again.
The “MamaBear” chip, as the name hints – is a protection chip. It’s a “killswitch” for emergency laser shutdown. It kills the projector to prevent a damaged laser device from shining into the user’s eyes. And the “Juliet” module, left without its paired “Romeo”, loses its purpose in life — and the entire TrueDepth system becomes unusable.
The Daily Grind of Tech-Priests
But this protection scheme has a flaw. The fact is that the dot projector is located on the top edge of the device, and next to the speaker. If liquid gets inside the iPhone, one of the most frequent places for this is precisely there. And capacitive sensors are sensitive to electrically conductive liquids. Therefore, it often happens that FaceID breaks after the device falls into water — even if the water ingress is minimal, and there is no other damage. “Romeo” simply misunderstood the situation and prematurely pulled the plug, so to speak.
Such devices are taken for repair. Often to unofficial repair shops. And since the iPhone checks the serial numbers of parts (hello, Apple), you can’t just swap the entire camera block for a working block from a donor. The phone will reject the new block, and FaceID still won’t work. So, you need to somehow fix the old one. But how can you “resurrect” a projector that has intentionally disabled itself?
Manufacturers of unofficial repair tools have come up with a whole range of different rituals for this. And straight-handed tech-priest repairmen faithfully follow them, and perform microsurgery on this complex and calibrated optical system. Unimaginable dexterity is needed — the components inside are only a few millimeters in size, and the optics are extremely sensitive. If the calibration drifts too much due to surgical interventions, the system will not work. There are no tools for software recalibration (hello again, Apple) — you either find a way to get into the original parameters, or you’re left without FaceID.
How does it work? Well, first you need to read the OTP data from the original “MamaBear” chip.
The data is readable even if the projector considers itself faulty. To read the data, the Chinese make special “repair” programmers — which come with sets of connector adapters and work with a whole range of different components from different iPhone models, including projectors.
And then you need to do two things — deal with the MOSFET that breaks the power supply, and replace the original protection chip. And there are many different methods here.
You can, for example, throw a jumper instead of the MOSFET, as in the photo above in the article, and replace the “MamaBear” chip by desoldering the original FPC flex cable and replacing it with a special flex cable with a Chinese cheat chip.
The original “MamaBear” chip may remain inside, and helplessly shout that the projector should in no case work. But it no longer has a MOSFET to forcibly turn off the projector, and the iPhone, for its part, only sees the Chinese chip — which gives out a copy of the original data залитую by the programmer, and reports that the projector is definitely, definitely working.
Or you can tear out the “MamaBear” chip entirely, and put a Chinese two-in-one replacement in its regular place — it both closes the MOSFET contacts and gives a copy of the OTP data to the phone.
Well, there is also an option with a minimum of soldering. An “adapter” with a cheat chip, which is placed between the original flex cable and the iPhone motherboard.
It does not solve the MOSFET problem, but the Chinese have also found an original approach to it, making “high-voltage” programmers.
Do you know how various ATtinys can be “unbricked” and rewritten using a special high-voltage programmer? The situation here is completely different. The Chinese high-voltage programmer brutally and irreversibly “programs” the MOSFET inside the projector into a short circuit between the drain and source.
At the last stage of repair, we connect the projector to the programmer again and заливаем the dump saved at the first stage into it. And the projector is ready to work, passing itself off as original and unmodified.
All these different devices are made and promoted by different sellers of repair equipment. All sorts of cheat chips only work with “native” programmers, and programmers often have DRM features such as account binding and a limited number of “repairs”, for the replenishment of which you have to pay.
Do the repairmen know that with their repair they completely destroy the system invented by Apple to protect the user’s eyes? In fact, no. They are not reverse engineers — they are shamans. They have no understanding of the principles of operation. They have rituals and results, and that’s enough for them. And the shrewd reverse engineers from China are reluctant to share their secrets with the public. What I have described in this article is fully known only to Apple engineers and a dozen Chinese “in the know.” And to me. And to you, now.
Why Apple are Bastards
You know, I can’t blame Apple engineers too much for their “killswitch” being too active and breaking projectors that could still work. Lasers are a dangerous topic, and the idea of protecting the user from “worst-case scenarios” is absolutely sound. Although the implementation of this protection needs refinement.
But Apple’s policy of fighting unofficial repairs is the worst of all evils. If TrueDepth blocks could be freely swapped from device to device, without regard to serial numbers, then there would be practically no point in the horrific, twisted repair rituals. Why bother with microsurgical soldering and dance with programmers if you can remove a perfectly working TrueDepth block from another “donor” device with a broken screen, put it in the client’s phone, fully restore functionality, and live in peace? It would be easier for repairmen, and safer for device owners.
But Apple’s history of ugly anti-repair behavior clearly shows that this will not happen. Well, unless various “Right to Repair” movements in the USA or the EU make the binding of spare parts by serial numbers illegal. And this is now possible. In the joke that the European Union adds more useful features to new iPhone models than Apple, there is a very high proportion of truth. So, we will follow the legislative initiatives.
This article was translated from Russian. The author of the article is acc0unt. We have tried to preserve the author’s original style and phrasing as much as possible during the translation.