A $7 Board That Drives Your iPhone

A $7 ESP32 lets a vision model drive a real iPhone. No jailbreak, no dev account.

ESP32 iPhone Agent lets an AI model operate a real iPhone using an ESP32-S3 board that costs about $7. iOS doesn't give you an easy way for one program to tap through other apps. This project gets around that with hardware. Once it's set up you don't need a computer, a jailbreak or a developer account to run it. You type a goal into the app, press Start, and the model takes it from there.

How it works

The board does two jobs. First, it is the eyes. It runs its own Wi-Fi hotspot and acts as an AirPlay receiver. You mirror the iPhone screen to it, and it passes the H.264 stream back to the app over BLE. Second, it is the hands. Over USB it shows up as a mouse and keyboard. With AssistiveTouch on, clicks become taps and drags become swipes.

The brain is the iPhone app itself. It decodes the stream and sends screenshots to any OpenAI-compatible vision model. A planner and an actor then decide what to do next. So the phone watches itself through a $7 loop and pokes itself with a fake mouse. I like that a lot. You can stop a run from the Dynamic Island, and you can replay and export every run afterwards.

Try it

arduino-cli lib install "NimBLE-Arduino@2.5.1"
hardware/flash.sh            # optional: [serial port] [--debug]

cp ios/Config/Local.xcconfig.example ios/Config/Local.xcconfig   # set DEVELOPMENT_TEAM and IA_BUNDLE_ID
open ios/IPhoneAgent.xcodeproj

You need a Mac with Xcode one time, to flash the board and build the app, and the README says to always flash with flash.sh because it sets the BLE buffer sizes.

My take

The approach is clever and honest about itself. Hardware is cheap. It was tested on a Seeed Studio XIAO ESP32S3, the 8 MB PSRAM version. The phone needs iOS 18 or later. You can use any vision endpoint you like, and your key is stored in the Keychain.

The limits are real, though. BLE caps the mirrored screen at 480x1040 and 5 fps, so each step takes a few seconds. iOS hides passcode, Apple Pay and DRM screens, and mirroring stops when the phone locks. The board's hotspot takes over your Wi-Fi, so the phone needs cellular data. Typing is ASCII only, and landscape apps don't work. With a free Apple ID you have to re-sign the app every 7 days. The app also uses a private iOS API, so it won't get into the App Store as it is.

Who should care: people working on mobile agents who want to test against real apps on a real phone, and anyone who enjoys ESP32 hacks. It's MIT licensed, except for the AirPlay code ported from UxPlay, which is GPL-3.0. One more thing the README stresses: only automate devices and accounts you're allowed to use.