Anan iPhone Use

Client-side iPhone Use.

Anan works right on your iPhone, with a small USB device for taps and typing. No computer, simulator, or external screen mirroring needed during use.

Open source on GitHubConnection guide
A note, from thought to action.
9:41
‹ Notes•••

Today, 9:41 AM

A little reminder

Start writing…
Your iPhone
AnanRunning on this iPhone

“Write a little reminder to take a walk.”

Native observationphone.observe
{}
01Screen → Anan → your model
02Anan → MCP → StopWatch
03USB input → iPhone
M5STACK
MCPUSB connectedReadyControl enabled
■ Stop control
M5Stack StopWatch · ESP32-S3

Input only. No model. No screenshots.

One USB-C cable · local Ethernet + mouse / keyboard input

You share your screen. Anan sends the image to your chosen model to understand what is visible.

HTML illustration · sample data, no device connection or real actions

01 / Before you start

What you need.

The agent runs in Anan on your iPhone. It reads the screen, asks your chosen model what to do, and sends actions to the connected USB device. The device acts like a mouse and keyboard. Once set up, you can use it without a computer or simulator, and without mirroring your screen to another device.

An iPhone

USB-C, portrait, unlocked. Anan’s screen observation currently requires iOS 27+.

M5Stack StopWatch

The current ESP32-S3 target, connected with a USB-C data cable. Other boards require adaptation.

Anan + your model

An image-capable model, your API key, and an internet connection for model requests.

Currently built for M5Stack StopWatch, connected by USB-C, with the iPhone in portrait. A computer is needed only to build and flash the firmware for the initial setup.

02 / Connect & set up

Connect it and get started.

1

Prepare the device

Use a computer for the initial firmware build and flash. The project targets ESP-IDF v5.5.5 and the StopWatch’s 16 MB ESP32-S3. With an authorized source checkout, install the pinned toolchain and source its export.sh before running:

./build.sh
shasum -a 256 build/anan-stopwatch-flash.bin
./flash.sh /dev/cu.usbmodemXXXX

Compare the image hash with the build manifest. Replace the placeholder with your actual serial port and enter the board’s download mode before flashing. The merged image is written at 0x0, not the app offset. Flashing replaces factory firmware; there is no published release binary yet.

2

Connect your iPhone

Connect the device to your unlocked iPhone with a data cable. Configure the USB Ethernet connection with the values below, then enable Settings → Accessibility → Touch → AssistiveTouch. Use a US hardware keyboard layout.

iPhone IPv4   192.168.7.2
Subnet mask   255.255.255.0
Router        —
DNS           —

Leave router and DNS empty. This USB link is a local connection, not an internet gateway; keep Wi-Fi or cellular available for your model provider.

3

Pair and import MCP

On StopWatch, open Settings → Connect MCP app. In iPhone Safari, open http://192.168.7.1 and start pairing. Compare the six-digit code, then confirm on the device. The setup window lasts ten minutes.

Copy or download the generated JSON. In Anan, open Me → Extensions → MCP, choose Import JSON, select the service and import it. Allow local-network access when requested. Use the exported configuration so its per-device Bearer header is included.

http://192.168.7.1/mcp

The default wired setup uses HTTP and requires no certificate installation. HTTPS is optional and has separate certificate/trust steps. Keep the exported JSON private; it contains a credential. Do not use the retired direct TCP connection.

4

Share the screen. Enable input.

Choose an image-capable model in an ordinary Anan chat. When Anan needs phone.observe, it asks you to prepare system screen sharing; choose the whole display, not only the Anan app. Keep the phone in portrait.

After importing MCP, return to the device and tap Start control. Screen-sharing consent, Anan’s tool approval and the hardware input grant are separate. Discovering tools or reading device_status does not enable input.

5

Try a small task

Open an empty test note and ask Anan to type the sample below. Start with observation, allow one small action, then observe the actual result. The current keyboard tool supports ASCII, not direct Chinese or other Unicode input.

In this test note, type "Take a little walk."
Observe the screen before and after each action.

Stop from the device’s stop control, either physical front button, or MCP stop; disconnecting USB also revokes input. Reconnect and explicitly enable control before continuing. Stopping does not undo previous actions.

03 / MCP

Six tools. One hardware connection.

The board provides these six tools. phone.observe belongs to Anan, not the firmware. Coordinates are normalized to the portrait screen: (0, 0) is the top-left, (1, 1) is the bottom-right.

ToolArgumentsWhat it does
device_status—Read USB connection, mode and control state. No screen capture.
tapx, y · 0…1Send an absolute mouse click at portrait coordinates.
dragx, y, toX, toY · 0…1 durationMs · 100…2000Send a bounded drag or swipe.
keykeycode · 4…231 modifiers · 0…255Send one USB keyboard press and release.
type_asciitext · 1…120 bytesType US-layout ASCII, including TAB and LF.
stop—Interrupt input and revoke the device control grant.

Each input action is bounded to eight seconds. Control permission ends on stop, disconnect or reboot; it is not restored by reconnecting. The first input session owns the device grant.

04 / Good to know

How it works.

Where does the AI run?

Anan orchestrates the task on your iPhone. Inference happens at your configured model provider, which receives the screenshots and context included in requests. StopWatch runs neither a model nor a screenshot service. Your model credentials are not stored on the board.

Is this built into iOS?

No. Observation uses user-consented system screen sharing; input comes from an attached USB mouse/keyboard device through AssistiveTouch. Anan’s native phone tool only observes. Hardware input is exposed separately through MCP.

Can another agent use it?

The device exposes six Streamable HTTP MCP tools. Another client needs its own screen-observation source and a network path to the iPhone’s USB subnet. MCP support alone is not enough. Desktop/cloud client interoperability remains to be validated; Wi-Fi and BLE are not implemented.

What happens if a command times out?

Input may already have partly reached the phone. Stop and inspect the screen rather than automatically repeating the command. Tool completion confirms input was sent, not that a target was reached. Background execution and protected screen content are also subject to iOS restrictions.

05 / Troubleshooting

If something feels stuck.

No USB Ethernet

Check the data cable, exit download mode, reset the board and verify the 16 MB firmware image.

MCP cannot connect

Check the manual IP/mask, local-network permission, /mcp endpoint and exported Bearer header.

No usable screen image

Unlock the phone, keep portrait and choose whole-display sharing. Protected content may be unavailable.

Input does not work

Enable AssistiveTouch and Start control on the device. Check device_status before trying again.

Text is wrong or incomplete

Use the US keyboard layout and ASCII. Inspect partial effects before reconnecting; do not automatically replay.

Still need a hand? Contact support