Project 02 / Factotum / In progress

An app that builds its own features.

I’m building Factotum, an iOS app — iPhone and iPad — where I describe a task and Claude and Codex build the automation — or a small app for it — that uses the permissions I grant.

Why I’m making it

  • I want one agent on my iPhone and iPad that builds the automation I ask for, instead of hunting for an app that already does it.
  • Most tools ship a fixed feature set. I want the feature written on demand when I describe what I need.

What I’m building

  • A prompt box: describe a task, and the agent builds an automation or a small app for it.
  • Those small apps run inside the host app — generated web mini-apps, WeChat-style — with access to the iOS permissions I grant.
  • Every step that changes, sends, or leaves the device is shown and waits for my tap before it runs.
  • A saved library of the mini-apps and automations it has built, to re-run later.

How I’m building it

  • A SwiftUI host app with a WKWebView runtime that loads a generated mini-app and a bridge from its JavaScript to native capabilities.
  • Capabilities for HTTP, Shortcuts, local notifications, clipboard, sharing and per-app storage, each gated by an iOS permission prompt.
  • A backend worker that runs Claude and Codex: Claude plans what to build, Codex writes the mini-app; the result is sent back and loaded.
  • Installed on my own device through sideloading, so it can use the permissions without going through App Review.

Decisions I’ve made

  • Generated apps are web, never native code — iOS forbids running downloaded native code, and web keeps them inside a sandbox the bridge controls.
  • Side effects are confirmed and the HTTP capability only reaches hosts I allow-list; a generated mini-app can’t exceed the bridge.
  • Each user brings their own API key or runs their own backend. I never proxy someone’s Claude or ChatGPT subscription through infrastructure I operate — providers ban that.
  • Scoped to a technical audience who can stand up their own backend, rather than automating account creation, which iOS can’t do and providers block.

The honest limits

  • iOS sandboxes apps, so this can’t drive other apps directly the way a Chinese super-app does. Its reach into other apps is Shortcuts, App Intents and URL schemes — powerful, but mediated, and rougher by design.
  • Some permissions (Health, push notifications) need a paid developer account, so they’re out of the first version.

What I’m building next

  • The generation loop end to end: prompt, watch it build, and run the new mini-app.
  • More capabilities — location, calendar, contacts, camera and speech.
  • Exposing saved automations as App Intents so Siri and the Shortcuts app can trigger them.