Four lessons from building a handwriting notes app for iPad

I've been building GraphNotes, a notes app for iPad, iPhone Duo, and Mac where handwriting, PDF highlights and typed notes share one library. Two rules shaped most of its technical decisions: The ink is the note. The app reads your handwriting so it can search it, but it never swaps your strokes for its guess at what they say. Every thought keeps its way back. A note written beside a highlighted passage stays linked to that exact passage, and links point at permanent IDs, never at titles. It's built with SwiftUI (and UIKit where touch handling needs it), PencilKit, SQLite (libSQL through a fork of GRDB) and CloudKit's CKSyncEngine. Here are four things that took me longer to get right than I expected, in case they save you some time. 1. Give every stroke an identity (PencilKit already has one) It's tempting to spot new strokes by comparing drawing.strokes.count. Undo, redo and the vector eraser all break that. What works is diffing the strokes of two states by PKStrokePath.creationDate, the wall-clock time the pen touched down. It survives undo, redo and serialization, and in my tests the pieces the eraser leaves behind keep the original's date, so one key names one gesture however many pieces it has become: let before = Dictionary(grouping: old.strokes, by: \.path.creationDate) let after = Dictionary(grouping: new.strokes, by: \.path.creationDate) let added = after.keys.filter { before[$0] == nil } let removed = before.keys.filter { after[$0] == nil } // A date in both whose pieces differ was partly erased. The same date gives you timestamps: creationDate plus a point's timeOffset is that point's absolute time. I used to record Date() in the canvas delegate, and it went stale as soon as someone undid a stroke and redid it. iOS 27 also gives strokes IDs. In my tests PKStrokePath.id survived the eraser's split and PKStroke.id didn't, so if you use them, key on the path's. 2. Reading handwriting with PKStrokeRecognizer iOS 27's PKStrokeRecognizer reads handwriting on device. GraphNotes uses it for search, and shows your strokes in the results, never the recognized text. Reading a drawing takes four lines: let recognizer = PKStrokeRecognizer() // one per drawing await recognizer.updateDrawing(drawing) let best = await recognizer.recognizedText(strokeIDs: nil) let candidates = await recognizer.indexableContent // e.g. "hello\nhelp" What I'd tell myself at the start: Use a fresh recognizer for each drawing. One shared recognizer fed different drawings logged "Invalid stroke for stroke provider" for every stroke of the previous drawing, and one reused after an erase returned nothing. indexableContent runs at low priority. It can come back nil while the person is writing. Ask again later rather than storing "nothing to read". SearchResult isn't Sendable. In Swift 6, calling search from another isolation fails even with @preconcurrency import. Map the results inside an extension before they leave: struct InkSearchHit: Sendable { let strokeIDs: Set let bounds: CGRect } extension PKStrokeRecognizer { func inkSearchHits(_ word: String) async -> [InkSearchHit] { await search(word).map { InkSearchHit(strokeIDs: $0.strokes, bounds: $0.bounds) } } } Store PKStrokeRecognizer.recognitionVersion beside the text, so you can read the ink again when a later OS reads handwriting differently. 3. CKSyncEngine hands you each page once CKSyncEngine fetches changes in pages and hands each page to your delegate once. handleEvent can't report a failure, there's no API to rewind, and the next stateUpdate already counts the page as fetched. Apple's documentation says to persist that state together with the changes fetched before it. So if your database transaction fails and you save the next state anyway, those records are gone for good. Three rules came out of that. Hold the state back until the page has landed. A page whose transaction fails stays in memory, with every later page queued behind it in order, and the engine's state is saved only once they've all landed. If the app quits first, the saved state is older than the lost page, so the engine fetches it again. func handleEvent(_ event: CKSyncEngine.Event, syncEngine: CKSyncEngine) async { switch event { case .fetchedRecordZoneChanges(let changes): // Lands now, or waits in memory, in order, with every page after it. await pages.land(Page(changes)) case .stateUpdate(let update): // Saved only while no fetched page is waiting; otherwise held. await pages.receive(update.stateSerialization) case .didFetchChanges: // Land whatever waits, then save the held state. await pages.finishFetch() default: break } } Don't let one record fail the page. A drawing this device can't decode, or one value a constraint refuses, used to fail the whole transaction. Now each record lands in its own savepoint, and a refused one is rolled back alone and parked to try again later. Let children wait for their parents. A block can arrive in an earlier page than the doc it belongs to. The engine won't deliver that block again unless it changes, so dropping it, as my first version did, loses it. It now waits in a table and lands as soon as its doc does. 4. Put the link's address in the fragment Every note, highlight and board has a link you can paste anywhere, and I wanted it to open the app when it's installed. The app has no backend and collects no data, so a link shouldn't tell a web server which note you opened. The trick is the URL fragment. A link looks like this: https://graphnotes.app/l/#doc?id=7C1F0E2A-… A browser never puts the fragment in its request, so the web server only ever sees /l/. The associated-domains file matches that path with a non-empty fragment and leaves out any URL with a query string, since a query would reach the server: { "applinks": { "details": [{ "appIDs": ["TEAMID.com.example.notes"], "components": [ { "/": "/l*", "?": "?*", "exclude": true }, { "/": "/l/", "#": "?*" } ] }] } } When the app is installed, iOS hands it the whole URL, and the app reads the fragment as its own graphnotes:// link. When it isn't, /l/ is a static page that explains what the link is. One catch: typed into Safari's address bar, a universal link stays in Safari, by Apple's design. Because each link carries a permanent ID, a link to a missing note never falls back to opening a different note that happens to share its title. Quick hits A few smaller things I'd want to know on day one: SwiftUI gestures can leave the Pencil alone. In iOS 27 a tap or drag can say which input it takes, for example DragGesture(minimumDistance: 6, inputKinds: [.directTouch, .indirectTouch, .pointer]), so a Pencil stroke across a view isn't also a drag. contextMenu takes no input kinds, though, so a resting Pencil can still open one. Palm rejection by contact size alone fails on an iPad Pro. My first rule called any contact with majorRadius >= 20 a palm, and it rejected every finger: fingertips there report more than that. What works is calling a touch a palm only when the Pencil is involved: any touch while the Pencil is on the glass (or lifted less than 0.6 s ago), or a broad one (majorRadius >= 40) while it hovers. One programmatic scroll per SwiftUI update. Two scroll requests through .scrollPosition in the same update made SwiftUI write them back and forth forever, at 100% CPU. Send the second on the next main-queue turn. ScrollGeometry.visibleRect includes the content insets. On iPad it runs under the sidebar, the navigation bar and the keyboard. Take the insets off for anything the person is meant to see. Apply .inspector before .navigationTitle and .toolbar. With the inspector applied after them, iPadOS 26 showed no navigation bar at all. Apple's SQLite and a file replaced by a rename. A connection reading a database stopped working (SQLITE_IOERR_VNODE) once another process renamed a new file over it, whatever the open flags, in any process that had used libdispatch. On Linux it kept reading the old file. Reading the file into memory with sqlite3_deserialize avoided it. Undo can live in SQLite. App-wide undo uses temporary triggers that log the inverse SQL of every change, the technique from SQLite's undo/redo page. Each command registers one action with the scene's UndoManager, so ⌘Z, the three-finger gesture and the Edit menu all drive the same stack. If you've shipped PencilKit or CKSyncEngine code, I'd like to hear what tripped you up, especially palm rejection and sync conflicts. I'll answer questions in the comments. I drafted this post with help from Claude, an AI assistant, working from my own project notes and code. I've checked the details and code samples and stand behind them.