Transit itineraries from a screenshot

Apple Maps plans a transit trip from its own data and offers you its own options. When I have already picked a trip on GO Transit's site, a specific 18:25 train and the 19:20 bus it connects to, there is no way to hand that to Maps. I end up flipping between a screenshot and the map.

Maps+ is a proof of concept of the missing step. It is a SwiftUI app styled after Maps with one addition: drop a screenshot of an itinerary on the map, or pick it from Photos, or paste it, and the trip fills in. For the screenshot in the video it produces a train from Union Station to Erindale at 18:25 with three stops in between, a walk to the bus loop, and the 21 bus to Square One at 19:20, arriving 19:33. One hour and eight minutes, the same as GO says.

It is a concept, with the limits that implies. It understands GO Transit's "Your Trip" layout and nothing else, the stations in the demo are pinned to coordinates I looked up, and nothing in it is live.

Apple's Vision framework reads the text on the device and gives back each line with its position on the page. I turned language correction off, because it "fixes" station names into dictionary words.

The position matters more than the text. GO's panel is two columns: stations and route names on the left, and on the right the words "Depart:" and "Arrive:" with a time under each. The parser treats each of those right-hand labels as an anchor and collects the station lines that sit beside it on the left. A station name can wrap onto a second line, so everything within the anchor's height is joined into one name.

The left-hand lines are sorted by what they look like. "MI - Milton GO" or "21 - Union Station" is a route: a short code, a dash, a destination, and a code that is a number means a bus. "Show 3 Stops in Between" gives the stop count. "Transfer" between two rides becomes a walking leg. A Depart followed by an Arrive closes a leg.

The bug that taught me the most was a single character. Vision read the wheelchair icon next to a station as the digit 5, so "Erindale GO Bus" arrived as "Erindale GO 5" and "Bus". The map search for that name landed in Brampton, and the transfer became a 271 minute walk. The fix was to clean each line of stray one-character tokens before joining lines, not after.

Each leg is drawn the way that kind of travel moves. The train runs station to station in the Milton line's stop order, which also places the three stops the screenshot only counted. The bus follows MapKit's driving directions between its two stops, so it sits on real streets, though not necessarily on the route the 21 takes. The walk uses MapKit's walking directions, and its length is where the "4 min" on the card comes from.

Two details took longer than the routes. A dashed line for the walk smeared into large blue blobs whenever the camera zoomed or tilted, so the walk is drawn as separate dots, one every 25 metres. And MapKit hides a stop's name whenever another marker overlaps it, which made "Erindale GO" blink each time the train arrived, so the app draws the stop names itself.

The trip card copies the transit card in Maps: total time, line badges, and a timeline of every leg with the stops expandable. Every time on it is the one from the screenshot, which is the point of importing a trip.

Tapping GO steps through the trip: board, ride, walk the transfer, board again, arrive. A train, bus, or walker icon travels along the current leg, and the stop dots fade as it passes over them so the icon never slides underneath one.

Most of the work here was making MapKit hold still. Redrawing the route behind the icon sixty times a second made the whole route flicker, so the line stays untouched and only the icon moves. MapKit also picks its own speed for an animated zoom and ignores the one you ask for: going from a single stop out to a whole leg took about a fifth of a second. The app now sets the camera itself on every frame, easing the zoom on a logarithmic scale over 1.6 seconds.

Slower moves exposed the next problem, map tiles loading on camera. While the screenshot is still being read, a second map hidden behind the visible one steps through every view GO mode will use, and the zoom levels between them, so those tiles are already loaded when the camera gets there.

The video is a screen recording of the app playing the whole flow by itself in the iOS simulator, placed inside a phone drawn in CSS and cut with Remotion. The edit is a JSON file listing which seconds of the recording to keep and where the captions go, so re-recording after a fix meant changing a few numbers and rendering again. I re-recorded it more than twenty times.