Walk this way: How Oko uses AI to make road crossings more accessible – Discover

About is a testament to the power of simplicity.

The 2024 Apple Design Award winner for Inclusivity and the 2024 App Store Award winner for Cultural Impact uses artificial intelligence to help blind or low-vision people navigate pedestrian walkways by alerting them to the state of the signals — “Walk,” “Don’t Walk,” and the like — through haptic, audio, and visual feedback. The app instantly gives more confidence to its users. Its bare bones UI masks a powerful mix of visual and AI tools under the hood. And it’s a particularly impressive achievement for a team that had no iOS or Swift development experience prior to launch.

“The biggest feedback we get is: ‘It’s so easy, there’s nothing complicated about it,’ and that’s great to hear,” says Vincent Janssen, one of Aboutits three Belgium-based founders. “But we designed it that way because that’s what we knew how to do. It was just the right thing to do.”


ADA FACT SHEET

Oko's three co-founders, Vincent Janssen, Michiel Janssen, and Willem Van de Mierop, gather around a bank of monitors to review their app.

From left: Willem Van de Mierop, Michiel Janssen and Vincent Janssen are the three co-founders of About. The name of the app is “Eyes”.

About

  • Winner: inclusiveness
  • Team: AYES BV
  • Available at: iPhone
  • Team Size: 6
  • Previous Awards: 2024 App Store Award Winner for Cultural Impact; App Store Editors’ Choice

Learn more about Oko

Download Oko from the App Store

For Janssen and his co-founders, brother Michiel and longtime friend Willem Van de Mierop, Oko – the name translates to “eye” – was a passion project that emerged during the pandemic. All three studied computer science with a concentration in AI, and had worked for years in their hometown of Antwerp. But by early 2021, the trio felt restless. “We all had full-time jobs,” says Janssen, “but the weekends were pretty boring.” But they knew their experience couldn’t compare to that of a longtime friend with poor vision, who Janssen noticed felt more affected as the fall and winter months wore on.

“We really started to notice that he felt more isolated than others,” says Janssen. “Here in Belgium we are allowed to go for walks, but you have to be alone or with your household. That meant he couldn’t go with a volunteer or a guide. As AI engineers, that made us think: ‘Well, there are all these stories about autonomous vehicles. Can we come up with a similar system of images or videos that help people find their way around public spaces?’

I had opened Xcode maybe three times a few years before, but otherwise none of us had any iOS or Swift experience.

Vincent Janssen, About Founder

The trio began building a prototype that consisted of a microcomputer, 3D-printed materials, and a small portable speaker borrowed from the Janssen brothers’ father. Today Janssen calls it “hacky hardware”, something similar to a small computer with a camera. But it allowed the team and their friend—now their primary tester—to kick around the idea and explore the technology’s potential. Could AI recognize the state of a pedestrian signal? How far away could it detect a Don’t Walk sign? How would it perform in the rain or wind or snow? There was only one way to know. “We went out for long walks,” says Janssen.

And while the AI ​​and hardware have performed well in their road tests, issues have arisen around the size and usability of the hardware, and the team is starting to realize that software offers a better solution. The fact that none of the three had the slightest experience building iOS apps was simply a hurdle to clear. “I had opened Xcode maybe three times a few years before,” says Janssen, “but otherwise none of us had any iOS or Swift experience.”

Two screenshots of the app Oko. The screenshot on the left shows a suggested walk through a map view. The screenshot on the right shows a live image of a street intersection with the words

About helps people navigate pedestrian paths through interactive maps and audio, visual, and haptic feedback.

So that summer, the team turned to software, quit their full-time jobs, and threw themselves into learning Swift through tutorials, videos, and trusted web searches. The core idea quickly crystallized: Build a simple app that relies on the camera, the Maps SDK, and a powerful AI algorithm that could help people get around the city. “Today it’s a bit more complex, but in the beginning the app basically opened a camera feed and a Core ML model to process the images,” says Janssen, noting that the original model was taken over from Python. “Fortunately, the tools made the conversion really smooth.” (AboutThe AI ​​models run locally on the device.)

With the software taking shape, more field testing was needed. The team reached out to accessibility-oriented organizations across Belgium, drafting a team of 100 testers to “code the app,” says Janssen. Among the initial feedback: Although About was originally designed to be used in landscape mode, pretty much everyone prefers to keep their phones in portrait mode. “I had the same experience, to be honest,” Janssen said, “but that meant we had to redesign the whole thing.”

A group of men stand in a room with green walls looking at a large monitor that contains notes and sketches for the app *Oko*.

Those About Team navigating through prototypes at a review session in their hometown of Antwerp, Belgium.

Other changes include changing the audio feedback to more closely mimic existing real-world sounds, and addressing requests to add more visual feedback. The experience was a real world education about accessibility on the run. “We learned very quickly about VoiceOver and haptic feedback,” says Janssen.

Nevertheless, the project proceeded remarkably quickly – About launched on the App Store in December 2021, not even a year after the trio conceived it. “It took a while to do things like make sure the UI wasn’t blocked, especially since we didn’t fully understand the code we wrote in Swift,” laughs Janssen, “but in the end the app did what it needed to do.”

We learned about VoiceOver and haptic feedback.

Vincent Janssen, About Founder

The accessibility community took notice. And in the following months the About The team continued to expand their reach – Michiel Janssen and Van de Mierop traveled to the United States to meet with accessibility organizations and gain experience with American road traffic and pedestrian patterns. But even as the app expanded, the team kept its focus on simplicity. In fact, Janssen says, they explored and ultimately jettisoned several expansion ideas—including one designed to help people find and board public transportation—that made the app a bit too complex.

Today is the About Team numbers 6, including a fleet of developers who handle more advanced Swift stuff. “About a year after we launched, we had feedback about extra features and speed improvements, and needed to find people who were better at Swift than we were,” laughs Janssen. At the same time, the original trio is now learning about business, marketing and expansion.

At the core, About remains a shining example of a simple app that does its job well. “It’s still a work in progress, and we’re learning every day,” says Janssen. In other words, there are still many roads to cross.

Meet the 2024 Apple Design Award winners

Behind the Design is a series that explores the design practices and philosophies of the finalists and winners of the Apple Design Awards. In each story, we go behind the scenes with the developers and designers of these award-winning apps and games to discover how they brought their remarkable creations to life.

Leave a Comment