Your Apple Watch is about to listen a little more (if you want it to). A beta rolling out this week gives Siri ears on the Series 12 and Ultra 4, with new features that can remember what was said in the last 15 seconds and summarize conversations throughout your day.
Faster Shazam and sound recognition for better accessibility
Audio Intelligence, as the new set of benefits is called, consists of four audio-related functions: music recognition, sound recognition, Siri Rewind and Siri Recap. The first two, available since launch, include a faster Shazam that can recognize songs without even opening the app.
I tested it in beta and noticed that it responds much faster, especially since I don’t even have to open the app to identify a song. Songs now automatically appear in my watch’s Smart Stack. This even includes the lullabies that I sing to my daughter at bedtime, although understandably, it could not identify them.
Sound recognition
Sound recognition can alert people who are deaf or hard of hearing to sounds like a crying baby, a barking dog or the doorbell.
Dillon Lopez/CNET
How to access: You must activate the function on your iPhone, under Settings > Accessibility > Sound and name recognition and turn it on. You can even choose which sounds you want them to identify.
In my test, it successfully identified my fake cough, doorbell and car horn. But then it spent the rest of the day gaslighting me, telling me it was a dog barking or a car whizzing when it clearly wasn’t, which made me question both my own reality and its listeners. But this may require further testing as I am still on the beta versions of all of these.
The newest features out now in beta are the most controversial, and I’ll be testing them more this week.
Siri Rewind: Like Live Photos for Audio Snippets
At any time during the day, Siri Rewind lets you double-click the Digital Crown to pull up a text snippet of what was said in the previous 15 seconds, much like the audio version of Live Photos (which lets you watch a few seconds of video before and after you press the shutter).
How to access the transcribed text? If you do nothing after seeing it on the screen, it will disappear. You can also ask Siri about it, or save it in the Siri app to see later on your phone. Old audio is continuously replaced with new audio and then deleted, so if you don’t trigger the feature to remember the last 15 seconds, it’s gone.
This is for unplanned moments when you need to review information without asking the speaker to repeat himself. Apple’s keynote example showed someone checking the specials menu at a restaurant after the waiter had already left the table. Whether you want to look away from the person speaking on your watch instead of just asking them to repeat what they said is more of a question of etiquette, but it could be helpful during large presentations.
Siri Recap: Automatically summarizes meetings and conversations
Siri Recap can listen in the background while activated and generate high-level summaries and key points of conversations throughout your day. This can be anything from a parent-teacher conference or work meeting to spur-of-the-moment conversations or brainstorming sessions with colleagues that you don’t go out of your way to record with voice memos.
How to access it: Similar to Siri Rewind, the notes from these recaps can either be saved in the Siri app and exported to the Notes or Journal app or deleted. They delete automatically after seven days if you don’t delete manually.
Who controls when Siri is listening?
Both features are opt-in, which means you have to go out of your way to enable them during setup for them to work.
When they’re on, Siri Rewind requires you to manually double-click the Digital Crown to activate it. Siri Recap is completely customizable.
You can leave it at any time and let Siri decide what counts as important conversations to recap, put it on a schedule based on important meetings for the day, or activate it manually via the new Siri Recap icon in the control center on your Apple Watch like you Focus Mode or Do Not Disturb.
If you forgot to turn it off, you’ll get a reminder in the morning telling you it’s on. You’ll also get reminders to check your recaps throughout the day and notifications when plans start or end, so you’ll be aware when it’s active.
And it won’t work for conferences or FaceTime if you have your AirPods or headphones in.
What about battery impact?
Even if you decide to leave all four on throughout the day, Apple says they won’t strain your battery. The Apple Watch Series 12 gets about a day and a half of use (in my real-world testing), while the Ultra 4 gets anywhere from two and a half to three days of use.
This is possible because of Apple’s new S11 chip and its dual architecture, in which one part handles background tasks without draining the more power-hungry main processor.
The audio intelligence feature does not affect battery life.
Vanessa Hand Orellana/CNET
The pressing privacy issues
The clock does not store audio. It processes raw audio in a hardware-isolated compartment on the S11 chip, which Apple says is also inaccessible, called the Secure Exclave.
Siri Recap also does not send a full transcript to the cloud. It condenses the text first, removes the full transcript and sends only the condensed version to Private Cloud Compute. It leaves financial data, authentication details, personal identifiers, language related to self-harm and government secrets.
The Information Pipeline for Siri Recap
- When activated, the watch detects when a conversation occurs.
- Audio goes into the S11 Secure Exclave for on-device processing and condensed text, removing filler words for a high-quality summary.
- Condensed text is encrypted and sent to Private Cloud Compute, where Apple says the data remains private.
- The recap is sent back to your iPhone and immediately deleted from Private Cloud Compute.
Apple makes a compelling case for how it protects privacy, with security and accessibility innovations at its core.
Still, there’s something about the watch on my wrist explicitly giving me the ability to eavesdrop on conversations that don’t sit right with me and many others. Especially when those conversations involve other people who may have no idea it’s happening.
There is a sound that alerts people when Rewind is active, but to my knowledge, there is no other warning when Siri resumes a conversation. Smart glasses like the Meta Ray-Bans have an indicator light when they’re recording video, although that’s not unexpected either, as there are ways around it.
Maybe I’ll change my mind once I can actually test it. Apple has certainly given me reasons to trust how it protects data. I’m just not yet convinced that this is the intelligence I want my watch to have.
The features will be rolled out in beta, with the update to iOS 27.3, initially in English, with additional languages planned later.
I’ll be testing in the coming weeks, so check back for more.
See full bio