Apple 2018 closed its $400 million acquisition of the music recognition app Shazam. Now, it’s bringing Shazam’s audio recognition capabilities to app developers in the form of the new ShazamKit. The new framework will allow app developers — including those on both Apple platforms and Android — to build apps that can identify music from Shazam’s massive database of songs or even from their custom catalog of pre-recorded audio.
Many consumers are already familiar with the mobile app Shazam, which lets you push a button to identify what song you’re hearing and then take other actions — like viewing the lyrics, adding the piece to a playlist, exploring music trends, and more. Having first launched in 2008, Shazam was already one of the oldest apps on the App Store when Apple snatched it up.
Now the company is putting Shazam to use better than being just a music identification utility. With the new ShazamKit, developers will now leverage Shazam’s audio recognition capabilities to create their own app experiences. There are three parts to the new framework: Shazam catalog recognition, which lets developers add song recognition to their apps; custom catalog recognition, which performs on-device matching against arbitrary audio; and library management.
You probably think of Shazam catalog recognition when you think of the Shazam experience today. The technology can recognize the song playing in the environment and then fetch the song’s metadata, like the title and artist. And it can identify where in the audio the match occurred. The ShazamKit API can also return other metadata like genre or album art.
Shazam doesn’t match the audio itself, to be precise, when matching music. Instead, it creates a lossy representation of it, called a signature, and matches against that. This method dramatically reduces the amount of data that needs to be sent over the network. Signatures also cannot be used to reconstruct the original audio, which protects user privacy.
The Shazam catalog comprises millions of songs and is hosted in the cloud and maintained by Apple. It’s regularly updated with new tracks as they become available. When customers use a developer’s third-party app for music recognition via ShazamKit, they may want to save the song in their library. This is found in the Shazam app if the user has it installed, or it can be accessed by long-pressing on the music recognition Control Center module. The library is also synced across devices.
Apple suggests that apps make their users aware that recognized songs will be saved to this library, as there’s no special permission required to write to the library. ShazamKit’s custom catalog recognition feature could be used to create synced activities or other second-screen experiences in apps by recognizing the developer’s audio, not that from the Shazam music catalog.
This could allow for educational apps where students follow along with a video lesson. Some portion of the lesson’s audio could prompt activity to begin in the student’s companion app. It could also enable mobile shopping experiences that popped up as you watched a favorite TV show. ShazamKit is in beta on iOS 15.0+, macOS 12.0+, Mac Catalyst 15.0+, tvOS 15.0+, and watchOS 8.0+. On Android, ShazamKitisf is an Android Archive (AAR) file and supports music and custom audio.
