Android's 10 ms Problem? SOLVED.

Gabor Szanto

Follow @szantog

In a series of extremely popular articles we explained Android’s audio architecture problems, preventing entire categories of apps from being built on an otherwise great and incredibly popular platform. We updated our findings with the recent improvements in Android M (Marshmallow), and recently released a solution for Android’s USB audio and MIDI challenges, opening a market of 1.1 billion devices for pro audio application creators.

Our testing, technical analyses and audio latency measurement database of more than 4,238 different Android models/builds shows that Google has been making great progress in order to solve the Android round-trip audio latency problem, however progress seems to be slowing as the current media server internals are not likely to be hacked much further unless fundamental changes should happen. To date, we have seen no improvements with Android N with regards to audio latency.

We receive emails from all around the world, almost on a daily basis, where developers beg us for a solution to Android Audio’s 10 ms Problem. Which is why we’re proud to announce a solution to Android Audio 10ms Problem, which you can install and demo today.

Want to embed this image? Copy the following code:

Contents

• Android's 10 ms Problem? Solved in 8 ms.
• Common Misconceptions
• Superpowered Media Server for Android
• Technical Discussion
• How does it work?
• Install Demo
• FAQ

Definition of Low Round-Trip Latency Audio and Android Benchmarks

The useful definition of round-trip audio low latency is:

+ Audio Input (time)
+ Audio Processing (time)
+ Audio Output (time)
= 10 milliseconds or less.

This is the necessary highest bound latency for interactions in apps to be perceived as natural and immersive. Even a few milliseconds more latency will makes listeners and app-users uncomfortable (albeit they won’t be able to tell you why.) Readers will note that iOS has been low latency since the iPhone’s introduction in 2007.

As of June 2016, the best official Android audio latencies stand at:

Source

Common Misconceptions

Most misconceptions as to why Android doesn’t offer low latency center around the following three themes:

It’s a hardware problem.

It’s a Linux audio problem.

It’s the ALSA drivers.

Based on our research and development, while minor issues exist in each area, none of the aforementioned are insurmountable or fatal. The official Superpowered Inc opinion is they’re all completely acceptable.

Ultimately, the Android audio tack has fundamental architectural problems stemming from engineering decisions made at Android’s genesis, well before Google’s acquisition of Android.

Android Audio's sub-10 ms Solution: Superpowered Media Server for Android

For a comparison of Android integrated with the Superpowered Media Server for Android vs Android unaltered, audio latency is measured across three different audio paths.

1. The inbuilt speaker to inbuilt microphone. This would be ‘normal’ usage.
2. Audio output plugged into headphone jack and inbuilt microphone. This would be a headset plugged into the Android device.
3. Audio loopback dongle. A dongle plugged into headphone jack that simultaneously routes audio in and audio out.

Technical Discussion

The Superpowered Media Server for Android is injected between the Android Audio HAL (which handles the audio driver) and the Android media server (the system component handling audio).

It allows for applications to bypass the Android media server, meaning that apps benefit from:

• Significantly reduced round-trip latency.
• Reduced CPU load and thus improved battery life (the current Android audio stack has too high of CPU cost for such small buffer sizes).
• Improved stability (fewer dropouts) thanks to “pull” architecture.

The Superpowered Media Server does not affect audio system routing, system applications, notifications or any other Android audio feature. It is 100% compatible with existing Android audio functionality, and it has absolutely no effect on applications not designed to to interface with it.

In plain English, any app will work just fine on an Android build with Superpowered Media Server, whether the app decides to use the Superpowered Media Server audio path or not. Every feature of the Android media server will work without any change.

Lastly, if no application requests Superpowered audio access, Superpowered Media Server uses zero CPU overhead.

The Superpowered Media Server:

• Does not change the Android media server.
• Does not change the audio driver.
• Does not change the HAL. It uses the current HAL.
• Does not change buffer size or any other audio setting of the system.

How does it work?

It uses the HAL and connects to user applications in a novel way. The purpose of this demo is to demonstrate that:

• Fundamental Android audio architectural problems can be solved today.
• The existing audio driver and HAL architecture is sufficient on many Android devices.
• Existing audio hardware is sufficient on most Android devices.
• Round-trip latency can be significantly improved without changing the buffer size.

Paramountly, the new Superpowered Media Server audio path works as a ‘pull mechanism’, while the current Android media server is a ‘push mechanism’. Using pull, the audio driver dictates when the media server and the user application should produce the next buffer of audio, whereas in ‘push’, the media server or the user application shoves the next buffer of audio to the audio driver whether it is ready or not.

In push, the tail wags the dog.

Because pull reduces the risk for dropouts, and handles audio “in the rhythm of the audio device” is the standard for professional audio systems like Mac OSX.

Install

The installation procedure is relatively easy and doesn’t require a new Android build. It can be installed on any Android userdebug build or rooted production build.

The Superpowered Media Server demo currently supports:

• Devices with Android M. It can be implemented for Android L as well, if required.
• Devices with Qualcomm HAL or Nvidia Tegra HAL. It can be installed on other devices, but it will not activate itself.

Installing on Samsung Galaxy devices is not recommended, because the HAL/audio driver adds additional latency to support the audio shortcut for the Samsung Professional Audio SDK.

The package contains:

• Installation instructions (readme.txt).
• The binary of the server, which is just 43 kb (yes, kilobytes).
• Modified version of the Superpowered Latency Measurement app.
• Modified source code of the Superpowered Latency Measurement app.
• Modified version of Google’s Dr. Rick O’Rang latency and audio glitch measurement app.
• Modified source code of the Rick O’Rang app.

The modified latency and audio glitch measurement apps are prepared to use Superpowered Media Server audio path, if that’s available.

To request access, please email mediaserver@superpowered.com.

FAQ

Is this a rip-and-replace solution?

No. The Superpowered Media Server is embedded into Android service space and AudioFlinger is left untouched. This allows for normal function, trivial integration, seamless operation, backwards-compatibility, and increased performance.

Does the Superpowered Media Server trade audio quality for speed?

No. Superpowered Media Server is exactly as glitch and drop-out free as Android’s AudioFlinger while providing super-powered sub-10ms performance.

How does the Superpowered Media Server fare in resisting kernel scheduling anomalies which are common in Android kernels?

The Superpowered Media Server uses exactly the same thread scheduling policy as the server thread of AudioFlinger and the OpenSL ES client audio processing thread.
Meaning it is identically resistant to kernel scheduling anomalies as the current solution.

How does the Superpowered Media Server handle sleeping?

The Superpowered media server uses the pull method, and unlike AudioFlinger, it never sleeps between each turn. AudioFlinger has no true connection to ALSA buffer events (such as incoming audio or available output buffer), because it’s designed with a “push” method.

The Superpowered media server thread is unblocked immediately on an ALSA buffer event, and together with the appropriate thread scheduling settings, it provides a maximum processing time for the client to process and/or provide audio.

This approach reduces the risk of dropouts, and is used exclusively in real-time pro audio systems, such as Core Audio.

Do OEM partners need to modify device HALs or device hardware?

No modification is needed for device HAL or hardware.

How does Superpowered Media Server affect other device audio functions?

It does not affect other audio functions such as system sounds. The device functions as before.

How can get it?

To request access, please email mediaserver@superpowered.com.

Superpowered. Who are we?

We squeeze audio latency out of app user-space with +150m app-installs and thousands of apps using the Superpowered Audio SDK on Android and iOS.

Now, we’re squeezing audio latency out of Android service space with Superpowered Media Server.