MP3 Decoding on Android

We're implementing a program for Android phones that plays audio streamed from the internet. Here's approximately what we do:

1. Download a custom encrypted format.
2. Decrypt to get chunks of regular MP3 data.
3. Decode MP3 data to raw PCM data in a memory buffer.
4. Pipe the raw PCM data to an AudioTrack

Our target devices so far are Droid and Nexus One. Everything works great on Nexus One, but the MP3 decode is too slow on Droid. The audio playback starts to skip if we put the Droid under load. We are not permitted to decode the MP3 data to SD card, but I know that's not our problem anyways.

We didn't write our own MP3 decoder, but used MPADEC (http://sourceforge.net/projects/mpadec/). It's free and was easy to integrate with our program. We开发者_JAVA技巧 compile it with the NDK.

After exhaustive analysis with various profiling tools, we're convinced that it's this decoder that is falling behind.

Here's the options we're thinking about:

1. Find another MP3 decoder that we can compile with the Android NDK. This MP3 decoder would have to be either optimized to run on mobile ARM devices or maybe use integer-only math or some other optimizations to increase performance.

2. Since the built-in Android MediaPlayer service will take URLs, we might be able to implement a tiny HTTP server in our program and serve the MediaPlayer with the decrypted MP3s. That way we can take advantage of the built-in MP3 decoder.

3. Get access to the built-in MP3 decoder through the NDK. I don't know if this is possible.

Does anyone have any suggestions on what we can do to speed up our MP3 decoding?

-- Rob Sz

The right way to do this is to build your own firmware and do the decryption as part of a custom OpenCORE codec. That, of course, will limit you to devices where you can install that firmware.

Bear in mind that the rest of this is somewhat speculative. I actually have a need to do something similar to what you're describing, but I don't have time set aside to tackle the problem for a couple of months. So, I'll describe this in the fashion of how I'd approach the problem.

One solution is the one described in twk's answer. You don't have to use the SD card, but you probably do have to have a world-readable temporary file in your app-local file store (getFilesDir()). Download the first chunk, decrypt it, write it out as a complete world-readable MP3 file (but with a suitably obscure directory/path), and hand it to a MediaPlayer via setDataSource(). While that plays, you download/decrypt and set up a second MediaPlayer instance, which starts playback as soon as the first one ends, for as seamless a transition as possible. You then reset the first MediaPlayer and reuse it with your third chunk, ping-ponging between the two.

A related solution would be in jleedev's comment. It's pretty much the same thing, except that you provide a FileDescriptor via a ContentProvider. This has an option to let you use a socket, which may let you avoid the temporary file. However, the ContentProvider will have to itself be publicly accessible, and so the temporary file, with an obscure directory, might actually be more private.

If you are worried about the fact that these things can be read by other processes, please understand that MediaPlayer itself (or, rather, the OpenCORE subsystem) is in another process. Also, your proposed HTTP server is world-readable on the device as well. So, security by obscurity is your only viable option if you are going to let MediaPlayer do the decoding.

AFAIK, the NDK does not grant access to OpenCORE, though I confess as to having limited NDK experience, so I may be wrong. There are certainly other MP3 decoders available (ffmpeg/mplayer, etc.), though how readily these could be converted into an NDK library is unclear.

So, it really boils down to who you're trying to defend against. If you're trying to defend against the user, you're probably going to have to decode it yourself, somehow.

MAD is an integer-arithmetic-only decoder library which seems to be well regarded.

(Wouldn't decoding the data onto SD card slow you down anyway?)

I haven't tried this, but I believe you can give the media player a file descriptor:

public void setDataSource (FileDescriptor fd, long offset, long length)

Maybe you could write the decrypted mp3 out to a file and play it using the file descriptor. This might even work for streaming assuming you know the file length ahead of time. If it works, it would be a lot simpler than doing a localhost webserver.

To decrypte an encrypted mp3 file before play it. There are 3 ways:

1,For use MediaPlayer api:

After Android version M:

you can play mp3 data in byte array directly by:

//Read encrypted mp3:
byte[] bytesAudioData =  Utils.readFile(path); // or from a url.

final byte[] bytesAudioDataDecrypted = YourUtils.decryptFileToteArray(bytesAudioData);

                ByteArrayMediaDataSource bds = new ByteArrayMediaDataSource(bytesAudioDataDecrypted) ;
                mediaPlayer.setDataSource(bds);

Before android Version M:

//Read encrypted mp3:
byte[] bytesAudioData =  Utils.readFile(path); // or from a url.

final byte[] bytesAudioDataDecrypted = YourUtils.decryptFileToteArray(bytesAudioData);

 File file =new File(dirofyouraudio.mp3");

                FileOutputStream fos = new FileOutputStream(file);
                fos.write(bytesAudioDataDecrypted);
                Log.d(TAG, "playSound :: file write to temp :: System. nanoTime() ="+System. nanoTime());
                fos.close();

                FileInputStream fis = new FileInputStream(file);
                mediaPlayer.setDataSource(fis.getFD());

2,For AudioTrack api if you use:

  int minBufferSize = AudioTrack.getMinBufferSize(sampleRate,
            AudioFormat.CHANNEL_OUT_MONO,
            AudioFormat.ENCODING_PCM_16BIT);

   AudioTrack track = new AudioTrack(AudioManager.STREAM_MUSIC, sampleRate,
            AudioFormat.CHANNEL_OUT_MONO,
            AudioFormat.ENCODING_PCM_16BIT, minBufferSize,
            AudioTrack.MODE_STREAM);

    int i = 0;
    byte[] tempMinBufferToRead = new byte[minBufferSize];

    try {

        byte[] bytesAudioData = Utils.readFile( lastRecordedAudiofilePath);

        bytesAudioData = yourUtils.decryptYourFileToByteArray(bytesAudioData);

        try {
            fileInputStremForPlay = new FileInputStream( lastRecordedAudiofilePath);

            dataInputStreamForPlay = new DataInputStream(new ByteArrayInputStream(bytesAudioData));

            track.play();
            while ((i = dataInputStreamForPlay.read(tempMinBufferToRead, 0, minBufferSize)) > -1) {
                Log.d(LOG_TAG, "mThreadExitFlag= " + mThreadExitFlag);

                if ((mThreadExitFlag == true) || (!audioFilePathInThisThread.equals(lastRecordedAudiofilePath))) {

                    m_handler.post(new Runnable() {
                        public void run() {
                            Log.d(LOG_TAG, "startPlaying: break and reset play button ");

                            startPlayBtnInToolbar.setEnabled(true);
                            stopPlayBtnInToolbar.setEnabled(false);
                        }
                    });

                    break;
                }

                track.write(tempMinBufferToRead, 0, i);
            }

            Log.d(LOG_TAG, "===== Playing Audio Completed ===== ");
            track.stop();

            track.release();

            dataInputStreamForPlay.close();

            fileInputStremForPlay.close();


        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

3, Use MediaExtractor , MediaCodec with AudioTrack , you can set datasource with extractor :

            extractor.setDataSource(this.url);

Or

            extractor.setDataSource(this.Mediadatasource);

ps: The ByteArrayMediaDataSource class:

import android.annotation.TargetApi;
import android.media.MediaDataSource;
import android.os.Build;
import java.io.IOException;



import android.content.res.AssetFileDescriptor;
 import android.media.MediaDataSource;
import android.util.Log;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.IOException;

@TargetApi(Build.VERSION_CODES.M)
public class ByteArrayMediaDataSource extends MediaDataSource {

    private static final String TAG = ByteArrayMediaDataSource.class.getSimpleName();


    private   byte[] data;

    public ByteArrayMediaDataSource(byte []data) {
//        assert data != null;
        this.data = data;
    }
    @Override
    public int readAt(long position, byte[] buffer, int offset, int size) throws IOException {


        if (position >= data.length) {
            return -1; // -1 indicates EOF
        }
        if (position + size > data.length) {
            size -= (position + size) - data.length;
        }

        Log.d(TAG, "MediaDataSource size = "+size);


        System.arraycopy(data, (int)position, buffer, offset, size);
        return size;

    }



    @Override
    public long getSize() throws IOException {

        Log.d(TAG, "MediaDataSource data.length = "+data.length);

        return data.length;
    }

    @Override
    public void close() throws IOException {
        // Nothing to do here
        data =null;
    }
}