Skip to content

ExoPlayer

Jump to bottom
Li-Wei Yap edited this page Jan 2, 2023 · 25 revisions

ExoPlayer

Problems with Android's built-in MediaPlayer

Whilst it is possible to play multiple audio files with multiple MediaPlayer instances simultaneously (see commit ace8314), some devices have problems using multiple MediaPlayers at the same time. For example, the code up to commit ace8314 works on my 64-bit phone (API 24) and my 64-bit emulator (API 30), but with my 32-bit emulator (API 30), every time mIntroMediaPlayer::start() is called, mBackgroundMediaplayer is temporarily disrupted until the end of the current segment of the introduction script being narrated. Other people have faced this issue as well:

Thus, we could experiment with ExoPlayer and see if it helps solve this issue.

General advantages of ExoPlayer

From Google I/O 2017:


Notes on using ExoPlayer

Updates

First attempt

I have something like this:

final Iterator<Integer> iter = introSegmentArrayList.iterator();
ConcatenatingMediaSource introPlaylist = new ConcatenatingMediaSource();
RawResourceDataSource rawResourceDataSource = new RawResourceDataSource(this);
MediaSourceFactory mediaSourceFactory;
while (iter.hasNext())
{
    try
    {
        DataSpec dataSpec = new DataSpec(RawResourceDataSource.buildRawResourceUri(iter.next()));
        rawResourceDataSource.open(dataSpec);
        MediaItem mediaItem = MediaItem.fromUri(rawResourceDataSource.getUri());
        mediaSourceFactory = new DefaultMediaSourceFactory(this);
        ProgressiveMediaSource mediaSource = (ProgressiveMediaSource) mediaSourceFactory.createMediaSource(mediaItem);
        introPlaylist.addMediaSource(mediaSource);
        SilenceMediaSource silence = new SilenceMediaSource(mPauseDurationInMilliSecs * 1000);
        introPlaylist.addMediaSource(silence);
    }
    catch (RawResourceDataSource.RawResourceDataSourceException e)
    {
        e.printStackTrace();
    }
}

try
{
    DataSpec dataSpec = new DataSpec(RawResourceDataSource.buildRawResourceUri(R.raw.backgroundsoftrock));
    rawResourceDataSource.open(dataSpec);
}
catch (RawResourceDataSource.RawResourceDataSourceException e)
{
    e.printStackTrace();
}
MediaItem mediaItem = MediaItem.fromUri(rawResourceDataSource.getUri());
mediaSourceFactory = new DefaultMediaSourceFactory(this);
ProgressiveMediaSource mediaSource = (ProgressiveMediaSource) mediaSourceFactory.createMediaSource(mediaItem);

MergingMediaSource mergingMediaSource = new MergingMediaSource(true, true, mediaSource, introPlaylist);
SimpleExoPlayer exoPlayer = new SimpleExoPlayer.Builder(this).build();
exoPlayer.addMediaSource(mergingMediaSource);
exoPlayer.prepare();
exoPlayer.play();

but I keep getting an MergingMediaSource.IllegalMergeException. And when, instead of creating a MergingMediaSource, I use two separate ExoPlayer instances, one for the ConcatenatingMediaSource and the other for the background music, I get the same problem as before.

Could also try TrackSelector

Further investigations

I did some further investigations by extending the Soft Sound app, which is a very simple application that creates a MutableMap (Kotlin) of ExoPlayers. It seems that the x86 emulator has problems playing the speech MP3 files (both with and without SSML) generated by the Google Cloud Text-To-Speech API at the same time as all the other background ambience noises.

I also notice that this problem doesn't occur 100% of the time if, rather than using MediaPlayer, I use an ArrayList of ExoPlayers. However, not only does it still occur often enough to be a concern but I also have concerns about the viability of using an ArrayList rather than just a single ExoPlayer. The ArrayList would complicate the coding of the UI updates and also increase the amount of run-time memory required.

One strategy could be to play exclusively background noises rather than music altogether, and the files containing these background noises should be small (so that we can loop) and also already contain noticeable gaps so that if they do get interrupted by the speech files, at least the interruptions won't be that noticeable.

I think, for the reasons stated above, we should also definitely replace MediaPlayer with ExoPlayer.

Other alternatives

Android has other in-built classes: SoundPool, AudioTrack. For comparison: see here.

More notes:

Use Service?

Sample code:

Breakthrough

I finally resolved the issue by playing (looping) the background noise in a SoundPool. The segments of the introduction script must continue to be played via a MediaPlayer or an ExoPlayer to take advantage of OnCompletionListener or onPositionDiscontinuity() respectively. Note that this means we definitely cannot have background music, only background noises that we can loop; these background noises should be short (try to keep them <1 Mb).

Note: Even so, it still does not work on all background sounds (see here) but at least it's a significant improvement from before.

Final thoughts

I'm thinking about using ExoPlayer in PlayIntroductionActivity but keeping MediaPlayer in CharacterSelectionActivity.

PlayIntroductionActivity

Advantages of ExoPlayer:

  • Allows us to create a playlist, so that prepare() only needs to be called once. Also allows us to disregard the use of a Handler by providing the SilenceMediaSource class. All in all, makes the code in PlayIntroductionActivity much, much cleaner and much easier to read and maintain in the long run (e.g. fewer conditionals in onResume()).
  • Less per-device variation (won't be sure exactly how big/small this advantage actually is because we can't test on every single device, and also since we aren't streaming any data; we're just playing simple audio files). I just want to be safe and make sure that the background noises do not interfere with the introduction audio text.

Advantages of MediaPlayer:

  • Shorter initial load time, because initially only the first segment of the introduction script is prepared, and all subsequent segments will be prepared as the app proceeds down the rest of the script.

CharacterSelectionActivity

Advantages of MediaPlayer:

  • Requires less code than ExoPlayer. This is the opposite of PlayIntroductionActivity. It is MediaPlayer that simplifies CharacterSelectionActivity (which itself is already rather clean IMO 😉) but ExoPlayer that simplifies PlayIntroductionActivity. With ExoPlayer, I might risk over-engineering. Besides, I think per-device variation won't be a huge problem here (if it's a problem at all), since we aren't streaming any data; we're just playing simple audio files.

Advantages of ExoPlayer:

  • Nothing other than the sake of consistency. If we are going to use ExoPlayer in PlayIntroductionActivity, then it would be nice to also use ExoPlayer in CharacterSelectionActivity.

Singleton?

I've decided not to use a Singleton because Java classes have no destructor; instead classes are garbage-collected. But I wanna know exactly if and when my resources are freed.

Clone this wiki locally