This document discusses the Android multimedia framework on Jelly Bean. It provides an introduction to OpenMAX and describes the simple stack architecture including the developer API, event handler, surface holder, StageFright, OpenMAX interface, and software/hardware codecs. It explains the workflows and sequence flows for playing a media file, including setting the data source, preparing to play, and starting playback. Finally, it covers the synchronization architecture and flow of StageFright.

1. Android Multimedia
Framework
on Jelly Bean
Author: Picker

2. Introduction to OpenMAX

3. Resource from Khronos::OpenMAX

4. Resource from Khronos::OpenMAX

5. from
Multimedia Framework
to
OpenMAX

6. Resource from Khronos::OpenMAX

7. Resource from Khronos::OpenMAX

8. Go through the OpenMAX
Operating System
Operating System
Multimedia Framework
Multimedia Framework
OpenMAX IL
OpenMAX IL
Software & Hardware Codec
Software & Hardware Codec

10. The simple stack architecture
Developer API (MediaPlayer)
Developer API (MediaPlayer)
EventHandler SurfaceHolder
StageFright
OpenMAX Interface
OpenMAX IL
Software & Hardware Codec

11. Big picture of the StageFright

12. The Simple Workflow

13. The StageFright Workflow

14. Path of All the Components

15. Path of the Source Files
frameworks/base/media/java/android/media
frameworks/base/media/jni
frameworks/av/media/libmedia
frameworks/av/media/libmediaplayerservice
frameworks/av/media/libstagefright

16. Big Picture
Class Diagram

18. High­Level Applications Framework

20. StageFright Framework

22. OpenMAX Interface

23. Four Steps for Playing the Media File
Step 1: MediaPlayer mp = new MediaPlayer();
Step 2: mp.setDataSource(mediaPath);
setDataSource
Step 3: mp.prepare();
prepare
Step 4: mp.start();
start

25. How to set the data source?

26. The workflow of the setDataSource

27. The sequence flow of the setDataSource

29. How to prepare to play?

30. The workflow of the prepare

31. The sequence flow of the prepare

33. How to start the playing?

34. The workflow of the start

35. The sequence flow of the start

36. Synchronization Architecture of StageFright

37. Synchronization Flow of StageFright

38. The Synchronization Formula of StageFright
sizedone
mNumFramesPlayed+
mFrameSize
mPositionTimeRealsUs=( )×1000000
mSampleRate
mTimeSourceDeltaUs=mPositionTimeRealUs−mPositionTimeMediaUs
nowUs= RealTimeUs−mTimeSourceDeltaUs
latenessUs=nowUs−timeUs
mPositionTimeRealsUs: the time in real playing
mPositionTimeMediaUs: the time which is defined in the media source

39. The Synchronization Conditions
Conditions of latenessUs
1. > 500000ll
2. > 40000
3. < -10000 1750 if (latenessUs > 500000ll
1751 && mAudioPlayer != NULL
1752 && mAudioPlayer->getMediaTimeMapping(
1753 &realTimeUs, &mediaTimeUs)) {
1757 mVideoBuffer->release();
1758 mVideoBuffer = NULL;
1763 postVideoEvent_l();
1764 return;
1765 }
1767 if (latenessUs > 40000) {
1768 // We're more than 40ms late. 1793 if (latenessUs < -10000) {
1794 // We're more than 10ms early.
1780 mVideoBuffer->release(); 1795 postVideoEvent_l(10000);
1781 mVideoBuffer = NULL; 1796 return;
1797 }
1787
1788 postVideoEvent_l();
1789 return;
1791 }

40. Conclusion
●Easy for maintaining rather than OpenCore
●Support Software/Hardware Codecs
●Unstable Product
● Rough Mechanisms
● Create the NuPlayer for playing the
multimedia streaming