Gtug
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Gtug

on

  • 2,367 views

SG-GTUG

SG-GTUG

Statistics

Views

Total Views
2,367
Views on SlideShare
2,188
Embed Views
179

Actions

Likes
4
Downloads
82
Comments
0

2 Embeds 179

http://krkmohan.wordpress.com 178
http://translate.googleusercontent.com 1

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />
  • <br />

Gtug Presentation Transcript

  • 1. Multimedia Application Development on Android KRK MOHAN Senior Engineer - Muvee Technologies www.muvee.com krk.mohan@gmail.com 1
  • 2. Agenda • Multimedia and Android • Video & Audio: MediaPlayer, MediaRecorder, JetPlayer, SoundPool, AudioTrack • Images: OpenGL-ES graphics, Bitmaps, ImageView 2
  • 3. Scope • Introductory level • Focus on what can and cannot be done with Android FroYo’s multimedia capabilities • References to APIDemos projects and developer site articles provided 3
  • 4. Multimedia Apps • Media Consumption - Slide Shows, Media Players,YouTube, facebook etc • Media Production - Image Editors, Camera enhancements • Huge scope for improvement - great opportunity for developers 4
  • 5. You will identify a great product when you see it Deep, Indulgent, Complete, Elegant (DICE) Guy Kawasaki “The Macintosh Way” 5
  • 6. android.media • Contains classes to play audio/video content from raw resource files, file streams and JET content • Contains classes to record audio/video content from cameras or mic http://developer.android.com/intl/de/guide/topics/media/ index.html 6
  • 7. File formats and codecs • File formats for recording media on device • 3GPP, MP4, RAW_AMR • Codecs used for recording media on device • H263, H264, MPEG4 (SP), AMR_NB 7
  • 8. The big 2 MediaPlayer and MediaRecorder classes encapsulate: File parsers for media file formats such as 3GP, MP4 & Codecs to encode or decode media content Two media frameworks present in FroYo OpenCORE and StageFright 8
  • 9. MediaPlayer - state diagram From Android Developers - MediaPlayer entry 9
  • 10. Audio • MediaPlayer & MediaRecorder • SoundPool - multiple audio sources • AudioTrack & AudioRecord- raw audio manipulation • JetPlayer - interactive music 10
  • 11. Audio - MediaPlayer Playing an audio resource MediaPlayer mp = MediaPlayer.create(context, R.raw.sound_file_1); mp.start(); Playing a file from file system MediaPlayer mp = new MediaPlayer(); mp.setDataSource(PATH_TO_FILE); mp.prepare(); mp.start(); 11
  • 12. Audio - SoundPool • Multiple sounds can be played from raw or compressed source. Sounds are identified by sound ids. • More efficient than multiple MediaPlayers • Each sound can be assigned a priority • Adjustable playback frequency • Support for repeat mode 12
  • 13. Audio - AudioTrack • Used to play raw audio samples (PCM) stored in memory buffers or files • Static mode for playing sounds that fit into memory and need to be played often with low overhead • Streaming mode for sounds that are too big to fit into memory 13
  • 14. Audio - AudioTrack AudioTrack audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,                             44100,                                           AudioFormat.CHANNEL_CONFIGURATION_MONO,             AudioFormat.ENCODING_PCM_16BIT,                               musicLength, AudioTrack.MODE_STREAM); audioTrack.play(); audioTrack.write(music, 0, musicLength); //blocking Check out AudioRecord class Some examples at: http://emeadev.blogspot.com/2009/09/raw-audio-manipulation-in-android.html 14
  • 15. Audio - JETPlayer • Interactive music playback by altering tempo, volume level etc of a series of MIDI clips • Used extensively in game development • JET Creator is used to author JET files http://developer.android.com/intl/de/guide/topics/media/jet/ jetcreator_manual.html 15
  • 16. Video - MediaPlayer mMediaPlayer = new MediaPlayer(); mMediaPlayer.setDataSource(localPath);            mMediaPlayer.setDisplay(holder); mMediaPlayer.setOnCompletionListener(this); mMediaPlayer.setOnPreparedListener(this); mMediaPlayer.prepareAsync(); public void onPrepared(MediaPlayer m) { mMediaPlayer.start(); } public void onCompletion(MediaPlayer m) { mMediaPlayer.release(); }
  • 17. Video - VideoView mVideoView = (VideoView) findViewById(R.id.surface_view); mVideoView.setVideoPath(localPath);            mVideoView.setMediaController (new MediaController(this)); mVideoView.requestFocus(); 17
  • 18. iMovie on Android!? • Classes needed to process individual video frames • API level 1 contained MediaPlayer.getFrame() • MediaPlayer.snoop() is used in music visualization live wall paper, but is marked @hide 18
  • 19. way forward... • OMXCodec.h or a similar interface needs to be exposed via NDK • This would facilitate comprehensive video processing and editing 19
  • 20. Graphics - OpenGL-ES • 3D graphics API to process and render images/textures • On Android GL10 loosely corresponds to OpenGL-ES 1.0/1.1 GL20 loosely corresponds to OpenGL-ES 2.0 • Performance depends on GPU • No major issues with code portability across GPUs 20
  • 21. GLSurfaceView From Android Developers: • Manages a surface, which is a special piece of memory that can be composited into the Android view system. • Manages an EGL display, which enables OpenGL to render into a surface. • Accepts a user-provided Renderer object that does the actual rendering. • Renders on a dedicated thread to decouple rendering performance from the UI thread. • Supports both on-demand and continuous rendering. • Optionally wraps, traces, and/or error-checks the renderer's OpenGL calls. 21
  • 22. OpenGL-ES Textures • Create a Bitmap out of the image to be processed • GLUtils class provides helper functions such as: • texImage2D(int target, int level, int internalformat, Bitmap bitmap, int border) • target = GL10.GL_TEXTURE_2D, level = 0, internalformat = GL10.GL_RGB, GL_UNSIGNED_SHORT_5_6_5 • Compressed textures 22
  • 23. GLSurfaceView.Renderer class MyRenderer implements GLSurfaceView.Renderer {     public void onDrawFrame(GL10 gl) {         // called repeatedly to animate the loaded texture }            public void onSurfaceChanged(GL10 gl, int width, int height) { } public void onSurfaceCreated(GL10 gl, EGLConfig config) { } } 23
  • 24. Together... • To begin rendering mGLSurfaceView = new GLSurfaceView(this); mGLSurfaceView.setRenderer(new MyRenderer()); setContentView(mGLSurfaceView); • To pause rendering mGLSurfaceView.onPause(); •To resume rendering mGLSurfaceView.onResume(); For full example, checkout CubeMapActivity example in ApiDemos 24
  • 25. Misc OpenGL-ES • setDebugFlags(DEBUG_CHECK_GL_ERROR | DEBUG_LOG_GL_CALLS); • Use GLSurfaceView.setRenderMode(RENDERMODE_WHEN_ DIRTY) & GLSurfaceView.requestRender()for passive applications • Use EGLChooser to select an appropriate EGL configuration • Minimize texture sizes and bit depth for greater performance • Use NDK wherever feasible to speed up the application 25
  • 26. Bitmaps & image processing • BitmapFactory.decode(filepath) returns a Bitmap of the image • Use getPixels to access the image’s pixels • getPixels(int[] pixels, int offset, int stride, int x, int y, int width, int height) • Manipulate the pixels and use Bitmap.compress() to encode the processed pixels into a JPEG image 26
  • 27. ImageView • Display PNG, JPG images • Apply effects like color tinting • View can be animated ImageView image = (ImageView) findViewById(R.id.image); image.setBackgroundResource(R.drawable.my_image); http://developer.android.com/intl/de/guide/topics/graphics/2d-graphics.html
  • 28. Appendix - Animations <animation-list xmlns:android="http://schemas.android.com/apk/res/android" android:oneshot="true"> <item android:drawable="@drawable/rocket_thrust1" android:duration="200" /> <item android:drawable="@drawable/rocket_thrust2" android:duration="200" /> <item android:drawable="@drawable/rocket_thrust3" android:duration="200" /> </animation-list> ImageView rocketImage = (ImageView) findViewById(R.id.rocket_image); rocketImage.setBackgroundResource(R.drawable.rocket_thrust); rocketAnimation = (AnimationDrawable) rocketImage.getBackground(); rocketAnimation.start(); 28
  • 29. Thank You! 29