SurfaceViews allow drawing to a separate thread to achieve realtime performance. Key aspects include: - Driving the SurfaceView with a thread that locks and draws to the canvas in a loop. - Using input buffering and object pooling to efficiently process touch/key events from the main thread. - Employing various timing and drawing techniques like fixed scaling to optimize for performance. Managing the SurfaceView lifecycle to ensure the drawing thread starts and stops appropriately.

1. Tricks to Making a Realtime
SurfaceView Actually Perform in
Realtime
Maarten Edgar

2. Hello, my name is …

3. Hello, my name is …
Maarten Edgar

4. What we’ll cover
SurfaceViews:
• Why
• When
• What
• How
• Hard earned lessons

5. Why use a SurfaceView?
SurfaceView
GL_SurfaceView
TextureView
SurfaceTexture
View

6. What is a SurfaceView?
A View which gives you access to a
Surface using .getHolder(), which is
drawn on a seperate thread and is
double/triple buffered behind the
scenes.
It cuts holes and displays underneath
the window it is in.

7. How to use it:
• Setup
• Threads vs Runnables and other
control mechanisms
• Loops
• UI communication
• Tips

8. Setup
• Activity/View
• SurfaceView and its Thread

9. Setup: Activity and View
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// set flags as needed
getWindow().setFormat(PixelFormat.RGBA_8888);
this.setVolumeControlStream(AudioManager.STREAM_MUSIC);
setContentView(R.layout.activity_game);
// get handles to the View from XML, and its Thread
mCSurfaceView = (MySurfaceView) findViewById(R.id.surfaceview);
setSurfaceType(View.LAYER_TYPE_SOFTWARE);
mSurfaceViewThread = mSurfaceView.getThread();
createInputObjectPool();

10. Your SurfaceView class
public class ChiBlastSurfaceView extends SurfaceView implements SurfaceHolder.Callback {
public ChiBlastSurfaceView(Context context) {
super(context);
mSurfaceCreated = false;
touchBool = true;
// register our interest in hearing about changes to our surface
SurfaceHolder holder = getHolder();
holder.addCallback(this);
myHandler = new MyInnerHandler(this);
// create thread only; it's started in surfaceCreated()
thread = new ChiBlastSurfaceViewThread(holder, context, myHandler);
setFocusable(true); // make sure we get key events
}

11. Your SurfaceView callbacks 1/3
SurfaceHolder.Callback:
@Override
public void surfaceCreated(SurfaceHolder holder) {
// start the thread here so that we don't busy-wait
in run() waiting for the surface to be created
if (mSurfaceCreated == false)
{
createThread(holder);
mSurfaceCreated = true;
touchBool = true;
}
}

12. Your SurfaceView callbacks 2/3
SurfaceHolder.Callback:
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
mSurfaceCreated = false;
cleanupResource();
terminateThread();
}

13. Your SurfaceView callbacks 3/3
SurfaceHolder.Callback:
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
thread.setSurfaceSize(width, height);
}

14. Setup: driving the SurfaceView
Runnables, thread and loops, oh my!

15. Setup: Thread
public class ChiBlastSurfaceViewThread extends Thread {
public ChiBlastSurfaceViewThread(SurfaceHolder surfaceHolder,
Context context, Handler handler) {
// get handles to some important objects
mSurfaceHolder = surfaceHolder;
mSurfaceHolder.setFormat(PixelFormat.RGBA_8888);
mContext = context;
res = context.getResources();
//any other initialization:
ops = new BitmapFactory.Options();
ops.inPurgeable = true;
ops.inDensity = 0;
ops.inDither = false;
ops.inScaled = false;
ops.inPreferredConfig = Bitmap.Config.ARGB_8888;
ops.inJustDecodeBounds = false;
}

16. @Override
public void run() {
while (mRun) {
Canvas c = null;
try {
// update game state
processInput();
//if (mMode == STATE_SCROLL_MAP)
if (mMode != STATE_PAUSE)
{
updatePhysics(timeDiff);
}
c = mSurfaceHolder.lockCanvas(null);
synchronized (mSurfaceHolder) {
doDraw(c);
}
} finally {
// do this in a finally so that if an exception is thrown
// during the above, we don't leave the Surface in an
// inconsistent state
if (c != null) {
mSurfaceHolder.unlockCanvasAndPost(c);
}
}
}
}
Setup: Thread

17. The Thread and your Activity
What does this now mean for your
Activity?
or
How do we make this fit into the
Android Lifecycle?

18. The Thread and your Activity
@Override
protected void onPause() {
super.onPause();
// pause game when Activity pauses
mSurfaceView.getThread().pause();
mSurfaceView.terminateThread();
System.gc();
}

19. The Thread and your Activity
@Override
protected void onResume()
{
super.onResume();
if (mSurfaceView.mSurfaceCreated)
{
mSurfaceView.createThread(mSurfaceView.getHolder());
setSurfaceType(View.LAYER_TYPE_SOFTWARE);
}
mSurfaceView.SetTouch(true);
}

20. The Thread and your Activity
@Override
protected void onRestoreInstanceState(Bundle inState) {
// just have the View's thread load its state from our Bundle
if (mSurfaceView.mSurfaceCreated)
{
mSurfaceView.createThread(mSurfaceView.getHolder());
setSurfaceType(View.LAYER_TYPE_SOFTWARE);
}
mSurfaceViewThread.restoreState(inState);
}

21. The main loop
• AFAFP
• Fixed step

22. @Override
public void run() {
long beginTime; // the time when the cycle begun
long timeDiff; // the time it took for the cycle to execute
int sleepTime; // ms to sleep (<0 if we're behind)
int framesSkipped; // number of frames being skipped
timeDiff = System.currentTimeMillis()+50;
sleepTime = 0;
while (mRun) {
Canvas c = null;
try {
beginTime = System.currentTimeMillis();
framesSkipped = 0; // resetting the frames skipped
// update game state
processInput();
//if (mMode == STATE_SCROLL_MAP)
if (mMode != STATE_PAUSE)
{
updatePhysics(timeDiff);
}
c = mSurfaceHolder.lockCanvas(null);
synchronized (mSurfaceHolder) {
doDraw(c);
}
The main loop 1/3

23. The main loop 2/3
// calculate how long did the cycle take
timeDiff = System.currentTimeMillis() - beginTime;
// calculate sleep time
sleepTime = (int)(FRAME_PERIOD - timeDiff);
if (sleepTime > 0) {
// if sleepTime > 0 we're OK
try {
// send the thread to sleep for a short period
// very useful for battery saving
Thread.sleep(sleepTime);
} catch (InterruptedException e) {}
}
while (sleepTime < 0 && framesSkipped < MAX_FRAME_SKIPS) {
// we need to catch up
// update without rendering
processInput();
if (mMode != STATE_PAUSE)
{
updatePhysics(timeDiff);
}
// add frame period to check if in next frame
sleepTime += FRAME_PERIOD;
framesSkipped++;
}

24. The main loop 3/3
} finally {
// do this in a finally so that if an exception is thrown
// during the above, we don't leave the Surface in an
// inconsistent state
if (c != null) {
mSurfaceHolder.unlockCanvasAndPost(c);
}
}
}
}

25. UI Communication
Use a Handler:

26. static class MyInnerHandler extends Handler {
private final WeakReference<ChiBlastSurfaceView> mView;
MyInnerHandler(ChiBlastSurfaceView aView) {
mView = new WeakReference<ChiBlastSurfaceView>(aView);
}
@Override
public void handleMessage(Message m) {
ChiBlastSurfaceView theView = mView.get();
theView.mStatusText.setText(m.getData().getString("text"));
if (m.getData().getInt("viz") == View.VISIBLE)
{
theView.mStatusText.setVisibility(View.VISIBLE);
//mStatusText.setAnimation(displayTextAnim);
//mStatusText.startAnimation(displayTextAnim);
}
else
{
if (m.getData().getInt("viz") == View.INVISIBLE)
{
theView.mStatusText.setVisibility(View.INVISIBLE);
theView.mStatusText.setAnimation(null);
}
else if (m.getData().getInt("viz") == View.GONE)
{
theView.mStatusText.setVisibility(View.GONE);
}
}
theView.mStatusText.invalidate();
}
}

27. Setup: Cleanup
public void terminateThread ()
{
boolean retry = true;
thread.setRunning(false);
while (retry) {
try
{
thread.join();
retry = false;
}
catch (InterruptedException e)
{
}
//break; //THIS BREAKS IT ON PUSHING HOME
}
//thread = null; //THIS BREAKS IT ON PUSHING HOME
}

28. Tips
• Input buffer
• Object creation
• Scaling
• Drawing, bitmaps and other dirty
things

29. Tips: input buffer in SVActivity
private void createInputObjectPool() {
inputObjectPool = new ArrayBlockingQueue<InputObject>(INPUT_QUEUE_SIZE);
for (int i = 0; i < INPUT_QUEUE_SIZE; i++) {
inputObjectPool.add(new InputObject(inputObjectPool));
}
}

30. public class InputObject {
public static final byte EVENT_TYPE_KEY = 1;
public static final byte EVENT_TYPE_TOUCH = 2;
public static final int ACTION_KEY_DOWN = 1;
public static final int ACTION_KEY_UP = 2;
public static final int ACTION_TOUCH_DOWN = MotionEvent.ACTION_DOWN;
public static final int ACTION_TOUCH_POINTER_DOWN = MotionEvent.ACTION_POINTER_DOWN;
//public static final int ACTION_TOUCH_POINTER_2_DOWN = MotionEvent.ACTION_POINTER_2_DOWN;
public static final int ACTION_TOUCH_MOVE = MotionEvent.ACTION_MOVE;
public static final int ACTION_TOUCH_UP = MotionEvent.ACTION_UP;
public static final int ACTION_TOUCH_POINTER_UP = MotionEvent.ACTION_POINTER_UP;
//public static final int ACTION_TOUCH_POINTER_2_UP = MotionEvent.ACTION_POINTER_2_UP;
public ArrayBlockingQueue<InputObject> pool;
public byte eventType;
public long time;
public int action;
public int keyCode;
public int x;
public int y;
public int x2;
public int y2;
public int pointerID;
public int pointerIndex;
public int pointerIndex2;
InputObject 1/5

31. InputObject 2/5
public InputObject(ArrayBlockingQueue<InputObject> pool) {
this.pool = pool;
}
public void useEvent(KeyEvent event) {
eventType = EVENT_TYPE_KEY;
int a = event.getAction();
switch (a) {
case KeyEvent.ACTION_DOWN:
action = ACTION_KEY_DOWN;
break;
case KeyEvent.ACTION_UP:
action = ACTION_KEY_UP;
break;
default:
action = 0;
}
time = event.getEventTime();
keyCode = event.getKeyCode();
}

32. public void useEvent(MotionEvent event) {
eventType = EVENT_TYPE_TOUCH;
int a = event.getAction();
switch (a) {
case MotionEvent.ACTION_DOWN:
action = ACTION_TOUCH_DOWN;
break;
case MotionEvent.ACTION_POINTER_DOWN:
action = ACTION_TOUCH_POINTER_DOWN;
break;
case MotionEvent.ACTION_POINTER_2_DOWN:
action = ACTION_TOUCH_POINTER_DOWN;
break;
case MotionEvent.ACTION_MOVE:
action = ACTION_TOUCH_MOVE;
break;
case MotionEvent.ACTION_UP:
action = ACTION_TOUCH_UP;
break;
case MotionEvent.ACTION_POINTER_UP:
action = ACTION_TOUCH_POINTER_UP;
break;
case MotionEvent.ACTION_POINTER_2_UP:
action = ACTION_TOUCH_POINTER_UP;
break;
default:
action = -1;
}
InputObject 3/5

33. InputObject 4/5
time = event.getEventTime();
pointerIndex = (event.getAction() &
MotionEvent.ACTION_POINTER_ID_MASK) >>
MotionEvent.ACTION_POINTER_ID_SHIFT;
pointerID = event.getPointerId(pointerIndex);
x = (int) event.getX(pointerIndex);
y = (int) event.getY(pointerIndex);
if (event.getPointerCount() > 1)
{
pointerIndex2 = pointerIndex== 0 ? 1 : 0;
x2 = (int)event.getX(pointerIndex2);
y2 = (int)event.getY(pointerIndex2);
}
}

34. InputObject 5/5
public void useEventHistory(MotionEvent event, int historyItem) {
eventType = EVENT_TYPE_TOUCH;
action = ACTION_TOUCH_MOVE;
time = event.getHistoricalEventTime(historyItem);
pointerIndex = (event.getAction() &
MotionEvent.ACTION_POINTER_ID_MASK) >>
MotionEvent.ACTION_POINTER_ID_SHIFT;
pointerID = event.getPointerId(pointerIndex);
x = (int) event.getHistoricalX(pointerIndex, historyItem);
y = (int) event.getHistoricalY(pointerIndex, historyItem);
if (event.getPointerCount() > 1)
{
pointerIndex2 = pointerIndex== 0 ? 1 : 0;
x2 = (int) event.getHistoricalX(pointerIndex2, historyItem);
y2 = (int) event.getHistoricalY(pointerIndex2, historyItem);
}
}
public void returnToPool() {
pool.add(this);
}

35. @Override
public boolean onTouchEvent(MotionEvent event) {
try {
// history first
int hist = event.getHistorySize();
if (hist > 0)
{
// add from oldest to newest
for (int i = 0; i < hist; i++)
{
//for (int i = hist-1; i > -1; i--) {
InputObject input = inputObjectPool.take();
input.useEventHistory(event, i);
mSurfaceViewThread.feedInput(input);
}
}
// current last
InputObject input = inputObjectPool.take();
input.useEvent(event);
mSurfaceViewThread.feedInput(input);
} catch (InterruptedException e) {
}
// don't allow more than 60 motion events per second
try {
Thread.sleep(16);
} catch (InterruptedException e) {
}
return true;
}
Back to the activity:

36. public void feedInput(InputObject input) {
synchronized(inputQueueMutex) {
try {
inputQueue.put(input);
} catch (InterruptedException e) {
//Log.e(TAG, e.getMessage(), e);
}
}
}
private void processInput() {
synchronized(inputQueueMutex) {
ArrayBlockingQueue<InputObject> inputQueue = ChiBlastSurfaceView.inputQueue;
while (!inputQueue.isEmpty()) {
try {
InputObject input = inputQueue.take();
if (input.eventType == InputObject.EVENT_TYPE_KEY) {
//processKeyEvent(input);
} else if (input.eventType == InputObject.EVENT_TYPE_TOUCH) {
processMotionEvent(input);
}
input.returnToPool();
} catch (InterruptedException e) {
//Log.e(TAG, e.getMessage(), e);
}
}
}
}
And in the SurfaceView.Thread:

37. Tips: object creation

38. Tips: object creation
Just don’t do it.

39. Tips: object creation
Or do it up front.
No matter how odd that sometimes
may seem.

40. Tips: scaling
Two types of scaling:
• Realtime whole view SV scaling
only works from Android N
• Fixed scaling (as done in Unreal
Tournament 3)

41. Tips: scaling
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
//thread.setSurfaceSize(width, height);
if (mCanvasWidth != width)
{
int scaledWidth = (int)(width*0.75f);
int scaledHeight = (int)(height*0.75f);
if (scaledHeight != height)
{
yRatio = (float)(scaledHeight / (float)height);
xRatio = (float)(scaledWidth / (float)width);
}
holder.setFixedSize(scaledWidth, scaledHeight);
thread.setSurfaceSize(scaledWidth, scaledHeight);
}
}

42. Tips: drawing, bitmaps and other
dirty things

43. Tips: drawing, bitmaps and other
dirty things
In SurfaceView.Thread doDraw():
canvas.drawBitmap(mBackgroundImage,
null, fullscreenRect, mPicPaint);

44. Q&A

45. Thank you!
Maarten Edgar
lifeboatsoft@gmail.com
Resources:
https://source.android.com/devices/graphics/architecture.html
https://github.com/google/grafika