O'Reilly Web20 Expo SF 2010 Conference Mobile Track

1. A Beautiful Mind:
Anatomy and
Architecture of the
iPhone App
Bess Ho
May 4, 2010 Tue
9:00 AM San Francsico

2. Silence

3. Follow QR Code
My Slides

4. Follow QR Code
Linkedin

5. Follow QR Code
vCard

6. My Bio
TEXT Short Code
99702

7. Friendly
Assessment

8. A Beautiful Mind

9. Anatomy and
Architecture of the
iPhone App

13. STANFORD UNIVERSITY
School of Engineering

16. Three Years Ago
Jan 2007

17. Widescreen iPod with touch controls

18. Revolutionary mobile phone

19. Breakthrough internet communicator

20. iPod Phone Internet

29. iMirror

31. M-RFID
Tag
iBadge

32. iDoor (Inside)

33. iDoor (Outside)

34. To boldly go where no man
has gone before:

36. Data Portable Data
Communication Sensing Analysis
s
Tricorder

37. Locator Scanner Compass
Recorder Sensor Search
Text Data Voice

38. Phone Camera
(Audio) (Photo / Video)
GPS
(Geo)
Magnetometer
(Compass)
Accelerometer
(XZY) Touch
(Device Orientation)
Wireless Bluetooth Network / WiFi
(File) (Web / Email)
External Accessory Microphone SMS
(Data) (Audio) (Data)
iPhone’s Anatomy

39. Phone Camera
(Audio) (Photo / Video)
GPS
(Geo)
Magnetometer
(Compass)
Accelerometer
(XZY) Touch
(Device Orientation)
Wireless Bluetooth Network / WiFi
(File) (Web / Email)
External Accessory Microphone SMS
(Data) (Audio) (Data)
Portable Sensors

40. iPhone SDK
Xcode Interface Builder
Data Analysis

41. Phone Camera
(Audio) (Photo / Video)
GPS
(Geo)
Magnetometer
(Compass)
Accelerometer
(XZY) Touch
(Device Orientation)
Wireless Bluetooth Network / WiFi
(File) (Web / Email)
External Accessory Microphone SMS
(Data) (Audio) (Data)
Data Communications

42. Touch
Audio
Visual
Sensor
Geo
Data
iPhone’s Anatomy

43. Laboratory robot has confirmed with a human
test: Apple’s iPhone touchscreen is by far the
most accurate touch panel on the market
iPhone
Google Nexus One
Motorola Droid
HTC Droid Eris
Palm Pre
BlackBerry Storm 2
Reference: http://www.appleinsider.com/articles/10/03/24/robotic_test_reconfirms_apples_iphone_touchscreen_superiority.html
iPhone’s Anatomy

44. Laboratory robot has confirmed with a human
test: Apple’s iPhone touchscreen is by far the
most accurate touch panel on the market
Reference: http://www.appleinsider.com/articles/10/03/24/robotic_test_reconfirms_apples_iphone_touchscreen_superiority.html
iPhone’s Anatomy

45. MOTO Lab Experiment
7mm robotic “finger” for “medium touch”
4mm robotic “finger” for a “very light
touch”
iPhone’s Anatomy

46. iPhone’s Anatomy

47. “All touchscreens are
not created equal.”
Screen sensitivity is a combination of
1) hardware component quality
2) design and
3) software integration - Operating System to
ensure responsiveness for the user
iPhone’s Anatomy

48. Resistive VS Capacitive
Touchscreen Technology

49. Resistive Touchscreens
A screen where two thin metallic layers are separated by a
narrow gap. A finger pushing down on the top layer makes
contact with the bottom surface and the point of contact is
computed by the accompanying electronics.
iPhone’s Anatomy

50. Resistive Touchscreens
A screen where two thin metallic layers are separated by a
narrow gap. A finger pushing down on the top layer makes
contact with the bottom surface and the point of contact is
computed by the accompanying electronics.
iPhone’s Anatomy

51. Resistive Touchscreens
A screen where two thin metallic layers are separated by a
narrow gap. A finger pushing down on the top layer makes
contact with the bottom surface and the point of contact is
computed by the accompanying electronics.
iPhone’s Anatomy

52. Resistive Touchscreens
Step 1: Step 2:
Touch Press
Pressure-sensitive
iPhone’s Anatomy

53. Capacitive Touchscreens
This capacitive technology responds to the electrical
properties of your skin, not the pressure of your finger
to figure out where you are touching the screen.
iPhone’s Anatomy

54. Capacitive Touchscreens
This capacitive technology responds to the electrical
properties of your skin, not the pressure of your finger
to figure out where you are touching the screen.
iPhone’s Anatomy

55. Resistive VS Capacitive
Touchscreen Technology

56. Resistive VS Capacitive
Touchscreen Technology

57. Resistive VS Capacitive
Touchscreen Technology

58. Web OS VS Native
HTML5 / JS / CSS JAVA Objective-C
Touchscreen Technology

59. WebOS VS Native
Touchscreen Technology

60. Web Plug-in WebOS
Web Run Time
Widget SDK PDK (C / C+
QT
Java +)
Symbian
Java
WebOS & Native Platforms

62. Path of Innovation

63. Web App Native App

64. Xcode
Interface
DashCode
Builder

65. Activity Monitor
CPU Sampler
Leaks
Object
Allocations
Core Data
File Activity
UI Recorder
Core Animation
Open GL ES
System Usage
Available Instruments

66. Activity Monitor overall CPU, memory, disk &
Monitor network activity
Precise time-based sampling of CPU
CPU Sampler
usage
Leaks Detects memory leaks
Object
Measures memory usage by class
Allocations
Monitor Core Data activity &
Core Data
performance
Monitor application’s interaction with
File Activity
the file system
Available Instruments

67. Captures & play back UI events to
UI Recorder run exact same sequence of user
interactions
Core Measures Core Animation graphics
Animation performance and resulting CPU load
Measures Open GL ES graphics
Open GL ES
performance and resulting CPU load
Monitors file, network & memory I/O
System Usage
use and duration for each method
Available Instruments

69. Touch

70. Cocoa Touch Layer
Media Layer
Core Services Layer
Core OS Layer
iPhone SDK Frameworks

71. 320 pixel
480 pixel
iPhone / iTouch Screen Size

72. 768 pixel
1024 pixel
iPad Screen Size

73. Large High (hdpi)
3.2” Screen
320 pixel Normal Medium (mdpi)
Small Low (ldpi)
480 pixel
Android <1.5 Android 1.6+
Android Screen Sizes

74. Screen Sizes
84 x 84 pixel
128 x 128 pixel
176 x 208 pixel
240 x 320 pixel
360 x 640 pixel
120 x 160 pixel
Screen Width
Tiny: 84, 96, 101, 128,
130,132
Small: 160, 176
Medium: 208, 220, 240
Large: 320, 360, 480
Nokia Screen Sizes

75. 1 2 3
iPhone iPad Universal
App
Native App

76. iPhone Icon
Promotion
57 x 57 pixel
iPad Icon
512 x 512 pixel 72 x 72 pixel
iPhone App Store Submission

77. 162 ppi 132 ppi
Core Text Framework
Text-rendering & layout
features
Animated text with special
effects
Display Density (pixels per inch)

78. Hardware Software
Prefer saving and
Accelerometer
loading data invisibly
Compass
Without an explicit
Core Location
“save” or “load” option
Wi-Fi
Supports Open GL ES
Bluetooth
2.0 with legacy
Microphone
support for Open GL
headphone socket
ES 1.0
iPad

79. Supports external
displays and
projectors
Supports external
Apple or third-party
keyboard
Likely supports
external devices -
music keyboards,
media controllers,
game hardware
iPad

80. iPhone UI Library

81. Carrier 10:32 AM
Carrier 10:32 AM
Carrier 3G 10:32 AM
Signal Strength
Carrier
Current Network
Connection
Time
Battery Charge
Status Bar

82. 6
Music Videos Podcasts Search More
9 99 999
Should be present on all
screens
switch between modes &
views
Badges are
superimposed in the tab
bar to inform user of
new items
Tab Bar & Badges

83. 6
Music Videos Podcasts Search More
Tab Bar

84. Activity indicator for nav bar
on grey background...
Network activity
Display if it takes more
than a couple of seconds
to perform the task
Activity Indicator

85. Back Button Pane Label Button
Optional instructions for this pane
Cancel Pane Label Save
Pane Label Disabled
Groups All Contacts +
Inbox (20) 2 of 50
Display title of current
view
Display buttons that
trigger action to the view
or navigate
Navigation Bar

86. Page Indicator

87. 5
Progress Indicator 24 of 39
Primary Secondary
Touch size 44 x 44 pixels
< 5 toolbar icons
Toolbar

88. Toolbar Icons

89. MockApp
mockapp.com/ Google
MockApp Cancel
http://mockapp.com/
MockApp Cancel
Google
Browser Bar

90. Sample search text
Search
Clear Directions Cancel
Start: Current Location
End:
Type a company name or stock ID.
Cancel
Search Cancel
Placeholder text
Bookmarks button & Clear
button
Prompt with descriptive title
above the search bar
Search Bar

91. Primary action
you can also do this
or maybe this
or why not that
Cancel
Delete
Cancel
Selection Or Confirmation Menu
Important or common action
should appear to the top
Destructive action use red button
Action Sheets

92. 0:00 0:28
OK Delete
Play Record
Action Sheets

93. Select Select All Paste
MockApp
Bubbles

94. Main Message
Optional explanation of what a user needs to do
Primary Button
Confirmation Message
Optional explanation of what the system is
asking
Secondary Primary
Confirmation Message
Please enter your password
Optional explanation of what the system is
john@doe.com
asking
Password
Cancel OK
Primary
Use it wisely
Require immediate user attention
Alerts

95. Web20 Expo SF
Sent from iPhone
It supports multiple lines
It supports scrolling
Text Views

96. From: Twitter Hide
To: Dotan Saguy
Everybody is now following you on Twitter!
April 1, 2035 1:33 PM Mark Unread
Display rich HTML content
Web / Email Views

97. Q W E R T Y U I O P
A S D F G H J K L
#+= Z X C V B N M
ABC space @ . return
1 2
ABC
3
DEF
4
GHI
5
JKL
6
MNO
7
PQRS
8
TUV
9
WXYZ
0
+
Keyboards

98. N G A
A B
This is a regular table B Pete Gardener C
D
C
view D E
F
M
E
F Tess Grady G
G H
I
Divided into sections H
I M.J. Grey J
(the letters are the sections) J K
K L
Each row is an item L
M
Jenn Guggenheim M
N
O
of the list N
O H P
P Q
and can contain several Q
Sara Hashimoto
R
R S
data elements (image, text, etc.) S T
T U
O U Em Hirsch V
W
Display lists of items
List is divided into
sections separated by grey
headers
Table Views

99. Unread 29
First Last name mobile
Item to delete or move
Ready to be deleted Delete
Not pressed yet Current status
When pressed it highlights
This is a regular table view
With icons like in
The “more” tab
of the iPod app
Podcasts
Table Views

100. This group has 3 items
This item has been selected
Items highlight briefly when hit
This group has only 1 item
You can insert headers too
This one lets you drill down
This one shows Current status
Segmented controls
Tab One Tab Two Tab Three
Tab One Tab Two Tab Three
Tab One Tab Two Tab Three
Grouped Table Views

101. Bess Ho
home (111) 222-3333
mobile (111) 222-3333
work (111) 222-3333
whatever (111) 222-3333
Text Message Share Contact
You can even insert instructions like these as
well if they’re helpful in this context.
Silent
This item is turned OFF
Ring
This item is turned ON
ON
Grouped Table Views

102. August 03 2007 7 45
September 04 2008 8 46
October 05 2009 9 47 AM
November 06 2010 10 48 PM
December 07 2011 11 49
Sat Oct 3 7 50 28
Sun Oct 4 8 55 29
Today 9 00 AM 0 hours 30 mins
Tue Oct 6 10 05 PM 1 31
Wed Oct 7 11 10 2 32
Date & Time Pickers

103. First & default value
Second value
Third value
Value Picker

104. Tap
~ Single Mouse Click
Swipe
Reveal the delete button
in a table-view row
Drag
Scroll or Pan
Gestures

105. Tap
~ Single Mouse Click
Swipe
Reveal the delete button
in a table-view row
Drag
Scroll or Pan
Gestures

106. Tap
~ Single Mouse Click
Swipe swipe
Reveal the delete button
in a table-view row
Drag
Scroll or Pan
Gestures

107. Tap
~ Single Mouse Click
Swipe
Reveal the delete button
in a table-view row
Drag
Scroll or Pan
Gestures

108. Tap
~ Single Mouse Click
Swipe
Reveal the delete button
in a table-view row
Drag drag
Scroll or Pan
Gestures

109. Double
Tap
Zoom-in / Zoom-out
Touch &
Hold view in
Display a magnified
editable text
Gestures

110. Double
Tap
Zoom-in / Zoom-out
2x
tap
Touch &
Hold view in
Display a magnified
editable text
Gestures

111. Double
Tap
Zoom-in / Zoom-out
2x
tap
Touch &
Hold view in
Display a magnified
touch
and
editable text hold
Gestures

112. Pinch
Close
Zoom-in
Pinch Open
Zoom-out
Gestures

113. Pinch
Close
Zoom-in
pinch close
Pinch Open
Zoom-out
Gestures

114. Pinch
Close
Zoom-in
pinch close
Pinch Open
Zoom-out
pinch open
Gestures

115. Audio

116. 30-pin dock connector
Require Certification
SDK
iPhone
External Accessory Framework
External Accessory (Data)

117. iPhone Headset Microphone

118. iPhone Headset Microphone

119. Audible Frequency
Ambient
Voice
Music
Inaudible Frequency
Heartbeat
Breathing
Sleeping
Instrument Signal
iPhone Microphone

120. Recording / Playing Audio Processing
iPhone Microphone

121. Audio Toolbox
Framework
AV Foundation
Framework
OpenAL Framework
Audio Unit Framework
Audio Queue Services
Remote IO Unit
Media Player Framework
iPhone Audio Frameworks

122. Visual Sensor

123. Digial Telephoto Camera

124. Digital Microscope

125. Camera
(Photo / Video)
Scanner
Bar Code
QR Code

126. Camera
(Photo / Video)
Card Scanner
Business Card

127. Camera
(Photo / Video)
Text Scanner
Optical Character Recognition
(OCR)

128. Camera
(Photo / Video)
Language Translator

129. Camera
(Photo / Video)
“Live” Scanner
Augmented Reality (AR)

130. Camera
(Photo / Video)
Recording
Playing

131. Kodak Pearl Module
Dental System
Practice Management Systems
3D & Extraoral Imaging
Intraoral Radiography
Intraoral Digital Image
Intraoral X-ray Image

132. Video Conferencing
Eye Tracking
Iris Scanning
Front-Facing Camera (Future)

133. Geo

134. Cocoa Touch Layer
Map Kit
Media Layer
Core Services Layer
Core Location
Core OS Layer
iPhone SDK Frameworks

135. MapKit Core
Location
iPhone SDK Frameworks

136. CLLocation Manager
CLLocation
CLHeading
Core Location Framework: Class

137. UIViewController Interface
CLLocationManager
#import <CoreLocation/CoreLocation.h>
@interface GetLocationViewController :
UIViewController <CLLocationManagerDelegate> {
CLLocationManager *locationManager;
CLLocation *bestEffortAtLocation;
}
@property (nonatomic, retain) CLLocationManager
*locationManager;
@property (nonatomic, retain) CLLocation
*bestEffortAtLocation;
Core Services Layer: Core Location

138. ViewController Method
CLLocationManager
// Create the manager object
self.locationManager = [[[CLLocationManager alloc]
init] autorelease];
locationManager.delegate = self;
locationManager.desiredAccuracy = [[setupInfo
objectForKey:kSetupInfoKeyAccuracy] doubleValue];
[locationManager startUpdatingLocation];
Core Services Layer: Core Location

139. CLLocation Class
Constants
CCLocationDegrees
CCLocationCoordinate2D
CCLocationAccuracy
Accuracy Constants
CCLocationSpeed
CCLocationDirection
Core Services Layer: Core Location

140. Accuracy Constants
CCLocation Class
locationManager.desiredAccuracy
is the most important property of
Location Manager. It determines the
amount of power it consumed.
Constant values are to specify the accuracy of a location.
kCLLocationAccuracyBest
Best
kCLLocationAccuracyNearestTenMete
10 Meters
rs
100 Meters
kCLLocationAccuracyHundredMeters
1000 Meters
kCLLocationAccuracyKilometer
3000 Meters
kCLLocationAccuracyThreeKilometers
Core Services Layer: Core Location

141. CLLocation Class
Constants
CCLocationDegrees
Delivers a latitude or longitude value specified in
degrees. Data type is double.
CCLocationSpeed
Delivers the speed at which the device is moving in
meters per second. Data type is double.
Core Services Layer: Core Location

142. CLLocation Class
Constants
CCLocationDirection
Delivers a direction that is measured in degrees
and relative to true north. Data type is double.
North is 0 degrees
East is 90 degrees
South is 180 degrees
Any “-” value indicates an invalid direction
Core Services Layer: Core Location

143. CLLocation Class
Properties
altitude
coordinate
course
horizontalAccuracy
speed
timestamp
verticalAccuracy
Core Services Layer: Core Location

144. Measurement Units
altitude (meters)
coordinate
course (degrees)
horizontalAccuracy (meters)
speed (meters per sec)
timestamp (NSDate)
verticalAccuracy (meters)
Core Services Layer: Core Location

145. CLLocation Manager
CLLocation
CLHeading
Core Location Framework: Class

146. CCLocationManager
Core Location
Create a CCLocationManager object to
get heading by invoking
[CCLocationManager
startUpdatingHeading].
iPhone 3GS contains a magnetometer - a
magnetic field detector. It displays the
raw x, y, and z magnetometer values.
Magnitude of the magnetic field is
computed in strength.
Core Services Layer: Core Location

147. CLLocationManager
Core Location
if (locationManager.headingAvailable == NO) {
self.locationManager = nil; // No compass is
available
} else {
// heading service configuration
locationManager.headingFilter =
kCLHeadingFilterNone;
// setup delegate callbacks
locationManager.delegate = self;
// start the compass
[locationManager startUpdatingHeading];
}
}
Core Services Layer: Core Location

148. CLLocation Manager
CLLocation
CLHeading
Core Location Framework: Class

149. CLHeading
Core Location
- (void)locationManager:(CLLocationManager *)manager
didUpdateHeading:(CLHeading *)heading {
// Update the labels with the raw x, y, and z values.
[xLabel setText:[NSString stringWithFormat:@"%.1f",
heading.x]];
[yLabel setText:[NSString stringWithFormat:@"%.1f",
heading.y]];
[zLabel setText:[NSString stringWithFormat:@"%.1f",
heading.z]];
}
MapKit Framework: Class

150. MapKit Core
Location
iPhone SDK Frameworks

151. MKAnnotationView
MKMapView
MKPinAnnotationView
MKPlacemark
MKReverseGeocoder
MKUserLocation
MapKit Framework: Class

152. MKReverseGeocoder
MKReverseGeocoder offers
services to convert a map
coordinate (latitude & Longitude)
to info such as country, city, or
street. It works with a network-
based map service to look up
placemark information for a
specified coordinate value.
Cocoa Touch Layer: MapKit Framework

153. MKReverseGeocoder
Each app is limited to amount
of reverse geocoding
Send one reverse-geocoding
request for any one user action
Reuse the results from initial
request
Suggest not to send one
reverse-geocode request per
minute
Cocoa Touch Layer: MapKit Framework

154. MKAnnotationView
MKMapView
MKPinAnnotationView
MKPlacemark
MKReverseGeocoder
MKUserLocation
MapKit Framework: Class

155. MKMapView Class
Properties
annotations scrollEnabled
annotationsVisibleRect selectedAnnotations
centerCoordinate showsUserLocation
delegate userLocation
mapType userLocationVisible
region zoomEnabled
Cocoa Touch Layer: MapKit Framework

156. MKMapView Class
MKMapType
It delivers the type of map to display.
MKMapTypeStandard
MKMapTypeSatellite
MKMapTypeHybrid
Cocoa Touch Layer: MapKit Framework

157. MKAnnotationView
MKMapView
MKPinAnnotationView
MKPlacemark
MKReverseGeocoder
MKUserLocation
MapKit Framework: Class

158. MKAnnotationView Class
Properties
annotation image
calloutOffset leftCalloutAccessoryView
canShowCallout reuseIdentifier
centerOffset rightCalloutAccessoryView
enabled selected
Cocoa Touch Layer: MapKit Framework

159. MKAnnotationView
MKMapView
MKPinAnnotationView
MKPlacemark
MKReverseGeocoder
MKUserLocation
MapKit Framework: Class

160. MKAnnotationView Class
Properties
Properties
animatesDrop
pinColor
Constants
MKPinAnnotationColor
MKPinAnnotationColorRed (Destination Points)
MKPinAnnotationColorGreen (Starting Points)
MKPinAnnotationColorPurple (User-specified
Points)
Cocoa Touch Layer: MapKit Framework

161. Google Location
Maps Services
External
Library
Android SDK

162. Interfaces
GpsStatus.Listener
GpsStatus.NmeaListener
LocationListener
package: android.location

163. Classes
Address GpsStatus
Criteria Location
Geocoder LocationManager
GpsSatellite LocationProvider
package: android.location

164. Location Class Methods
getAccuracy()
getAltitude() getLongitude()
getBearing() getProvider()
getExtras() getSpeed()
getLatitude() getTime()
package: android.location

165. Classes
Address GpsStatus
Criteria Location
Geocoder LocationManager
GpsSatellite LocationProvider
package: android.location

166. LocationManager Class
Methods
getAllProviders() getLastKnownLocation
getBestProvider() getProvider()
getGpsStatus() getProviders()
package: android.location

167. Google Location
Maps Services
External
Library
Android SDK

168. Google Maps
External Library
Use Google APIs add-on
Download Maps external library
Must register with Google Maps
service
Obtain a Maps API Key
Android SDK

169. AndroidManifest.xml
Declare Maps Library
Request internet permission
Hide title bar
<uses-library
android:name=”com.google.android.maps” />
<uses-permission
android:name=”android.permission.INTERNET” />
<activity android:name=”.HelloMaps”
android:label=”@string/app_name”
android:theme=”@android:style/Theme.NoTitleBar”>
Android SDK

170. res/layout/main.xml
<?xml version=”1.0” encoding=”utf-8”?>
<com.google.android.maps.MapView
xmlns:android=”http://schmas.android.com/
apk/res/android”
android:id=@+id/mapview”
android:layout_width=”fill_parent”
android:layout_height=”fill_parent”
android:clickable=”true”
android:apiKey=”Map API Key”
/>
Android SDK

171. + HelloMaps.java
public class HelloMaps extends MapActivity
@Override protected boolean isRouteDisplayed() {
return false;
}
@Override public void onCreate(Bundle
savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
}
MapView mapView = (MapView)
findViewById(R.id.mapview);
mapView.setBuiltInZoomControls(true);
Android SDK

172. Data
iPhone Screen Size

173. y axis x axis z axis
Accelerometer

174. TEXT Email
openURL: Method
To
From
CC
BCC
HTML Email
MessuageUI Framework
Attachment
Email

175. Push Notification

176. HTTP protocol
GET / POST
Data Store
.plist files
SQLite
CoreData
Network

177. Controller
Heliptor
Automobile
Bluetooth / WiFi

178. Yes Yes Yes
iPhone Cupcake 1.5 S40
OS 3.1 S60
Wireless Bluetooth

179. 30-pin dock connector
Point of Sale (POS)
Credit Card Terminal
External Accessory

181. Medical Tricorder

185. STANFORD UNIVERSITY
School of Engineering (EE46)
Engineering For Good
Save The World
Have Fun Doing It

186. Save The World
Malaria
TB
HIV

187. Augmented Reality (AR)
Camera
(Photo / Video)
Bar Code
Optical Character
Recognition
QR Code (OCR)
Scanner: 2D Objects

188. Detect Diseases
Scanner: Micro Life Forms

189. Identify Patients
Scanner: Body Parts

190. Scan
Diseases & Patients
Scanner: Life Forms

191. Challenges

194. Scanner: Mosquitoes

195. Q&A

196. @bess
Attendee Directory
http://www.slideshare.net/bess.ho