"In a world where mobile content is becoming more prevalent by the day, how can we enhance the experience of gathering images, audio and video for consumption on mobile devices? Brian will demonstrate how you can leverage the Wireless Universal Resource File (WURFL), PHP, ImageMagick, ffmpeg, and other open source tools to optimize content for consumption on mobile devices."

1. Rickroll To Go With PHP, WURFL, and
Other Open Source Tools
Brian DeShong
September 16, 2008

2. Intro
2

3. Intro
Brian DeShong
2

4. Intro
Brian DeShong
Senior Manager, Technology for Schematic
2

5. Intro
Brian DeShong
Senior Manager, Technology for Schematic
Application and systems architect
2

6. Intro
Brian DeShong
Senior Manager, Technology for Schematic
Application and systems architect
Developer
2

7. Who is Schematic?
3

8. Who is Schematic?
Full-service interactive agency
3

9. Who is Schematic?
Full-service interactive agency
Offices in Atlanta; New York; LA; San Francisco;
Austin, TX; Minneapolis, MN; San Jose, Costa Rica
3

10. Who is Schematic?
Full-service interactive agency
Offices in Atlanta; New York; LA; San Francisco;
Austin, TX; Minneapolis, MN; San Jose, Costa Rica
Some clients include...
Dell
Target
Disney 3
Nokia
Comcast
MTV

11. What are we talking about?
4

12. What are we talking about?
PC and mobile development are very different
4

13. What are we talking about?
PC and mobile development are very different
Identifying devices with WURFL
4

14. What are we talking about?
PC and mobile development are very different
Identifying devices with WURFL
Generating content for mobile consumption
Images, video, audio
PHP, open source tools
4

15. Computers vs. Mobile Devices
5

16. 6

17. 6

18. 6

19. 6

20. 6

21. Dial-up Broadband T1+, etc.
6

22. Dial-up Broadband T1+, etc.
6

23. 7

24. 7

25. 7

26. 7

27. 7
EDGE 3G EVDO

28. 7
EDGE 3G EVDO Proxy, no proxy

29. WURFL
8

30. What is it?
9

31. What is it?
Wireless Universal Resource File
9

32. What is it?
Wireless Universal Resource File
XML file containing characteristics of many mobile devices
9

33. What is it?
Wireless Universal Resource File
XML file containing characteristics of many mobile devices
Open source
9

34. What is it?
Wireless Universal Resource File
XML file containing characteristics of many mobile devices
Open source
http://wurfl.sourceforge.net/
9

35. WURFL Examples
10

36. WURFL Examples
10

37. WURFL Examples
10

38. WURFL Examples
10

39. WURFL Examples
10

40. How can we use WURFL?
11

41. How can we use WURFL?
Lookup device by user agent
11

42. How can we use WURFL?
Lookup device by user agent
Device data indicates its capabilities
iPhone
320x480
3GP, MP4 video @ 320x240, 30 fps
11

43. Data offered
12

44. Data offered
Display size
12

45. Data offered
Display size
User agent strings
12

46. Data offered
Display size
User agent strings
Supported audio, video, image formats
12

47. Data offered
Display size
User agent strings
Supported audio, video, image formats
Streaming capabilities
12

48. Data offered
Display size
User agent strings
Supported audio, video, image formats
Streaming capabilities
Has QWERTY keyboard?
12

49. Data offered
Display size
User agent strings
Supported audio, video, image formats
Streaming capabilities
Has QWERTY keyboard?
12
XHTML, Ajax capabilities

50. Gotchas
13

51. Gotchas
wurfl.xml is ~8 MB in size!
13

52. Gotchas
wurfl.xml is ~8 MB in size!
Not a great idea to parse it from front end
13

53. Gotchas
wurfl.xml is ~8 MB in size!
Not a great idea to parse it from front end
Extract data you need, place in database
Youʼll need a plan for updating from your source data
13

54. Use what you need
14

55. Use what you need
Display sizes
14

56. Use what you need
Display sizes
Relevant video, audio formats supported
14

57. Use what you need
Display sizes
Relevant video, audio formats supported
MIDP, CLDC versions supported
Handy in determining compatibility for applications
14

58. Use what you need
Display sizes
Relevant video, audio formats supported
MIDP, CLDC versions supported
Handy in determining compatibility for applications
Bitmasks values are handy
14

59. Applying bitmask and DB
<?php
define(‘VIDEO_MP4’, 1);
define(‘VIDEO_3GP’, 2);
define(‘VIDEO_WMV’, 4);
// $videoFormats = 7;
$videoFormats =
VIDEO_MP4 | VIDEO_3GP | VIDEO_WMV; 15
?>
id | manuf | model | video_formats
----------------------------------
1 | Nokia | N95 | 7

60. Using it from PHP
16

61. Using it from PHP
Itʼs just XML!
16

62. Using it from PHP
Itʼs just XML!
SimpleXML, DOM, etc.
16

63. Using it from PHP
Itʼs just XML!
SimpleXML, DOM, etc.
Caching your lookups
Memcached
SQLite, other DBs
16

64. Using it from PHP
Itʼs just XML!
SimpleXML, DOM, etc.
Caching your lookups
Memcached
SQLite, other DBs
Lookup by user agent
16
$_SERVER[‘HTTP_USER_AGENT’]
Opera: X-OperaMini-Phone-UA

65. Summary and other options
17

66. Summary and other options
Use WURFL responsibly and efficiently!
PHP scripts on its website
Employ some caching techniques
17

67. Summary and other options
Use WURFL responsibly and efficiently!
PHP scripts on its website
Employ some caching techniques
WURFL and other options are key to a great mobile
experience
DeviceAtlas
17

68. Mobile Content Challenges
18

69. Some challenges
19

70. Some challenges
Varying amounts of bandwidth available
19

71. Some challenges
Varying amounts of bandwidth available
Costly data plans
Unlimited vs. per-MB/GB
19

72. Some challenges
Varying amounts of bandwidth available
Costly data plans
Unlimited vs. per-MB/GB
Different capabilities across devices
19

73. Some challenges
Varying amounts of bandwidth available
Costly data plans
Unlimited vs. per-MB/GB
Different capabilities across devices
Limited memory in devices
19

74. How to workaround them
20

75. How to workaround them
Serve content tailored to userʼs device
Efficient use of resources and bandwidth
Determine if a given piece of content will work on the device
before download
20

76. How to workaround them
Serve content tailored to userʼs device
Efficient use of resources and bandwidth
Determine if a given piece of content will work on the device
before download
Hope for the best!
User will sort it out
It will work or it wonʼt
20

77. To be fair...
21

78. To be fair...
None of this may be relevant in a few years
21

79. To be fair...
None of this may be relevant in a few years
Mobile browsers and devices are evolving
21

80. To be fair...
None of this may be relevant in a few years
Mobile browsers and devices are evolving
Low-end devices are prevalent in many parts of the world
Who is your audience, and where are they located?
21

81. Tailoring Content for Devices
22

82. My Personal Approach
23

83. My Personal Approach
I pre-generate files and write them to disk
23

84. My Personal Approach
I pre-generate files and write them to disk
I can rely on them being in place when making decision
23

85. My Personal Approach
I pre-generate files and write them to disk
I can rely on them being in place when making decision
Images:
foo_240x320.jpg
foo_320x240.jpg
23

86. My Personal Approach
I pre-generate files and write them to disk
I can rely on them being in place when making decision
Images:
foo_240x320.jpg
foo_320x240.jpg
Video: 23
foo_176x144.3gp
foo_176x144.mp4

87. My Personal Approach
I pre-generate files and write them to disk
I can rely on them being in place when making decision
Images:
foo_240x320.jpg
foo_320x240.jpg
Video: 23
foo_176x144.3gp
foo_176x144.mp4
Audio:
foo.aac

88. Images
24

89. Optimizing image delivery
25

90. Optimizing image delivery
User visits site
25

91. Optimizing image delivery
User visits site
Device
identified?
25

92. Optimizing image delivery
User visits site
Device
identified?
No 25

93. Optimizing image delivery
User visits site
Device
identified?
No Yes 25

94. Optimizing images with PHP
26

95. Optimizing images with PHP
Graphics extensions available
GD
imagick (ImageMagick)
26

96. Optimizing images with PHP
Graphics extensions available
GD
imagick (ImageMagick)
Command-line ImageMagick binaries
26

97. Using imagick
$imagick = new Imagick();
$imagick->readImage(‘/astley/rick.jpg’);
27

98. Example
28

99. Example
Source Image: 507x398
28

100. Example
Thumbnailed image: 240x188
28

101. Example
Thumbnailed image: 240x188
28
$imagick->thumbnailImage(240, 0);

102. Example
Cropped thumbnail image: 240x320
28

103. Example
Cropped thumbnail image: 240x320
28
$imagick->cropThumbnailImage(240, 320);

104. Gotchas
29

105. Gotchas
Generating images on-demand is expensive
29

106. Gotchas
Generating images on-demand is expensive
Keep it cheap and/or infrequent
Write the different sizes to disk; routine cleanup
Memcached
Pre-generate them
29

107. Summary
30

108. Summary
Serving image tailored for device
Uses least amount of bandwidth possible
Uses device resources efficiently
30

109. Summary
Serving image tailored for device
Uses least amount of bandwidth possible
Uses device resources efficiently
Be smart with the server-side implementation!
30

110. Summary
Serving image tailored for device
Uses least amount of bandwidth possible
Uses device resources efficiently
Be smart with the server-side implementation!
Less complex than dealing with video, audio
30

111. Video
31

112. Video on mobile devices
32

113. Video on mobile devices
Most devices canʼt play the formats weʼre accustomed to
on PCs
32

114. Video on mobile devices
Most devices canʼt play the formats weʼre accustomed to
on PCs
3GP is widely supported
32

115. Video on mobile devices
Most devices canʼt play the formats weʼre accustomed to
on PCs
3GP is widely supported
MPEG-4 on newer devices
32

116. Optimizing video delivery
21
33

117. Optimizing video delivery
User visits site
21
33

118. Optimizing video delivery
User visits site
Device
supports
MPEG-4
video?
21
33

119. Optimizing video delivery
User visits site
Device
supports
MPEG-4
video?
No 21
33
Serve
3GP

120. Optimizing video delivery
User visits site
Device
supports
MPEG-4
video?
No Yes 21
33
Serve Serve
3GP MPEG-4

121. Optimizing video delivery
User visits site
Device
supports Max video
MPEG-4 resolution?
video?
No Yes 21
33
Serve Serve
3GP MPEG-4

122. Mobile Video Sites
34

123. Mobile Video Sites
34

124. Mobile Video Sites
34

125. Mobile Video Sites
34

126. Mobile Video Sites
34

127. Optimizing for mobile
35

128. Optimizing for mobile
Lower resolution
176x144, 320x240
35

129. Optimizing for mobile
Lower resolution
176x144, 320x240
Reduce video framerate
Playback looks jerky
But it requires less bandwidth!
35

130. Optimizing for mobile
Lower resolution
176x144, 320x240
Reduce video framerate
Playback looks jerky
But it requires less bandwidth!
Reduce bitrates
35

131. Tools
36

132. Tools
FFmpeg, Mencoder
36

133. Tools
FFmpeg, Mencoder
On2 Flix Engine for Flash (commercial)
36

134. Tools
FFmpeg, Mencoder
On2 Flix Engine for Flash (commercial)
Anystream Agility
36

135. Tools
FFmpeg, Mencoder
On2 Flix Engine for Flash (commercial)
Anystream Agility
Commercial tools to read Windows Media, other DRMʼd
formats
36

136. Expense
37

137. Expense
Generating videos is not cheap
37

138. Expense
Generating videos is not cheap
Should not be done in real time
End user will have to wait for it!
37

139. Expense
Generating videos is not cheap
Should not be done in real time
End user will have to wait for it!
Pre-generate with an offline process
Store them on disk
Purge when and if needed
37

140. Expense
Generating videos is not cheap
Should not be done in real time
End user will have to wait for it!
Pre-generate with an offline process
Store them on disk
Purge when and if needed
37
Weʼll look at FFmpeg shortly!

141. Audio
38

142. Audio on mobile devices
39

143. Audio on mobile devices
Devices have been playing MP3s for years now
39

144. Audio on mobile devices
Devices have been playing MP3s for years now
Wider support than video
39

145. Audio on mobile devices
Devices have been playing MP3s for years now
Wider support than video
MP3, AAC formats are well supported
39

146. Optimizing audio delivery
21
29
40

147. Optimizing audio delivery
User visits site
21
29
40

148. Optimizing audio delivery
User visits site
AAC
audio
supported?
21
29
40

149. Optimizing audio delivery
User visits site
AAC
audio
supported?
No 21
29
40
Serve
MP3

150. Optimizing audio delivery
User visits site
AAC
audio
supported?
No Yes 21
29
40
Serve Serve
MP3 AAC

151. Tools
41

152. Tools
FFmpeg
41

153. Tools
FFmpeg
TiMidity++ (MIDI to PCM)
41

154. Tools
FFmpeg
TiMidity++ (MIDI to PCM)
Note:
Iʼm not dealing with MP3 licensing issues here!
41

155. Expense
42

156. Expense
Again, generating audio isnʼt cheap
42

157. Expense
Again, generating audio isnʼt cheap
Shouldnʼt transcode in real time
42

158. Expense
Again, generating audio isnʼt cheap
Shouldnʼt transcode in real time
Pre-generate in an offline process
42

159. Expense
Again, generating audio isnʼt cheap
Shouldnʼt transcode in real time
Pre-generate in an offline process
42

160. Expense
Again, generating audio isnʼt cheap
Shouldnʼt transcode in real time
Pre-generate in an offline process
What can we use?
Surprise! FFmpeg! 42

161. What’s this FFmpeg stuff?
43

162. Basics
44

163. Basics
Decodes and encodes audio and video streams
44

164. Basics
Decodes and encodes audio and video streams
Many A/V formats supported
44

165. Basics
Decodes and encodes audio and video streams
Many A/V formats supported
Supports use of external libraries to handle additional
formats
44

166. Basics
Decodes and encodes audio and video streams
Many A/V formats supported
Supports use of external libraries to handle additional
formats
Runs on many platforms (Windows, OS X, Linux)
44

167. Usage
ffmpeg [[infile options] -i infile]...
{[outfile options] outfile}...
45

168. Usage
ffmpeg [[infile options] -i infile]...
{[outfile options] outfile}...
45
ffmpeg -h: 504 lines!
Extensive amount of options for both audio and
video

169. Usage
ffmpeg [[infile options] -i infile]...
{[outfile options] outfile}...
45
ffmpeg -h: 504 lines!
Extensive amount of options for both audio and
video

170. FFmpeg: Video
46

171. Notable video encoding options
47

172. Notable video encoding options
-acodec: audio codec to use
47

173. Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
47

174. Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
-s: frame size
47

175. Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
-s: frame size
-r: video framerate
47

176. Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
-s: frame size
-r: video framerate
-b: bitrate
47

177. Notable video encoding options
-acodec: audio codec to use
-vcodec: video codec to use
-s: frame size
-r: video framerate
-b: bitrate
47
ffmpeg -i rr.mp4
-acodec libfaac
-vcodec h263
-s 176x144
-r 12
foo.3gp

178. Examples
48

179. Examples
48

180. Examples
48
320x240
MPEG-4, AAC
30 fps
244kbps
6.2 MB

181. Examples
48
320x240 176x144
MPEG-4, AAC 3GP, AAC
30 fps 30 fps
244kbps 90kbps
6.2 MB 2.3 MB

182. Examples
48
320x240 176x144 176x144
MPEG-4, AAC 3GP, AAC 3GP, AAC
30 fps 30 fps 8 fps
244kbps 90kbps 63kbps
6.2 MB 2.3 MB 1.6 MB

183. Grab screenshots!
$ ffmpeg
-i rr.mp4
-f image2
49
frame%2d.jpg

184. 50

185. 50

186. FFmpeg: Audio
51

187. Notable audio encoding options
52

188. Notable audio encoding options
-acodec: audio codec to use
52

189. Notable audio encoding options
-acodec: audio codec to use
-ar: audio sampling rate
52

190. Notable audio encoding options
-acodec: audio codec to use
-ar: audio sampling rate
-ab: audio bitrate
52

191. Notable audio encoding options
-acodec: audio codec to use
-ar: audio sampling rate
-ab: audio bitrate
-ac: number of channels
52

192. Notable audio encoding options
-acodec: audio codec to use
-ar: audio sampling rate
-ab: audio bitrate
-ac: number of channels
52
ffmpeg -i iib.mp3
-ab 64000
-ac 1
foo.mp3

193. Examples
53

194. Examples
53
Original:
128kbps
4.1MB

195. Examples
53
Original: Second Clip:
128kbps 64kbps
4.1MB 2.05MB (1/2)

196. Examples
53
Original: Second Clip: Third Clip:
128kbps 64kbps 32kbps
4.1MB 2.05MB (1/2) 1.025MB (1/4)

197. Examples
53
Original: Second Clip: Third Clip: Final Clip:
128kbps 64kbps 32kbps 8kbps
4.1MB 2.05MB (1/2) 1.025MB (1/4) 1.025MB (???)

198. FFmpeg Summary
54

199. FFmpeg Summary
Powerful tool!
54

200. FFmpeg Summary
Powerful tool!
Takes a lot to master it
Probably makes more sense to A/V experts
54

201. FFmpeg Summary
Powerful tool!
Takes a lot to master it
Probably makes more sense to A/V experts
But itʼs easy enough to get the job done right
54

202. FFmpeg Summary
Powerful tool!
Takes a lot to master it
Probably makes more sense to A/V experts
But itʼs easy enough to get the job done right
Its manpage is your friend!
54

203. In Closing...
55

204. Mobile is tricky!
56

205. Mobile is tricky!
All of those combinations
Devices
Providers
Bandwidth / data plans
Capabilities
56

206. Mobile is tricky!
All of those combinations
Devices
Providers
Bandwidth / data plans
Capabilities
Use resources efficiently
56

207. Mobile is tricky!
All of those combinations
Devices
Providers
Bandwidth / data plans
Capabilities
Use resources efficiently
Optimize the experience for the end user 56

208. Resources
57

209. Resources
WURFL: http://wurfl.sourceforge.net/
57

210. Resources
WURFL: http://wurfl.sourceforge.net/
FFmpeg: http://ffmpeg.mplayerhq.hu/
57

211. Resources
WURFL: http://wurfl.sourceforge.net/
FFmpeg: http://ffmpeg.mplayerhq.hu/
ImageMagick: http://www.imagemagick.org/
PECL extension: http://pecl.php.net/package/imagick/
57

212. We’re Hiring!
It goes without saying: Schematic is only as good
as the people who work here. That’s why we’re so
particular about recruiting, training, nurturing, and
retaining the very best people in our field.
If you have digital expertise (technical, creative, managerial–or
something else entirely), enthusiasm, curiosity, and the 58
ability to collaborate with others, we’d love to hear from you.
For more information, please see me or visit
http://www.schematic.com/#/JoinUs/

213. Thanks!
http://www.deshong.net/
http://www.schematic.com/
brian@deshong.net 59
bdeshong@schematic.com

214. Thanks!
59
http://www.deshong.net/
http://www.schematic.com/
brian@deshong.net
bdeshong@schematic.com