Rickroll To Go With PHP, WURFL, and Other Open Source Tools

Rickroll To Go With PHP, WURFL, and Other Open Source Tools Brian DeShong September 16, 2008

Intro 2

Intro Brian DeShong 2

Intro Brian DeShong Senior Manager, Technology for Schematic 2

Intro Brian DeShong Senior Manager, Technology for Schematic Application and systems architect 2

Intro Brian DeShong Senior Manager, Technology for Schematic Application and systems architect Developer 2

Who is Schematic? 3

Who is Schematic? Full-service interactive agency 3

Who is Schematic? Full-service interactive agency Ofﬁces in Atlanta; New York; LA; San Francisco; Austin, TX; Minneapolis, MN; San Jose, Costa Rica 3

Who is Schematic? Full-service interactive agency Ofﬁces in Atlanta; New York; LA; San Francisco; Austin, TX; Minneapolis, MN; San Jose, Costa Rica Some clients include... Dell Target Disney 3 Nokia Comcast MTV

What are we talking about? 4

What are we talking about? PC and mobile development are very different 4

What are we talking about? PC and mobile development are very different Identifying devices with WURFL 4

What are we talking about? PC and mobile development are very different Identifying devices with WURFL Generating content for mobile consumption Images, video, audio PHP, open source tools 4

Computers vs. Mobile Devices 5

6

Dial-up Broadband T1+, etc. 6

7

7 EDGE 3G EVDO

7 EDGE 3G EVDO Proxy, no proxy

WURFL 8

What is it? 9

What is it? Wireless Universal Resource File 9

What is it? Wireless Universal Resource File XML ﬁle containing characteristics of many mobile devices 9

What is it? Wireless Universal Resource File XML ﬁle containing characteristics of many mobile devices Open source 9

What is it? Wireless Universal Resource File XML ﬁle containing characteristics of many mobile devices Open source http://wurﬂ.sourceforge.net/ 9

WURFL Examples 10

How can we use WURFL? 11

How can we use WURFL? Lookup device by user agent 11

How can we use WURFL? Lookup device by user agent Device data indicates its capabilities iPhone 320x480 3GP, MP4 video @ 320x240, 30 fps 11

Data offered 12

Data offered Display size 12

Data offered Display size User agent strings 12

Data offered Display size User agent strings Supported audio, video, image formats 12

Data offered Display size User agent strings Supported audio, video, image formats Streaming capabilities 12

Data offered Display size User agent strings Supported audio, video, image formats Streaming capabilities Has QWERTY keyboard? 12

Data offered Display size User agent strings Supported audio, video, image formats Streaming capabilities Has QWERTY keyboard? 12 XHTML, Ajax capabilities

Gotchas 13

Gotchas wurﬂ.xml is ~8 MB in size! 13

Gotchas wurﬂ.xml is ~8 MB in size! Not a great idea to parse it from front end 13

Gotchas wurﬂ.xml is ~8 MB in size! Not a great idea to parse it from front end Extract data you need, place in database Youʼll need a plan for updating from your source data 13

Use what you need 14

Use what you need Display sizes 14

Use what you need Display sizes Relevant video, audio formats supported 14

Use what you need Display sizes Relevant video, audio formats supported MIDP, CLDC versions supported Handy in determining compatibility for applications 14

Use what you need Display sizes Relevant video, audio formats supported MIDP, CLDC versions supported Handy in determining compatibility for applications Bitmasks values are handy 14

Using it from PHP 16

Using it from PHP Itʼs just XML! 16

Using it from PHP Itʼs just XML! SimpleXML, DOM, etc. 16

Using it from PHP Itʼs just XML! SimpleXML, DOM, etc. Caching your lookups Memcached SQLite, other DBs 16

Using it from PHP Itʼs just XML! SimpleXML, DOM, etc. Caching your lookups Memcached SQLite, other DBs Lookup by user agent 16 $_SERVER[‘HTTP_USER_AGENT’] Opera: X-OperaMini-Phone-UA

Summary and other options 17

Summary and other options Use WURFL responsibly and efﬁciently! PHP scripts on its website Employ some caching techniques 17

Summary and other options Use WURFL responsibly and efﬁciently! PHP scripts on its website Employ some caching techniques WURFL and other options are key to a great mobile experience DeviceAtlas 17

Mobile Content Challenges 18

Some challenges 19

Some challenges Varying amounts of bandwidth available 19

Some challenges Varying amounts of bandwidth available Costly data plans Unlimited vs. per-MB/GB 19

Some challenges Varying amounts of bandwidth available Costly data plans Unlimited vs. per-MB/GB Different capabilities across devices 19

Some challenges Varying amounts of bandwidth available Costly data plans Unlimited vs. per-MB/GB Different capabilities across devices Limited memory in devices 19

How to workaround them 20

How to workaround them Serve content tailored to userʼs device Efﬁcient use of resources and bandwidth Determine if a given piece of content will work on the device before download 20

How to workaround them Serve content tailored to userʼs device Efﬁcient use of resources and bandwidth Determine if a given piece of content will work on the device before download Hope for the best! User will sort it out It will work or it wonʼt 20

To be fair... 21

To be fair... None of this may be relevant in a few years 21

To be fair... None of this may be relevant in a few years Mobile browsers and devices are evolving 21

To be fair... None of this may be relevant in a few years Mobile browsers and devices are evolving Low-end devices are prevalent in many parts of the world Who is your audience, and where are they located? 21

Tailoring Content for Devices 22

My Personal Approach 23

My Personal Approach I pre-generate ﬁles and write them to disk 23

My Personal Approach I pre-generate ﬁles and write them to disk I can rely on them being in place when making decision 23

My Personal Approach I pre-generate ﬁles and write them to disk I can rely on them being in place when making decision Images: foo_240x320.jpg foo_320x240.jpg 23

My Personal Approach I pre-generate ﬁles and write them to disk I can rely on them being in place when making decision Images: foo_240x320.jpg foo_320x240.jpg Video: 23 foo_176x144.3gp foo_176x144.mp4

My Personal Approach I pre-generate ﬁles and write them to disk I can rely on them being in place when making decision Images: foo_240x320.jpg foo_320x240.jpg Video: 23 foo_176x144.3gp foo_176x144.mp4 Audio: foo.aac

Images 24

Optimizing image delivery 25

Optimizing image delivery User visits site 25

Optimizing image delivery User visits site Device identiﬁed? 25

Optimizing image delivery User visits site Device identiﬁed? No 25

Optimizing image delivery User visits site Device identiﬁed? No Yes 25

Optimizing images with PHP 26

Optimizing images with PHP Graphics extensions available GD imagick (ImageMagick) 26

Optimizing images with PHP Graphics extensions available GD imagick (ImageMagick) Command-line ImageMagick binaries 26

Using imagick $imagick = new Imagick(); $imagick->readImage(‘/astley/rick.jpg’); 27

Example 28

Example Source Image: 507x398 28

Example Thumbnailed image: 240x188 28

Example Thumbnailed image: 240x188 28 $imagick->thumbnailImage(240, 0);

Example Cropped thumbnail image: 240x320 28

Example Cropped thumbnail image: 240x320 28 $imagick->cropThumbnailImage(240, 320);

Gotchas 29

Gotchas Generating images on-demand is expensive 29

Gotchas Generating images on-demand is expensive Keep it cheap and/or infrequent Write the different sizes to disk; routine cleanup Memcached Pre-generate them 29

Summary 30

Summary Serving image tailored for device Uses least amount of bandwidth possible Uses device resources efﬁciently 30

Summary Serving image tailored for device Uses least amount of bandwidth possible Uses device resources efﬁciently Be smart with the server-side implementation! 30

Summary Serving image tailored for device Uses least amount of bandwidth possible Uses device resources efﬁciently Be smart with the server-side implementation! Less complex than dealing with video, audio 30

Video 31

Video on mobile devices 32

Video on mobile devices Most devices canʼt play the formats weʼre accustomed to on PCs 32

Video on mobile devices Most devices canʼt play the formats weʼre accustomed to on PCs 3GP is widely supported 32

Video on mobile devices Most devices canʼt play the formats weʼre accustomed to on PCs 3GP is widely supported MPEG-4 on newer devices 32

Optimizing video delivery 21 33

Optimizing video delivery User visits site 21 33

Optimizing video delivery User visits site Device supports MPEG-4 video? 21 33

Optimizing video delivery User visits site Device supports MPEG-4 video? No 21 33 Serve 3GP

Optimizing video delivery User visits site Device supports MPEG-4 video? No Yes 21 33 Serve Serve 3GP MPEG-4

Optimizing video delivery User visits site Device supports Max video MPEG-4 resolution? video? No Yes 21 33 Serve Serve 3GP MPEG-4

Mobile Video Sites 34

Optimizing for mobile 35

Optimizing for mobile Lower resolution 176x144, 320x240 35

Optimizing for mobile Lower resolution 176x144, 320x240 Reduce video framerate Playback looks jerky But it requires less bandwidth! 35

Optimizing for mobile Lower resolution 176x144, 320x240 Reduce video framerate Playback looks jerky But it requires less bandwidth! Reduce bitrates 35

Tools 36

Tools FFmpeg, Mencoder 36

Tools FFmpeg, Mencoder On2 Flix Engine for Flash (commercial) 36

Tools FFmpeg, Mencoder On2 Flix Engine for Flash (commercial) Anystream Agility 36

Tools FFmpeg, Mencoder On2 Flix Engine for Flash (commercial) Anystream Agility Commercial tools to read Windows Media, other DRMʼd formats 36

Expense 37

Expense Generating videos is not cheap 37

Expense Generating videos is not cheap Should not be done in real time End user will have to wait for it! 37

Expense Generating videos is not cheap Should not be done in real time End user will have to wait for it! Pre-generate with an ofﬂine process Store them on disk Purge when and if needed 37

Expense Generating videos is not cheap Should not be done in real time End user will have to wait for it! Pre-generate with an ofﬂine process Store them on disk Purge when and if needed 37 Weʼll look at FFmpeg shortly!

Audio 38

Audio on mobile devices 39

Audio on mobile devices Devices have been playing MP3s for years now 39

Audio on mobile devices Devices have been playing MP3s for years now Wider support than video 39

Audio on mobile devices Devices have been playing MP3s for years now Wider support than video MP3, AAC formats are well supported 39

Optimizing audio delivery 21 29 40

Optimizing audio delivery User visits site 21 29 40

Optimizing audio delivery User visits site AAC audio supported? 21 29 40

Optimizing audio delivery User visits site AAC audio supported? No 21 29 40 Serve MP3

Optimizing audio delivery User visits site AAC audio supported? No Yes 21 29 40 Serve Serve MP3 AAC

Tools 41

Tools FFmpeg 41

Tools FFmpeg TiMidity++ (MIDI to PCM) 41

Tools FFmpeg TiMidity++ (MIDI to PCM) Note: Iʼm not dealing with MP3 licensing issues here! 41

Expense 42

Expense Again, generating audio isnʼt cheap 42

Expense Again, generating audio isnʼt cheap Shouldnʼt transcode in real time 42

Expense Again, generating audio isnʼt cheap Shouldnʼt transcode in real time Pre-generate in an ofﬂine process 42

Expense Again, generating audio isnʼt cheap Shouldnʼt transcode in real time Pre-generate in an ofﬂine process What can we use? Surprise! FFmpeg! 42

What’s this FFmpeg stuff? 43

Basics 44

Basics Decodes and encodes audio and video streams 44

Basics Decodes and encodes audio and video streams Many A/V formats supported 44

Basics Decodes and encodes audio and video streams Many A/V formats supported Supports use of external libraries to handle additional formats 44

Basics Decodes and encodes audio and video streams Many A/V formats supported Supports use of external libraries to handle additional formats Runs on many platforms (Windows, OS X, Linux) 44

Usage ffmpeg [[infile options] -i infile]... {[outfile options] outfile}... 45

Usage ffmpeg [[infile options] -i infile]... {[outfile options] outfile}... 45 ffmpeg -h: 504 lines! Extensive amount of options for both audio and video

FFmpeg: Video 46

Notable video encoding options 47

Notable video encoding options -acodec: audio codec to use 47

Notable video encoding options -acodec: audio codec to use -vcodec: video codec to use 47

Notable video encoding options -acodec: audio codec to use -vcodec: video codec to use -s: frame size 47

Notable video encoding options -acodec: audio codec to use -vcodec: video codec to use -s: frame size -r: video framerate 47

Notable video encoding options -acodec: audio codec to use -vcodec: video codec to use -s: frame size -r: video framerate -b: bitrate 47

Notable video encoding options -acodec: audio codec to use -vcodec: video codec to use -s: frame size -r: video framerate -b: bitrate 47 ffmpeg -i rr.mp4 \\ -acodec libfaac \\ -vcodec h263 \\ -s 176x144 \\ -r 12 \\ foo.3gp

Examples 48

Examples 48 320x240 MPEG-4, AAC 30 fps 244kbps 6.2 MB

Examples 48 320x240 176x144 MPEG-4, AAC 3GP, AAC 30 fps 30 fps 244kbps 90kbps 6.2 MB 2.3 MB

Examples 48 320x240 176x144 176x144 MPEG-4, AAC 3GP, AAC 3GP, AAC 30 fps 30 fps 8 fps 244kbps 90kbps 63kbps 6.2 MB 2.3 MB 1.6 MB

Grab screenshots! $ ffmpeg \\ -i rr.mp4 \\ -f image2 \\ 49 frame%2d.jpg

50

FFmpeg: Audio 51

Notable audio encoding options 52

Notable audio encoding options -acodec: audio codec to use 52

Notable audio encoding options -acodec: audio codec to use -ar: audio sampling rate 52

Notable audio encoding options -acodec: audio codec to use -ar: audio sampling rate -ab: audio bitrate 52

Notable audio encoding options -acodec: audio codec to use -ar: audio sampling rate -ab: audio bitrate -ac: number of channels 52

Notable audio encoding options -acodec: audio codec to use -ar: audio sampling rate -ab: audio bitrate -ac: number of channels 52 ffmpeg -i iib.mp3 \\ -ab 64000 \\ -ac 1 \\ foo.mp3

Examples 53

Examples 53 Original: 128kbps 4.1MB

Examples 53 Original: Second Clip: 128kbps 64kbps 4.1MB 2.05MB (1/2)

Examples 53 Original: Second Clip: Third Clip: 128kbps 64kbps 32kbps 4.1MB 2.05MB (1/2) 1.025MB (1/4)

Examples 53 Original: Second Clip: Third Clip: Final Clip: 128kbps 64kbps 32kbps 8kbps 4.1MB 2.05MB (1/2) 1.025MB (1/4) 1.025MB (???)

FFmpeg Summary 54

FFmpeg Summary Powerful tool! 54

FFmpeg Summary Powerful tool! Takes a lot to master it Probably makes more sense to A/V experts 54

FFmpeg Summary Powerful tool! Takes a lot to master it Probably makes more sense to A/V experts But itʼs easy enough to get the job done right 54

FFmpeg Summary Powerful tool! Takes a lot to master it Probably makes more sense to A/V experts But itʼs easy enough to get the job done right Its manpage is your friend! 54

In Closing... 55

Mobile is tricky! 56

Mobile is tricky! All of those combinations Devices Providers Bandwidth / data plans Capabilities 56

Mobile is tricky! All of those combinations Devices Providers Bandwidth / data plans Capabilities Use resources efﬁciently 56

Mobile is tricky! All of those combinations Devices Providers Bandwidth / data plans Capabilities Use resources efﬁciently Optimize the experience for the end user 56

Resources 57

Resources WURFL: http://wurﬂ.sourceforge.net/ 57

Resources WURFL: http://wurﬂ.sourceforge.net/ FFmpeg: http://ffmpeg.mplayerhq.hu/ 57

Resources WURFL: http://wurﬂ.sourceforge.net/ FFmpeg: http://ffmpeg.mplayerhq.hu/ ImageMagick: http://www.imagemagick.org/ PECL extension: http://pecl.php.net/package/imagick/ 57

We’re Hiring! It goes without saying: Schematic is only as good as the people who work here. That’s why we’re so particular about recruiting, training, nurturing, and retaining the very best people in our ﬁeld. If you have digital expertise (technical, creative, managerial–or something else entirely), enthusiasm, curiosity, and the 58 ability to collaborate with others, we’d love to hear from you. For more information, please see me or visit http://www.schematic.com/#/JoinUs/

Thanks! http://www.deshong.net/ http://www.schematic.com/ brian@deshong.net 59 bdeshong@schematic.com

Thanks! 59 http://www.deshong.net/ http://www.schematic.com/ brian@deshong.net bdeshong@schematic.com

Rickroll To Go With PHP, WURFL, and Other Open Source Tools

0 comments

Notes on slide 1

1 Favorite