Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Writing and using php streams and sockets

46,763 views

Published on

Slides from my tutorial at Zendcon 2011 on PHP streams, sockets and filters

Published in: Technology
  • Be the first to comment

Writing and using php streams and sockets

  1. 1.  Idea originating in 1950’s  Standard way to get Input and Output A source or sink of dataDefinitions   C – stdin, stderr, stdout  C++ iostreamWho uses them  Perl IO  Python io  Java  C#
  2. 2.  We will write code (in pairs) There will be a quiz You get out what you put in – participate! This is really 3 talks in one - Streams Filters Sockets
  3. 3. These are the people you can blame for the PHP streams implementation
  4. 4. Abstracting I/O
  5. 5.  Access input and output generically Can write and read linearly May or may not be seekable Comes in chunks of data
  6. 6.  EVERYTHING include/require _once stream functions file system functions many other extensions
  7. 7. Stream Contexts Stream StreamALL IO Filter Wrapper
  8. 8.  Stream › Resource that exhibits a flow or succession of data Wrapper › Tells a stream how to handle specific protocols and encodings Context › A set of parameters and options to tell a stream (or filter) how to behave
  9. 9.  Scheme › The name of the wrapper to be used. file, http, https, ftp, etc. Target › Depends on the wrapper, filesystem uses a string path name, ssh2 uses a PHP resource home/bar/foo.txt file:///home/bar/foo.txt http://www.example.com/foo.txt ftp://user:pass@ftp.example.com/foo.txt php://filter/read=string.toupper|string.rot13/resource =http://www.example.com
  10. 10.  flock wrapper limitations non-existent pointers (infinite loops can and will happen) error handling
  11. 11.  Parameters Options Modify or enhance a stream stream_context_set_param stream_context_set_option stream_context_create
  12. 12.  stream_set_read_buffer stream_context_get_params stream_resolve_include_path stream_supports_lock
  13. 13.  Open_basedir allow_url_fopen allow_url_include
  14. 14. 1. Get a Feed from a webservice (I’m using flickr) - http://www.flickr.com/services/feeds/2. No curl installed, allow_url_fopen is on3. Display however you would like
  15. 15. Grab the feed as xml, pop it through simplexml, loop and display the output to a webpage
  16. 16. Streams available by default
  17. 17.  file:// http:// ftp:// data:// glob://
  18. 18.  SSL  Phar › https:// › phar:// › ftps://  Zlib › ssl:// › compress.zlib:// › tls:// › zlib:// SSH  Bzip › ssh2.shell:// › compress.bz2:// › ssh2.exec:// › ssh2.tunnel:// › ssh2.sftp:// › ssh2.scp://
  19. 19.  Pipes STDIN, STDOUT, STDERR proc_open popen
  20. 20.  php://stdin php://stdout php://stderr
  21. 21.  php://output php://input php://fd
  22. 22.  php://filter (5.0.0) php://memory (5.1.0) php://temp (5.1.0)
  23. 23. 1. Talk to a command line program on your system (I’m using hg)2. Make it do something and read the output into a variable
  24. 24. Talk to hg, get the data, make an array of branches
  25. 25. Userland Streams
  26. 26.  There are no interfaces Implement as though there were an interface Seekable is optional Flushable is optional Directory support is optional and there’s even more available
  27. 27.  stream_open stream_read stream_write stream_eof stream_close stream_stat
  28. 28.  fopen  file_get_contentsInformation  Return true or false  $this->context will have any context metadataCode
  29. 29.  fread  fgets  file_get_contentsInformation  etc…  Return string data or false  $this->context will have any context metadataCode
  30. 30.  fwrite  file_put_contentsInformation  get in a string of data to deal with  return how many bytes you wroteCode
  31. 31.  feof  file_get_contents  freadInformation  etc…  Return true or false  $this->context will have any context metadataCode
  32. 32.  fclose  file_get_contentsInformation  Don’t return anything  any cleanup should go hereCode
  33. 33.  fstat  file_get_contents file_put_contentsInformation Code
  34. 34.  fstat calls stream_stat and should always be implemented EVERYTHING ELSE uses url_stat Good idea to do both Return an array of data identical to stat()
  35. 35.  Called for just about everything  fileperms() fileinode() filesize() fileowner() filegroup() fileatime()Information filemtime() filectime() filetype() is_writable() is_readable() is_executable() is_file() is_dir() is_link()Code
  36. 36.  stream_seek stream_tell
  37. 37.  fseek()  isn’t always called – php streams have read buffering on by defaultInformationCode (stream_set_read_buffer() in 5.3.3 +)
  38. 38.  fseek()  This one is ALWAYS called to get the current position, even if stream_seek isInformationCode NOT
  39. 39.  stream_flush rename unlink stream_lock
  40. 40.  fflush()  return true if data was stored, OR there was no data to store, otherwise returnInformationCode false
  41. 41.  called on rename  no real caveats, works as describedInformationCode
  42. 42.  called on unlink  works just as describedInformationCode
  43. 43.  flock()  file_put_contents() LOCK_EX stream_set_blocking()Information   on stream closeCode
  44. 44.  mkdir rmdir dir_closedir dir_opendir dir_readdir dir_rewinddir
  45. 45.  mkdir()  watch for the recursive flag in the options bitmaskInformationCode
  46. 46.  rmdir()  watch for the recursive flag in the options bitmaskInformationCode
  47. 47.  opendir()  option is “enforce safe mode” (which you should just ignore)In formationCode
  48. 48.  closedir()  all resource locked, allocated, created should be releasedInformationCode
  49. 49.  readdir()  should return a string filename or false return value is cast to a string!Information Code
  50. 50.  rewinddir()  should reset the output generated by readdir, next call should retrieve the firstInformationCode entry
  51. 51.  stream_set_option() 5.3 stream_cast() 5.3 stream_metadata() 5.4 › chown() › chmod() › chgrp() › touch()
  52. 52. Create a custom wrapper for any kind of network or shared memory storage you want (except s3 which is way overdone)Implement at least the basics – extra points for more features
  53. 53. Wrapper for wincache (btw, porting this to apc would be a breeze)Does basics + seek + flush (since we’re caching) + rename + unlinkNo directories
  54. 54. Use Case land – when streamsmake sense
  55. 55.  Data in s3 Data locally during development Easy switch out if alternative storage is ever desired Storing image files
  56. 56.  Existing Zend Framework Code Register the s3:// wrapper Use a configuration setting for the stream to use for all images on the system
  57. 57.  Store and edit template files in a database Have the snappiness of including from disk Minimal Configuration
  58. 58.  db:// stream simple stream wrapper that looks for the template in the db, and writes it to the filesystem before returning the data The cached location is FIRST in the include path, so if it fails, the db stream gets hit
  59. 59.  Talk to mercurial (hg binary) Talk to git communicates via command line Use pipes to get data out into format for use in system
  60. 60.  Use proc_open to keep a pipe to the binary going Pass commands through stdin pipe as necessary Abstract this out to other binaries that are used by the system
  61. 61.  Test filesystem related functionality in simpletest or phpunit without actually touching the filesystem
  62. 62.  A stream wrapper that mocks all wrappable file system functionality that can be used with any testing system
  63. 63. 1. Name 3 Built in PHP streams.2. What is a Context? A Wrapper? A Stream?3. How do you identify a stream?4. Name two extensions that provide additional PHP streams.
  64. 64.  http://php.net/streams http://php.net/filesystem http://ciaranmcnulty.com/blog/2009/04/ simplifying-file-operations-using-php- stream-wrappers
  65. 65. Because everyone needs to rot13a file they open on the fly
  66. 66.  Performs operations on stream data Can be prepended or appended (even on the fly) Can be attached to read or write When a filter is added for read and write, two instances of the filter are created.
  67. 67.  Data has an input and output state When reading in chunks, you may need to cache in between reads to make filters useful Use the right tool for the job
  68. 68.  string filters › string.rot13 › string.toupper › string.tolower › string.strip_tags
  69. 69.  convert filters › convert.*  base64-encode  base64-decode  quoted-printable-encode  quoted-printable-decode dechunk › decode remote HTTP chunked encoding streams consumed › eats data (that’s all it does)
  70. 70.  bzip.compress and bzip.compress convert.iconv.* zlib.inflate and zlib.deflate mcrypt.* and mdecrypt.*
  71. 71.  Given a list of bad words (we’ll use the 7 bad words), write a filter to remove them from text Given a commonly misspelled word, fix the spelling in the text
  72. 72.  I only did the bad words one, and created an extremely naïve regex that stripped the “bad” words from whatever text it finds. It buffers the entirety of the text before doing the replacement – this could be done by looking for word boundaries and doing it piecemeal to improve performance
  73. 73. Manipulate data on the fly
  74. 74.  Extend an internal class php_user_filter It’s not abstract… Yes that’s a horrible name Remember this pre-dates php 5.0 decisions Note the method names are camelcased…
  75. 75.  onCreate  basically a constructor Called every time PHP needs a new filterInformation  (on every stream)  return true or falseCode
  76. 76.  php_user_filter › $this->filtername › $this->params › $this->stream
  77. 77.  onClose  basically a destructorInformation  no returnCode
  78. 78.  MUST return › PSFS_PASS_ON › PSFS_FEED_MEInformation › PSFS_ERR_FATAL  You get buckets of data and do stuff to themCode
  79. 79.  $in and $out are “bucket brigades” containing opaque “buckets” of data You can only touch buckets and brigades with the stream_bucket_* functions You get a bucket using stream_bucket_make_writeable
  80. 80.  Given a list of bad words (we’ll use the 7 bad words), write a filter to remove them from text Given a commonly misspelled word, fix the spelling in the text
  81. 81.  I only did the bad words one, and created an extremely naïve regex that stripped the “bad” words from whatever text it finds. It buffers the entirety of the text before doing the replacement – this could be done by looking for word boundaries and doing it piecemeal to improve performance
  82. 82. Use Case land – when filtersmake sense
  83. 83.  Need to encrypt files uploaded by a user to add an extra security layer Encryption needs to happen before it goes “on the wire” Don’t want to have encryption for all files – don’t need overhead of adapter system
  84. 84.  Use mcrypt’s stream filtering Attach the mcrypt filter to the write stream when uploading data Attach the mcrypt filter to the read stream when downloading the data
  85. 85.  Upload documents to remote storage taking least amount of space possible in transmission
  86. 86.  Use PHP’s compressions filters (bz2, zlib) in conjunction with php’s streams that allow to compression to crush every last bit of space out of the files
  87. 87. 1. What term is used to describe data handling in filters?2. Name two built in filters you should be using and why.3. What is the default mode filters are appended with?4. What is the difference between appending and prepending a filter?
  88. 88.  http://php.net/streams http://php.net/filters
  89. 89. Putter about the network with me
  90. 90.  Socket › Bidirectional network stream that speaks a protocol Transport › Tells a network stream how to communicate Wrapper › Tells a stream how to handle specific protocols and encodings
  91. 91.  Network Stream, Network Transport, Socket Transport Slightly different behavior from a file stream Bi-directional data
  92. 92.  Sockets block › stream_set_blocking › stream_set_timeout › stream_select feof means “connection_closed”? huge reads or writes (think 8K) stream_get_meta_data is READ ONLY
  93. 93.  New APIS in streams and filesystem functions are replacements Extension is old and not always kept up to date (bit rot) Extension is very low level stream_socket_server stream_socket_client
  94. 94.  tcp udp unix udg SSL extension › ssl › sslv2 › sslv3 › tls
  95. 95.  Given two files of quotes, write a server with PHP streams methods to return random quotes, optionally allow client to pick which file to retrieve quotes from Given a simple tcp server that will return a random quote – make a client that pulls quotes for users
  96. 96.  socket_server.php – uses stream_socket_server and stream_accept socket_client.php – uses stream_socket_client and a bit of loop magic to make a little “cli” app
  97. 97. Use Case land – when socketsmake sense
  98. 98.  Send email using an external smtp server with special requirements for both authentication and encryption
  99. 99.  Simple smtp class, using PHP sockets to communicate with the server and pass the mail on, complete with tls encryption
  100. 100.  Push data from a PHP script into flash player with any additional extensions or tools
  101. 101.  http://ria.dzone.com/articles/php-and- flex-sockets Use sockets to talk directly to the flash player
  102. 102. 1. How is a socket different from a stream?2. What transports does PHP provide by default?3. Name the two most commonly used stream socket functions.4. How can a user add additional transports to PHP?
  103. 103.  http://php.net/streams http://php.net/transports http://wikipedia.org/wiki/Unix_domain_s ocket http://tools.ietf.org/html/rfc1122
  104. 104. Does your brain feel like mush yet?
  105. 105.  Stream › Resource that exhibits a flow or succession of data  WrapperStreams › Tells a stream how to handle specific protocols and encodings  Context › A set of parameters and options to tell a stream (or filter) how to behave  Performs operations on stream data  Can be prepended or appended (even on the fly)Filters  Can be attached to read or write
  106. 106.  pipes are like sockets but only go one wayProcesses (pipes)  stdin, stdout, stderr are all pipes  proc_open and popen do pipes  You can use stream functions on pipes, can’t do custom pipes  Socket › Bidirectional network stream that speaks a protocol  TransportSockets › Tells a network stream how to communicate  Wrapper › Tells a stream how to handle specific protocols and encodings
  107. 107.  http://edit.php.net Blog posts Articles Cool apps using streams, sockets and filters properly
  108. 108.  Set of interfaces for streams Improved based filter class Wrappers for chmod and touch More tests! Any you have?
  109. 109.  http://emsmith.net http://joind.in/3747 auroraeosrose@gmail.com IRC – freenode – auroraeosrose #php-gtk #coapp and others

×