Your SlideShare is downloading. ×
Process any amount of data. Any time - Symfony Live
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Process any amount of data. Any time - Symfony Live

1,113
views

Published on

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,113
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Process any amount of data.Any timeJuozas Kaziukėnas // juokaz.com // @juokaz
  • 2. Juozas Kaziukėnas, LithuanianYou can call me JoeMore info http://juokaz.com
  • 3. The problem
  • 4. CRON
  • 5. PHP developers arelazy
  • 6. Code should be lazy
  • 7. file_get_contens
  • 8. Buffering
  • 9. Wait
  • 10. Memory usage
  • 11. Saw graph
  • 12. Memory vsperformance
  • 13. Reading files
  • 14. Line-by-line reading$file_handle = fopen("myfile", "r");while (!feof($file_handle)) {$line = fgets($file_handle);echo $line;}fclose($file_handle);
  • 15. Line-by-line processing$file_handle = fopen("myfile", "r");while (!feof($file_handle)) {$line = fgets($file_handle);process($line);}fclose($file_handle);
  • 16. XML
  • 17. Read XML files
  • 18. $z = new XMLReader;$z->open(data.xml);$doc = new DOMDocument;// move to the first <product /> nodewhile ($z->read() && $z->name !== product);// now that were at the right depth, hop to the next <product/> until the end of the treewhile ($z->name === product){// either one should work//$node = new SimpleXMLElement($z->readOuterXML());$node = simplexml_import_dom($doc->importNode($z->expand(), true));// now you can use $node without going insane about parsingvar_dump($node->element_1);// go to next <product />$z->next(product);}
  • 19. $z = new XMLReader;$z->open(data.xml);$doc = new DOMDocument;// move to the first <product /> nodewhile ($z->read() && $z->name !== product);// now that were at the right depth, hop to the next <product/> until the end of the treewhile ($z->name === product){// either one should work//$node = new SimpleXMLElement($z->readOuterXML());$node = simplexml_import_dom($doc->importNode($z->expand(), true));// now you can use $node without going insane about parsingvar_dump($node->element_1);// go to next <product />$z->next(product);}
  • 20. Running processes
  • 21. Pipes
  • 22. Linuxwget http://example.com/lol.xml.gzgunzip lol.xml.gz
  • 23. Linuxwget -O- http://example.com/lol.xml.gz | gunzip
  • 24. $command = "wget -O- $url | gunzip";$process = proc_open($command,array(array("pipe","r"),array("pipe","w"),array("pipe","w")),$pipes);while(!feof($pipes[1])) {$buffer .= fgets($pipes[1], 128);if (strpos($buffer, <Item>)) {// detected start of the item}if (strpos($buffer, </Item>)) {// detected end of the item}}fclose($pipes[1]);proc_close($process);
  • 25. Reading from MySQL
  • 26. mysql_unbuffered_query$lh = mysql_connect(server, uname, pword);$qry = "SELECT * FROM my_bigass_table";$rh = mysql_unbuffered_query($qry, $lh);while ($res = mysql_fetch_row($rh)){process($res);}
  • 27. Outputting data
  • 28. PHP$result = ;foreach ($data as $item) {$line = Name . $item[name] . PHP_EOL;$result .= $line;}echo $result;
  • 29. PHP$result = ;ob_end_clean();foreach ($data as $item) {echo Name . $item[name] . PHP_EOL;flush();}
  • 30. PHP 5.5
  • 31. Generators
  • 32. Yield
  • 33. Line-by-line processing$file_handle = fopen("myfile", "r");while (!feof($file_handle)) {$line = fgets($file_handle);process($line);}fclose($file_handle);
  • 34. Line-by-line processingfunction read() {$file_handle = fopen("myfile", "r");while (!feof($file_handle)) {$line = fgets($file_handle);yield $line;}fclose($file_handle);}$data = read();foreach ($data as $line) {}
  • 35. Making HTTP requests
  • 36. Open connections limit
  • 37. Connection Sharing with CURL inPHP: How to re-use HTTPconnections to knock 70% off RESTnetwork time.http://technosophos.com/content/connection-sharing-curl-php-how-re-use-http-connections-knock-70-rest-network-time
  • 38. Next steps
  • 39. Gearman
  • 40. Hadoop
  • 41. Why PHP?
  • 42. What’s possible
  • 43. Using it for good
  • 44. THANKS!Juozas Kaziukėnas@juokaz