Developing apps  using Perl    Anatoly Sharifulin   YAPC::Russia 2012
Hello, World!
«Happy Perl developer»     about me by Andy Shitov
I program in Perl less   But it doesnt mean I love it less :-)
JFDI became JFTI   Just F*cking Talk about It
Success-story  of one appAnd how Perl helped me/us with it
By us I mean Applifto
www.applifto.com
Our applications
Audience polling
DLTTR.com
«Why?» and «For what?»      in details http://www.slideshare.net/sharifulin/ss-12313103
A powerful reason            for me      I was moving from LiveJournal to Posterous,used transfer scripts and changed Twit...
Current analogues are nice... not! Slow, awkward, plain, trouble working:TwitWipe, TwitCide, Delete My Tweets, ...
«Stop managering,let’s start programming»     And have a talk at #yr2012 :-)
Approach and solutions
Main aims for me• App must delete tweets quickly• Appropriate deleting• Cross-platform app: iOS, Android, Web, ...
Server API         — hello, Perl!All logic — on our server API (not Twitter API),            clients is simple interface
Getting to know  Twitter API
Little experience:• Getting user profiles after login through  Twitter on websites• Auto-posting to Twitter with  Net::Twit...
Blocking wayTask can be solved with Net::Twitter(::Lite)
LWP::UserAgent::POETransport may be changed in Net::Twitter(::Lite)
NO
Idea — 2 separate   asynchronous queuesGet timeline and delete tweets using REST Twitter API             and asynchronous ...
This has to work fast!
Authorization and OAuth
Net::OAuth::AllWeve written our OAuth module long time ago, supports all OAuth versions (1.0, 1.0A, 2.0)
Net::OAuth::Allhttps://github.com/likhatskiy/Net-OAuth-All
my $oauth = Net::OAuth::All->new(	 consumer_secret => $conf->{consumer_secret},	 consumer_key => $conf->{consumer_key },	 ...
$oauth	 ->via(GET)	 ->protected_resource_url(https://api.twiiter.com/...’)	 ->put_extra(	 	 include_rts => true,	 	 user_i...
Patched the module            Unauthorized user:limit of 150 requests per hour from one IP, not 350 requests per hour from...
Asynchronous requests
I use Mojolicious   Not a secret at all :-)
Mojo::UserAgent Mojo::IOLoop Good asynchronous HTTP client
my $delay = Mojo::IOLoop->delay;for (@$tasks) {   ...   $delay->begin;   $ua->get($url => {Authorization => $h} => sub {  ...
Server API
Server APIAPI supports GET and POST requests,      JSON transmission format,     response codes 200 and 50x.
Server API   Starman + MojoliciousMojolicious::Plugin::ApiHelpers
Server API http://api.dlttr.com
package App::Api;use App::Base -controller,  with => [ App::Task, App::User ];sub error { shift->api_error(code => 1) }sub...
my $api = $r->route(/api)->to(api#, api => 1);$api->route(/:action, action => qr/login|oauth/)->to;my $apiu = $api->bridge...
10 methods,123 tests and documentation Most methods require user authorization     and check sign (api_id + secret)
Test::Mojo         Test::More       Testing API methods access,input-output data, signature check and etc.
Test::Mojo Test::MoreA real task from a test user
All methods       are covered        Without Mock-objects, no checkif a correct cluster of tweets have been deleted
Too laboriousand needless A real cycle cannot be tested:posting tweets – deleting – check
$t->get_ok("$url/")  ->status_is(200)  ->header_is(Access-Control-Allow-Origin, *)  ->json_content_is({hello => Hello DLTT...
Too risky to test onyour own account :-)— You dont care about you Twitter acc, do you?— Yep!— So I can delete all your twe...
Simple         debug reguestsTo check the quality of client work requests/responses  with before_dispatch + after_dispatch...
Three queues
1. Queue for timelines  Search and filters on tweets ID to delete,      paging with since_id and max_id
1. Queue for timelines       Max. 200 tweets per request,   getting timeline one by one per user,              Rate Limit ...
«Limits» by Twitter               No access to all tweets,only the latest ~3200 or starting from any exact date,          ...
2. Queue for deleting      tweetsDeleting with clusters of 200 tweets, no limits.           «Buster» or «Cannon» :-)
2. Queue for deleting      tweets  Error processing — Twitter API error      doesnt always mean a failure
3. Queue for           Push notify                       On iOS   Net::APNS::Persistent (also AnyEvent::APNS),to check if ...
3. Queue for          Push notify                On AndroidWWW::Google::C2DM and WWW::Google::ClientLogin
Database
So easy — mysql  Storing and task orders,      just to relax :-)
2 finer points:
1. Create correct index         For queues
2. Disable request        cachingselect SQL_NO_CACHE * from task where ...
Site and web-version
www.dlttr.com
www.dlttr.com
Site and web-version• Starman + Mojolicious• Mojolicious::Plugin::I18N2  — my fork of I18N plugin• Mojolicious::Plugin::OA...
Admin and statistics
«Big Brother»All details on users, number of tasks, feedbacks             and real-time statistics
Statistics on users
Users wall
«Ordinary» apps donthave statistics like that
1M+ tweets deleted,   3K+ users,20% make purchase      For 2 months
Promotion
Promotion on Twitter Following, favorites, replies and search —             Net::Twitter(::Lite)
Promotion on Twitter But they blocked me three times already :-)
Summary
Positive• Easy solution using pure Perl  (work time is about 30 hours)• Works fast, really fast• Right delete tweets• Any ...
Only one negative:    Servers down — the app doesnt work,users become mad, low rates and spell curses :-)
Check out DLTTR!http://dlttr.com/app http://dlttr.com/android                www.dlttr.com
«Seems like a simple app but in fact its awesome       and in Perl»
P. S.
At #BarCampKrr       I had a talk     «Developing app  from idea to release»(Perl wasnt mentioned)
But everything came  down to... Perl           «What language do you use?»,                     «But why?»,«Is it difficult...
use Perl or die;
Thanks! Anatoly SharifulinYAPC::Russia 2012
Developing apps using Perl
Developing apps using Perl
Upcoming SlideShare
Loading in...5
×

Developing apps using Perl

4,736

Published on

Talk from PerlMova 2012 and YAPC::Russia "May Perl"

Published in: Education, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,736
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
14
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Developing apps using Perl

  1. 1. Developing apps using Perl Anatoly Sharifulin YAPC::Russia 2012
  2. 2. Hello, World!
  3. 3. «Happy Perl developer» about me by Andy Shitov
  4. 4. I program in Perl less But it doesnt mean I love it less :-)
  5. 5. JFDI became JFTI Just F*cking Talk about It
  6. 6. Success-story of one appAnd how Perl helped me/us with it
  7. 7. By us I mean Applifto
  8. 8. www.applifto.com
  9. 9. Our applications
  10. 10. Audience polling
  11. 11. DLTTR.com
  12. 12. «Why?» and «For what?» in details http://www.slideshare.net/sharifulin/ss-12313103
  13. 13. A powerful reason for me I was moving from LiveJournal to Posterous,used transfer scripts and changed Twitter and Facebook crossposting settings when started to get replies like Tolya! Stop it! Enough!
  14. 14. Current analogues are nice... not! Slow, awkward, plain, trouble working:TwitWipe, TwitCide, Delete My Tweets, ...
  15. 15. «Stop managering,let’s start programming» And have a talk at #yr2012 :-)
  16. 16. Approach and solutions
  17. 17. Main aims for me• App must delete tweets quickly• Appropriate deleting• Cross-platform app: iOS, Android, Web, ...
  18. 18. Server API — hello, Perl!All logic — on our server API (not Twitter API), clients is simple interface
  19. 19. Getting to know Twitter API
  20. 20. Little experience:• Getting user profiles after login through Twitter on websites• Auto-posting to Twitter with Net::Twitter(::Lite)• AnyEvent::Twitter::Stream for real-time search (hey, @miyagawa!)
  21. 21. Blocking wayTask can be solved with Net::Twitter(::Lite)
  22. 22. LWP::UserAgent::POETransport may be changed in Net::Twitter(::Lite)
  23. 23. NO
  24. 24. Idea — 2 separate asynchronous queuesGet timeline and delete tweets using REST Twitter API and asynchronous requests
  25. 25. This has to work fast!
  26. 26. Authorization and OAuth
  27. 27. Net::OAuth::AllWeve written our OAuth module long time ago, supports all OAuth versions (1.0, 1.0A, 2.0)
  28. 28. Net::OAuth::Allhttps://github.com/likhatskiy/Net-OAuth-All
  29. 29. my $oauth = Net::OAuth::All->new( consumer_secret => $conf->{consumer_secret}, consumer_key => $conf->{consumer_key }, token => $data->{access_token }, token_secret => $data->{access_token_secret},);
  30. 30. $oauth ->via(GET) ->protected_resource_url(https://api.twiiter.com/...’) ->put_extra( include_rts => true, user_id => $user->{id}, count => 200, ) ->request(protected_resource);$oauth->to_header;$oauth->url; # $oauth->url_with_extra;
  31. 31. Patched the module Unauthorized user:limit of 150 requests per hour from one IP, not 350 requests per hour from one user
  32. 32. Asynchronous requests
  33. 33. I use Mojolicious Not a secret at all :-)
  34. 34. Mojo::UserAgent Mojo::IOLoop Good asynchronous HTTP client
  35. 35. my $delay = Mojo::IOLoop->delay;for (@$tasks) { ... $delay->begin; $ua->get($url => {Authorization => $h} => sub { my ($ua, $tx) = @_; ... $delay->end( ... ); });}say $delay->wait;
  36. 36. Server API
  37. 37. Server APIAPI supports GET and POST requests, JSON transmission format, response codes 200 and 50x.
  38. 38. Server API Starman + MojoliciousMojolicious::Plugin::ApiHelpers
  39. 39. Server API http://api.dlttr.com
  40. 40. package App::Api;use App::Base -controller, with => [ App::Task, App::User ];sub error { shift->api_error(code => 1) }sub any { shift->api_error(code => 5) }...1;
  41. 41. my $api = $r->route(/api)->to(api#, api => 1);$api->route(/:action, action => qr/login|oauth/)->to;my $apiu = $api->bridge->to(#auth, sign => 1);$apiu->bridge(/task/new)->to(#task_check) ->route->to(#task_new);$api->bridge(/)->to(#auth, maybe => 1, sign => 0) ->route->to(#hello);
  42. 42. 10 methods,123 tests and documentation Most methods require user authorization and check sign (api_id + secret)
  43. 43. Test::Mojo Test::More Testing API methods access,input-output data, signature check and etc.
  44. 44. Test::Mojo Test::MoreA real task from a test user
  45. 45. All methods are covered Without Mock-objects, no checkif a correct cluster of tweets have been deleted
  46. 46. Too laboriousand needless A real cycle cannot be tested:posting tweets – deleting – check
  47. 47. $t->get_ok("$url/") ->status_is(200) ->header_is(Access-Control-Allow-Origin, *) ->json_content_is({hello => Hello DLTTR!});$t->get_ok(sign get => "$url/task/new") ->status_is(200) ->json_content_is({error => {msg => User authorization failed, code => 2}});
  48. 48. Too risky to test onyour own account :-)— You dont care about you Twitter acc, do you?— Yep!— So I can delete all your tweets?— Nooooooooooooooo!!!
  49. 49. Simple debug reguestsTo check the quality of client work requests/responses with before_dispatch + after_dispatch may be used
  50. 50. Three queues
  51. 51. 1. Queue for timelines Search and filters on tweets ID to delete, paging with since_id and max_id
  52. 52. 1. Queue for timelines Max. 200 tweets per request, getting timeline one by one per user, Rate Limit check
  53. 53. «Limits» by Twitter No access to all tweets,only the latest ~3200 or starting from any exact date, counter is not on zero and etc.
  54. 54. 2. Queue for deleting tweetsDeleting with clusters of 200 tweets, no limits. «Buster» or «Cannon» :-)
  55. 55. 2. Queue for deleting tweets Error processing — Twitter API error doesnt always mean a failure
  56. 56. 3. Queue for Push notify On iOS Net::APNS::Persistent (also AnyEvent::APNS),to check if tokens are valid — Net::APNS::Feedback
  57. 57. 3. Queue for Push notify On AndroidWWW::Google::C2DM and WWW::Google::ClientLogin
  58. 58. Database
  59. 59. So easy — mysql Storing and task orders, just to relax :-)
  60. 60. 2 finer points:
  61. 61. 1. Create correct index For queues
  62. 62. 2. Disable request cachingselect SQL_NO_CACHE * from task where ...
  63. 63. Site and web-version
  64. 64. www.dlttr.com
  65. 65. www.dlttr.com
  66. 66. Site and web-version• Starman + Mojolicious• Mojolicious::Plugin::I18N2 — my fork of I18N plugin• Mojolicious::Plugin::OAuth — login through Net::OAuth::All• Mojolicious::Plugin::ShareHelpers• Mojolicious::Plugin::UtilHelpers
  67. 67. Admin and statistics
  68. 68. «Big Brother»All details on users, number of tasks, feedbacks and real-time statistics
  69. 69. Statistics on users
  70. 70. Users wall
  71. 71. «Ordinary» apps donthave statistics like that
  72. 72. 1M+ tweets deleted, 3K+ users,20% make purchase For 2 months
  73. 73. Promotion
  74. 74. Promotion on Twitter Following, favorites, replies and search — Net::Twitter(::Lite)
  75. 75. Promotion on Twitter But they blocked me three times already :-)
  76. 76. Summary
  77. 77. Positive• Easy solution using pure Perl (work time is about 30 hours)• Works fast, really fast• Right delete tweets• Any changes, updates, new services — everythings on the server, no hard• Real-time statistics
  78. 78. Only one negative: Servers down — the app doesnt work,users become mad, low rates and spell curses :-)
  79. 79. Check out DLTTR!http://dlttr.com/app http://dlttr.com/android www.dlttr.com
  80. 80. «Seems like a simple app but in fact its awesome and in Perl»
  81. 81. P. S.
  82. 82. At #BarCampKrr I had a talk «Developing app from idea to release»(Perl wasnt mentioned)
  83. 83. But everything came down to... Perl «What language do you use?», «But why?»,«Is it difficult to find programmers like those?» :-)
  84. 84. use Perl or die;
  85. 85. Thanks! Anatoly SharifulinYAPC::Russia 2012
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×