Building an aws sdk for Perl - Granada Perl Workshop 2014

1,079 views

Published on

Update of the advance of building an AWS SDK for Perl

Published in: Technology
  • Be the first to comment

Building an aws sdk for Perl - Granada Perl Workshop 2014

  1. 1. Building an AWS SDK Granada Perl Workshop 2014 Jose Luis Martinez JLMARTIN on CPAN Twitter: @pplu_io joseluis.martinez@capside.com
  2. 2. About me  CTO @CAPSiDE  Training Partner for AWS  Delivering AWS official training  All four AWS certifications  Real experience with real platforms  Deliver consulting and continuous operation  Writing Perl for +10 years  And still having fun with it!
  3. 3. About AWS  Mostly known for EC2 and S3  But lot's more services! (+30)  Everything is an API  “Programmable datacenter”  Software Defined Everything
  4. 4. About AWS  Mostly known for EC2 and S3  But lot's more services! (+30)  Everything is an API  “Programmable datacenter”  Software Defined Everything It’s the your wet dream
  5. 5. Perl & AWS Bad news   No official SDK  Official ones: Ruby, PHP, Python, JS, .Net, Java  Note: python wasn't official initially!
  6. 6. Perl & AWS Good news   CPAN  And an active community  Lots of modules for different services (sometimes more than one)
  7. 7. Perl & AWS: The CPAN  Inconsistencies  In method naming  Arbitrary defaults ('eu-west-1', 'us-east-1')  Not up to date with latest APIs  Keeping up with AWS is hard!!!  “Not so popular” services not covered  IAM, CloudFormation, SWF, RedShift, CloudSearch, etc  Almost no Role support  AWS::CLIWrapper is a good generic solution. Not native
  8. 8. So lets write an entire SDK!
  9. 9. Hard enough
  10. 10. You can end up with… or
  11. 11. Challenges: # of services  30+ services  Some with 100+ API calls  Each call with it’s parameters  Want to return objects  Evolution
  12. 12. Challenges: # of services
  13. 13. So lets start classifying
  14. 14. Challenges: Endpoints Endpoint Type Services Regional Services available in a región the rest Single Global services 6
  15. 15. Challenges: WebService Types One URL REST Params: Query Response: XML 17 3 Params: JSON Response: JSON 10 1
  16. 16. Challenges: Signatures Type Services v2 2 v3https 1 cloudfront 1 v4 the rest
  17. 17. Challenges: API versions  More tan one API version is live  That’s why not-up-to-date callers still work  RDS  4 versions  CloudFront  3 versions  EC2  4 versions
  18. 18. AWS Pace of innovation  Every month there are changes  Changes mean APIs change (slightly)  Very hard to keep up!
  19. 19. AWS Pace of innovation  Every week there are announcements  Changes mean APIs change (slightly)  Very hard to keep up!
  20. 20. Lets start writing!
  21. 21. Hand write classes and methods, and parameters
  22. 22. Hand write classes and methods, and parameters
  23. 23. Parse docs  After all… you would have to read them to write the classes
  24. 24. Parse docs  After all… you would have to read them to write the classes
  25. 25. No published spec  Introspect an official SDK!  After all AWS already has read the docs for you
  26. 26. No published spec  Introspect an official SDK!  That’s your spec!  After all AWS already has read the docs for you
  27. 27. Data driven SDKs!  JS SDK has datastructures that define calls  Proof of concept  Win++! Boto and PHP2 too!  Later migrate to using Boto
  28. 28. Data driven SDKs!  JS SDK has datastructures that define calls  Proof of concept  Win++! Boto and PHP2 too!  Later migrate to using Boto
  29. 29. The solutions
  30. 30. Modern Perl to the rescue  Try to be very lazy  Parameter validation: Create Moose classes that validate the parameters passed to the call. May change in the future.  Got very quick results!  Hide all possible implementation from the user  Try not to leak implementation  That way you can change it at will  Meta lets you inspect the objects!  Call objects get marshalled into datastructured via meta  Result datastructures get marshalled into objects via meta
  31. 31. Roles
  32. 32. Modern Perl: Roles  All functionality gets pulled in via Roles  Code is basically in different roles, Classes are “serialized configurations” of how to call APIs  Makes it easy to  Generate service classes from datastructures  Change behaviour for tests
  33. 33. Roles are used for  One global endpoint vs endpoints per region  Caller (Query, JSON)  Response (XML, JSON)  Signatures (V2, V3, V4)  IO
  34. 34. Aws is a class factory
  35. 35. Dynamic class construction  User never constructs service classes directly  Aws class method uses default config my $ec2 = Aws->service(‘EC2’)->new( . . . )
  36. 36. Dynamic class construction  User wants custom functionality my $aws = Aws->new( config => AWS::SDK::Config->new( caller => 'Net::AWS::TestCaller‘ ) ); my $ec2 = $aws->service(‘EC2’)->new( . . . ) service loads Aws::EC2, creates a new class with TestCaller
  37. 37. Win: Hello to the async world  Still playing around in examples my $aws = Aws->new( config => AWS::SDK::Config->new( caller => 'Net::AWS::MojoCaller‘ ) ); my $ec2 = $aws->service(‘EC2’)->new( . . . ) my $asgs = $ec2->DescribeAutoScalingGroups; See examples/asyncAutoScaling.pl
  38. 38. Win: Also going “fully async” my $aws = Aws->new( config => AWS::SDK::Config->new(caller => 'Net::AWS::MojoAsyncCaller') ); my $delay = Mojo::IOLoop->delay(sub { my ($delay, @responses) = @_; Dumper($_) for @responses; }); foreach my $region ( "us-east-1", "ap-northeast-1", "sa-east-1", "ap-southeast-1", "ap-southeast-2", "us-west-2", "us-west-1", "eu-west-1", 'invented-region') { my $ctrail = $aws->service('CloudTrail')->new( region => $region, delay => $delay, ); print "Doing a call for $regionn"; my $list = $ctrail->DescribeTrails; } $delay->wait;
  39. 39. Lessons Learned
  40. 40. Lessons learned  First shot with MooseX::Declare  Error messages are HORRIBLE!  Change to Moops didn't go well  Finally plain old Moose
  41. 41. Lessons learned  Namespace finding (still in the works)  AWS vs Aws  Thanks @clintongormley for the article!
  42. 42. Lessons learned  Enums (still in the works)  APIs sometimes return unexpected stuff
  43. 43. Lessons learned  Abstract $request object passed around  Finally marshalled into the UserAgents’ taste
  44. 44. Still open stuff  Load time for big services (EC2)  Not all classes will be used all the time  Generate classes on demand  Calling convention  Now CamelCased  Not so perlish  Calling from objects (not service objects)  $queue->DeleteQueue(); from a queue object  Attribute Traits  Access to Arrays and HashRefs via nicer methods  Lots more!!!
  45. 45. Use AWS? Use Perl? Want to help?  Fork your heart out: https://github.com/pplu/aws-sdk-perl/

×