Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Paws – A Perl AWS SDK
@pplu_io
03/09/2015 - Granada
YAPC::EU 2015
Jose Luis Martinez
AWS is…
• Cloud Computing
• Consume computing/database/queuing/etc services via an API
• Everything is an API 
AWS is…
Programmers wet dream
Why?
Isn’t there support for AWS on CPAN?
AWS Services on CPAN
• There are a LOT
• EC2, SQS, S3, RDS, DynamoDB, etc
• But lots are were missing
• AWS::CLIWrapper is...
Different authors, different opinions
• Default region (eu-west-1 for some, us-east-1 for others)
• Different HTTP clients...
Different authors, different photo
• Some regions not supported due to bugs
• Subtle name changes in region endpoints
• Cr...
Credential handling
• Roles in AWS help you not have to distribute credentials (AccessKey
and SecretKey)
• Support depends...
UpToDate-ness
• Being up to date depends on authors needs, time, etc
• AWS APIs are updated a lot
I want up to date APIs
Lets write an SDK!
Some numbers
52 services
Some numbers
52 services
~1600 actions
Some numbers
52 services
~1600 actions
~3700 distinct input/output objects
Some numbers
52 services
~1600 actions
~3700 distinct input/output objects
~12000 attributes
Write by hand?
Write by hand?
Paws is autogenerated
Paws is autogenerated
• AWS has some JSON definition files in their SDKs (data-driven)
• Pick them up to generate classes ...
Code generators
• In builder-lib (not distributed on CPAN)
• Paws::API::Builder
• Paws::API::Builder::EC2
• Paws::API::Bui...
Using Paws
Each AWS API is a “Service Class”
• Each Action in the API is a method on the Service Class
• EC2 API -> Paws::EC2 service...
How do I get an instance of a service
class?
use Paws;
my $ec2 = Paws->service(‘EC2’, region => ‘eu-west-1’);
my $iam = Pa...
How do I get an instance of a service
class? (II)
my $paws = Paws->new(config => {
region => ‘eu-west-1’,
caller => ‘Paws:...
Calling a method
$ec2->Method1(
Param1 => ‘Something’,
Param2 => 42,
Complex1 => {
x => 1,
y => 2,
z => 3
},
Complex2 => [...
Calling a method
$ec2->Method1(
Param1 => ‘Something’,
Param2 => 42,
Complex1 => {
x => 1,
y => 2,
z => 3
},
Complex2 => [...
Calling a method: maps
• Arbitrary key/value pairs
• Don’t build an object either. Paws will handle it for you
• $ec2->Met...
Methods return objects
my $object = $x->Method1(…)
Method1 returns Paws::Service::Method1Result
has ‘X’, has ‘Y’, has ‘Com...
Tricks
Tricks: CLI
• Paws ships with a CLI
paws SERVICE --region xx-east-1 DescribeFoo Arg1 Val1
Uses ARGV::Struct to convey nest...
Tricks: open_aws_console
• Opens a browser with the AWS console (using the SignIn service)
• Uses your current credentials...
Tricks: Changing endpoints
my $predictor = $paws->service('ML', region_rules =>
[ { uri => $endpoint_url } ]);
• Works for...
Tricks: Credential providers
• Default one tries to behave like AWS SDKs
• Environment (AWS_ACCESS_KEY_ID and AWS_SECRET_A...
Internals
Note: actual implementation as of Sept 2015
Read the code / changelog to get a hold of changes
Each method has parameters
• Parameters are converted into Moose objects for validation
package Paws::EC2
sub Method(Param...
Each method has parameters
• Parameters are converted into Moose objects for validation
package Paws::EC2
sub Method(Param...
After coercing parameters into an object
• $self->caller->do_call($self, $call_object)
• Service classes have a “caller”. ...
Call Object to APIRequest
(prepare_request_for_call)
• Looks in the call object where it has to place parameters to the AP...
handle_response
• Takes a look if there were error conditions in the HTTP call
• Future: should determine how to retry
• D...
Callers
• Do the IO
• Have to handle some common logic (still)
• Asyc callers don’t need to return the result immediately
...
Future
Future (hint: help needed and accepted)
• Testing Async stuff
• Retrying
• Some APIs return temporary failures
• Want auto...
Future (hint: help needed and accepted)
• Object Oriented results
• $ec2->TerminateInstances(InstanceIds => [ ‘i-12345678’...
Future (hint: help needed and accepted)
• Refactoring generator clases
• MooseX::DataModel
• Template::Toolkit
• Split Paw...
Support for APIs
0
5
10
15
20
25
30
35
40
45
50
REST Plain
Types of APIs
Query+XML JSON EC2
Support for APIs
0
5
10
15
20
25
30
35
40
45
50
REST Plain
Types of APIs
Query+XML JSON EC2
Implemented
Support for APIs
0
5
10
15
20
25
30
35
40
45
50
REST Plain
Types of APIs
Query+XML JSON EC2
Implemented
Need love and test...
Fork your heart out
https://github.com/pplu/aws-sdk-perl/
Contact me:
Twitter: @pplu_io
Mail: joseluis.martinez@capside.co...
Paws - A Perl AWS SDK
Paws - A Perl AWS SDK
Paws - A Perl AWS SDK
Paws - A Perl AWS SDK
Paws - A Perl AWS SDK
Paws - A Perl AWS SDK
Paws - A Perl AWS SDK
Paws - A Perl AWS SDK
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
Paws - Perl AWS SDK Update - November 2015
Next
Upcoming SlideShare
Paws - Perl AWS SDK Update - November 2015
Next
Download to read offline and view in fullscreen.

Share

Paws - A Perl AWS SDK

Download to read offline

Updated information presented at the European Perl Conference in Granada this September.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Paws - A Perl AWS SDK

  1. 1. Paws – A Perl AWS SDK @pplu_io 03/09/2015 - Granada YAPC::EU 2015 Jose Luis Martinez
  2. 2. AWS is… • Cloud Computing • Consume computing/database/queuing/etc services via an API • Everything is an API 
  3. 3. AWS is… Programmers wet dream
  4. 4. Why? Isn’t there support for AWS on CPAN?
  5. 5. AWS Services on CPAN • There are a LOT • EC2, SQS, S3, RDS, DynamoDB, etc • But lots are were missing • AWS::CLIWrapper is a generic solution too • Shells off to the oficial AWS CLI (python) I want Perl support for ALL of them
  6. 6. Different authors, different opinions • Default region (eu-west-1 for some, us-east-1 for others) • Different HTTP clients • LWP, HTTP::Tiny, Furl, etc I want explicit, required, region. Croak if not specified Pluggable HTTP client?
  7. 7. Different authors, different photo • Some regions not supported due to bugs • Subtle name changes in region endpoints • Credential handling • Module just covers their needs I want as broad support as we can get
  8. 8. Credential handling • Roles in AWS help you not have to distribute credentials (AccessKey and SecretKey) • Support depends on author of module knowing of them / needing them I want support for Instance Roles, STS AssumeRole, Federation for all services
  9. 9. UpToDate-ness • Being up to date depends on authors needs, time, etc • AWS APIs are updated a lot I want up to date APIs
  10. 10. Lets write an SDK!
  11. 11. Some numbers 52 services
  12. 12. Some numbers 52 services ~1600 actions
  13. 13. Some numbers 52 services ~1600 actions ~3700 distinct input/output objects
  14. 14. Some numbers 52 services ~1600 actions ~3700 distinct input/output objects ~12000 attributes
  15. 15. Write by hand?
  16. 16. Write by hand?
  17. 17. Paws is autogenerated
  18. 18. Paws is autogenerated • AWS has some JSON definition files in their SDKs (data-driven) • Pick them up to generate classes for: • Actions • Inputs to actions (parameters) • Outputs from actions (outputs) • HTML documentation -> POD make gen-classes
  19. 19. Code generators • In builder-lib (not distributed on CPAN) • Paws::API::Builder • Paws::API::Builder::EC2 • Paws::API::Builder::query • Paws::API::Builder::json • Paws::API::Builder::restjson • Paws::API::Builder::restxml • Leaves all auto-generated code in auto-lib (distributed on CPAN) • Hand-written code is in lib Note: this is not needed if you only want to use Paws. This is intended for developers. We’ll see more internals later 
  20. 20. Using Paws
  21. 21. Each AWS API is a “Service Class” • Each Action in the API is a method on the Service Class • EC2 API -> Paws::EC2 service class • Paws::EC2 objects have methods like • RunInstances • TerminateInstances • DescribeInstances
  22. 22. How do I get an instance of a service class? use Paws; my $ec2 = Paws->service(‘EC2’, region => ‘eu-west-1’); my $iam = Paws->service(‘IAM’); # $ec2 and $iam are instances of Paws::EC2 and Paws::IAM # they use Paws default config (they just work )
  23. 23. How do I get an instance of a service class? (II) my $paws = Paws->new(config => { region => ‘eu-west-1’, caller => ‘Paws::Net::LWPCaller’, credentials => ‘My::Custom::Credential::Provider’ }); my $ec2 = $paws->service(‘EC2’); # ec2 is bound to region ‘eu-west-1’ # and called with LWP # and gets it’s credentials from some custom source
  24. 24. Calling a method $ec2->Method1( Param1 => ‘Something’, Param2 => 42, Complex1 => { x => 1, y => 2, z => 3 }, Complex2 => [ { x => 1, y => 2 }, { x => 2, y => 3 } ])
  25. 25. Calling a method $ec2->Method1( Param1 => ‘Something’, Param2 => 42, Complex1 => { x => 1, y => 2, z => 3 }, Complex2 => [ { x => 1, y => 2 }, { x => 2, y => 3 } ]) Docs tell you that this is a Paws::Service::XXX object, but you don’t have to instance it !!! Just pass the attributes and the values as a hashref 
  26. 26. Calling a method: maps • Arbitrary key/value pairs • Don’t build an object either. Paws will handle it for you • $ec2->Method1( Map1 => { x => 1, y => 2, z => 3 });
  27. 27. Methods return objects my $object = $x->Method1(…) Method1 returns Paws::Service::Method1Result has ‘X’, has ‘Y’, has ‘Complex’ => (isa => ‘Paws::Service::Complex1’) $object->X $object->Complex->Complex1Attribute
  28. 28. Tricks
  29. 29. Tricks: CLI • Paws ships with a CLI paws SERVICE --region xx-east-1 DescribeFoo Arg1 Val1 Uses ARGV::Struct to convey nested datastructures via command line
  30. 30. Tricks: open_aws_console • Opens a browser with the AWS console (using the SignIn service) • Uses your current credentials (extends a temporary token)
  31. 31. Tricks: Changing endpoints my $predictor = $paws->service('ML', region_rules => [ { uri => $endpoint_url } ]); • Works for any service: SQS, EC2…
  32. 32. Tricks: Credential providers • Default one tries to behave like AWS SDKs • Environment (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) • File (~/.aws/credentials, an ini file) • From the metadata service (Instance Roles) • Your own • Just attach Role “Paws::Credential” and get the credentials from wherever
  33. 33. Internals Note: actual implementation as of Sept 2015 Read the code / changelog to get a hold of changes
  34. 34. Each method has parameters • Parameters are converted into Moose objects for validation package Paws::EC2 sub Method(Param1 => Str, Param2 => Int) Coerces its @_ into Paws::EC2::Method (has ‘Param1’, has ‘Param2’) Note: not using Moose coercion. Using new_with_coercions
  35. 35. Each method has parameters • Parameters are converted into Moose objects for validation package Paws::EC2 sub Method(Param3 => Complex1) Complex1 has it’s own “input class” Paws::EC2::Complex1 has [‘X’, ‘Y’, ‘Z’ ] new_with_coercions knows how to coerce { x => 1, y => 2, z => 3 } into a Paws::EC2::Complex1
  36. 36. After coercing parameters into an object • $self->caller->do_call($self, $call_object) • Service classes have a “caller”. Caller is defined when constructing the service object. • Callers are responsable for • Getting a Paws::Net::APIRequest (via prepare_request_for_call) • Prepare_request_for_call is specialized for each type of service in Paws::Net::*Caller roles • Doing I/O • Paws::Net::Caller uses HTTP::Tiny (Paws default) • Paws::Net::LWPCaller uses LWP (contributed) • Paws::Net::MojoAsyncCaller uses Mojo::UserAgent (experimental) • Passing results to handle_response
  37. 37. Call Object to APIRequest (prepare_request_for_call) • Looks in the call object where it has to place parameters to the API • Headers • In a serialized body • JSON • Query Parameters • Arrays get coded in f(x) of the API • att.0=xxx • att.members.0=xxx • In the body • In the URL (REST APIs) • Signs the request (via roles that know how to sign for that service)
  38. 38. handle_response • Takes a look if there were error conditions in the HTTP call • Future: should determine how to retry • Deserializes the response • XML • JSON • Deserializes into objects • Note: sometimes decides it wants an exception • Doesn’t throw: just creates an exception object
  39. 39. Callers • Do the IO • Have to handle some common logic (still) • Asyc callers don’t need to return the result immediately • The experimental Mojo caller returns a Future  • The future fails if the result was an exception
  40. 40. Future
  41. 41. Future (hint: help needed and accepted) • Testing Async stuff • Retrying • Some APIs return temporary failures • Want automatic exponential backoff with jitter • Paging • Some APIs return paged results • Want a “give me all of them” • Waiters • Wait until some condition is met • Want a call to wait until Instance is in running state • A lot more: take a look at GitHub issues
  42. 42. Future (hint: help needed and accepted) • Object Oriented results • $ec2->TerminateInstances(InstanceIds => [ ‘i-12345678’ ]) • $instance->Terminate • Special properties • En/Decode base64, URIescape, etc • Better access to ArrayRefs • Use ArrayRef Moose trait for • Number of elements • Get element i • Get list of elements
  43. 43. Future (hint: help needed and accepted) • Refactoring generator clases • MooseX::DataModel • Template::Toolkit • Split Paws into separately instalable modules • Rinse and Repeat • For other APIs • AWS API as a Service
  44. 44. Support for APIs 0 5 10 15 20 25 30 35 40 45 50 REST Plain Types of APIs Query+XML JSON EC2
  45. 45. Support for APIs 0 5 10 15 20 25 30 35 40 45 50 REST Plain Types of APIs Query+XML JSON EC2 Implemented
  46. 46. Support for APIs 0 5 10 15 20 25 30 35 40 45 50 REST Plain Types of APIs Query+XML JSON EC2 Implemented Need love and testing S3 not working Route53? Lambda? …
  47. 47. Fork your heart out https://github.com/pplu/aws-sdk-perl/ Contact me: Twitter: @pplu_io Mail: joseluis.martinez@capside.com CPAN: JLMARTIN CAPSiDE Twitter: @capside Mail: hello@capside.com
  • studioego

    Dec. 15, 2016
  • rorybramwell

    Aug. 23, 2016
  • rorybramwell

    Aug. 23, 2016
  • zerojinx

    Nov. 2, 2015

Updated information presented at the European Perl Conference in Granada this September.

Views

Total views

2,294

On Slideshare

0

From embeds

0

Number of embeds

21

Actions

Downloads

10

Shares

0

Comments

0

Likes

4

×