7. Cutting edge interactions
Latest technologies
Modern languages
LPW
The Hype
Scala
Erlang
8th November 2014 7
8. It's just HTTP
Some event triggers action
Thing makes an HTTP request
Server sends response
Thing does something
LPW
The Reality
Usually
8th November 2014 8
9. Any language works
Perl just as effective as other languages
Perl has a long history of writing HTTP
servers
And HTTP clients
Many useful modules
LPW
The Reality
Of course
8th November 2014 9
10. Your “thing” is an HTTP client
But its interface will be unusual
Limited input/output channels
Hardware sensors
Device::SerialPort
Device::BCM2835
Arduino workshop
LPW
Hardware
8th November 2014 10
11. Your “thing” won't be displaying web pages
So returning HTML is probably unhelpful
Data-rich representation
LPW
HTTP Response
JSON
8th November 2014 11
13. We're used to writing web server
applications in Perl
But we can write web clients too
Programs that act like a browser
Make an HTTP request
Parse the HTTP reponse
Take some action
LPW
Web Clients
And we'll cover more about that later
8th November 2014 13
14. Most people would reach for LWP
LWP is “libwww-perl”
Library for writing web clients in Perl
Powerful and flexible HTTP client
Install from CPAN
LPW
LWP
8th November 2014 14
15. Small web client library for Perl
Part of standard Perl installation
Tiny is good for IoT
LPW
HTTP::Tiny
Since Perl 5.14
8th November 2014 15
16. use HTTP::Tiny;
LPW
Using HTTP::Tiny
my $response =
HTTP::Tiny->new->get('http://example.com/');
die "Failed!n" unless $response->{success};
print "$response->{status} $response->{reason}n";
while (my ($k, $v) = each %{$response->{headers}}) {
for (ref $v eq 'ARRAY' ? @$v : $v) {
print "$k: $_n";
}
}
print $response->{content}
if length $response->{content};
8th November 2014 16
17. Create a new user agent object (“browser”)
with new()
Various configuration options
my $ua = HTTP::Tiny->new(%options);
LPW
HTTP::Tiny->new
8th November 2014 17
18. agent – user agent string
cookie_jar – HTTP::CookieJar object
default_headers – hashref
LPW
Options
Or equivalent
8th November 2014 18
21. verify_SSL – default is false
SSL_options – passed to IO::Socket::SSL
LPW
SSL Options
8th November 2014 21
22. Use the request() method
$resp = $ua->request(
Useful options
LPW
Making Requests
$method, $url, %options
);
headers
content
8th November 2014 22
23. Response is a hash
success – true if status is 2xx
url – URL that returned response
status
reason
content
headers
LPW
Responses
Not an object
8th November 2014 23
24. Higher level functions for HTTP requests
get
head
post
put
delete
LPW
Easier Requests
8th November 2014 24
25. Each is a shorthand way to call request()
$resp = $ua->get($url, %options)
$resp = $ua->request(
Same options
Same response hash
LPW
Easier Requests
'get', $url, %options
)
8th November 2014 25
26. $ua->request('post', $url,
$ua->post($url, %options)
Where do post parameters go?
In %options
LPW
POSTing Data
%options)
content key
Build it yourself
www_form_urlencode()
8th November 2014 26
27. $ua->post_form($url, $form_data)
$form_data can be hash ref
Or array ref
Sort order
LPW
POSTing Data
{ key1 => 'value1', key2 => 'value2' }
[ key1 => 'value1', key2 => 'value2' ]
8th November 2014 27
28. HTTP::Tiny::UA
HTTP::Thin
HTTP::Tiny::Mech
LPW
See Also
Higher level UA features
Wrapper adds HTTP::Request/HTTP::Response
compatibility
WWW::Mechanize wrapper for HTTP::Tiny
8th November 2014 28
30. LPW
8th November 2014
Dancer
Dancer is a simple route-based web
framework for Perl
Easy to get web application up and running
See Andrew Solomon's class at 12:00
We're actually going to be using Dancer2
31. LPW
8th November 2014
Simple API
Return information about MP3s
GET /mp3
List MP3s
GET /mp3/1
Info about a single MP3
32. LPW
8th November 2014
Database
CREATE TABLE mp3 (
id integer primary key,
title varchar(200),
artist varchar(200),
filename varchar(200)
);
34. LPW
8th November 2014
Data
Insert some sample data
sqlite> select * from mp3;
1|Royals|Lorde|music/lorde/pure-heroine/
royals.mp3
2|The Mother We Share|Chvrches|
music/chvrches/the-bones-of-what-we-believe/the-mother-
we-share.mp3
3|Falling|Haim|music/haim/days-are-gone/
falling.mp3
38. LPW
8th November 2014
Implement Routes
Our application doesn't do anything
Need to implement routes
Routes are defined in MP3/lib/MP3.pm
get '/' => sub {
template 'index';
};
39. LPW
8th November 2014
Implement Routes
Our application doesn't do anything
Need to implement routes
Routes are defined in MP3/lib/MP3.pm
get '/' => sub {
template 'index';
};
40. LPW
8th November 2014
Implement Routes
Our application doesn't do anything
Need to implement routes
Routes are defined in MP3/lib/MP3.pm
get '/' => sub {
template 'index';
};
41. LPW
8th November 2014
Implement Routes
Our application doesn't do anything
Need to implement routes
Routes are defined in MP3/lib/MP3.pm
get '/' => sub {
template 'index';
};
42. LPW
8th November 2014
/mp3 Route
Display a list of MP3s
Get them from the database
Use Dancer2::Plugin::DBIC
In config.yml
Plugins:
DBIC:
default:
dsn: dbi:SQLite:dbname=mp3.db
43. LPW
8th November 2014
/mp3 Route
In MP3/lib/MP3.pm
get '/mp3' => sub {
my @mp3s = schema->resultset('Mp3')->all;
content_type 'text/plain';
return join "n",
map { $_->title . ' / ' . $_->artist }
@mp3s;
};
44. LPW
8th November 2014
/mp3 Route
In MP3/lib/MP3.pm
get '/mp3' => sub {
my @mp3s = schema->resultset('Mp3')->all;
content_type 'text/plain';
return join "n",
map { $_->title . ' / ' . $_->artist }
@mp3s;
};
45. LPW
8th November 2014
/mp3 Route
In MP3/lib/MP3.pm
get '/mp3' => sub {
my @mp3s = schema->resultset('Mp3')->all;
content_type 'text/plain';
return join "n",
map { $_->title . ' / ' . $_->artist }
@mp3s;
};
46. LPW
8th November 2014
/mp3 Route
In MP3/lib/MP3.pm
get '/mp3' => sub {
my @mp3s = schema->resultset('Mp3')->all;
content_type 'text/plain';
return join "n",
map { $_->title . ' / ' . $_->artist }
@mp3s;
};
47. LPW
8th November 2014
/mp3 Route
In MP3/lib/MP3.pm
get '/mp3' => sub {
my @mp3s = schema->resultset('Mp3')->all;
content_type 'text/plain';
return join "n",
map { $_->title . ' / ' . $_->artist }
@mp3s;
};
57. LPW
8th November 2014
Plain Text?
Yes, plain text is bad
Easy fix
Serializer: JSON
In config.yml
Rewrite routes to return data structures
Dancer serialises them as JSON
58. LPW
8th November 2014
Return Data
get '/mp3' => sub {
my @mp3s = schema->
resultset('Mp3')->all;
return { mp3s => [
map { {
title => $_->title,
artist => $_->artist
} } @mp3s
] };
};
59. LPW
8th November 2014
Return Data
get '/mp3' => sub {
my @mp3s = schema->
resultset('Mp3')->all;
return { mp3s => [
map { {
title => $_->title,
artist => $_->artist
} } @mp3s
] };
};
60. LPW
8th November 2014
Return Data
get '/mp3' => sub {
my @mp3s = schema->
resultset('Mp3')->all;
return { mp3s => [
map { {
title => $_->title,
artist => $_->artist
} } @mp3s
] };
};
61. LPW
8th November 2014
Return Data
get '/mp3' => sub {
my @mp3s = schema->
resultset('Mp3')->all;
return { mp3s => [
map { {
title => $_->title,
artist => $_->artist
} } @mp3s
] };
};
62. LPW
8th November 2014
Return Data
get '/mp3' => sub {
my @mp3s = schema->
resultset('Mp3')->all;
return { mp3s => [
map { {
title => $_->title,
artist => $_->artist
} } @mp3s
] };
};
63. LPW
8th November 2014
Return Data
get '/mp3/:id' => sub {
my $mp3 = schema->
resultset('Mp3')->find(param('id'));
unless ($mp3) {
status 404;
return 'Not found';
}
return {
title => $mp3->title,
artist => $mp3->artist,
filename => $mp3->filename,
};
};
64. LPW
8th November 2014
Return Data
get '/mp3/:id' => sub {
my $mp3 = schema->
resultset('Mp3')->find(param('id'));
unless ($mp3) {
status 404;
return 'Not found';
}
return { $mp3->get_columns };
};
69. LPW
8th November 2014
URLs
It's good practice to return URLs when you
can
Easier for clients to browse our data
We can do that for our list
70. LPW
8th November 2014
URLs
get '/mp3' => sub {
my @mp3s = schema->resultset('Mp3')->all;
my $url = uri_for('/mp3') . '/';
return { mp3s => [
map { {
title => $_->title,
artist => $_->artist,
url => $url . $_->id,
} } @mp3s
] };
};
71. LPW
8th November 2014
URLs
get '/mp3' => sub {
my @mp3s = schema->resultset('Mp3')->all;
my $url = uri_for('/mp3') . '/';
return { mp3s => [
map { {
title => $_->title,
artist => $_->artist,
url => $url . $_->id,
} } @mp3s
] };
};
72. LPW
8th November 2014
URLs
$ GET http://localhost:3000/mp3
{"mp3s":[
{
"url":"http://localhost:3000/mp3/1",
"title":"Royals","artist":"Lorde"
},
{
"url":"http://localhost:3000/mp3/2",
"artist":"Chvrches","title":"The Mother We Share"
},
{
"url":"http://localhost:3000/mp3/3",
"artist":"Haim","title":"Falling"
}
]}
73. LPW
8th November 2014
More on URLs
Currently our system has only one resource
mp3
It's usual to have links to other resources
MP3s have artists
Link to other resources using URLs
Make it easier for clients to walk our data
model
74. LPW
8th November 2014
More on URLs
In our MP3 JSON we have this
“artist”:”Lorde”
It would be better to have
“artist_name”:”Lorde”
“artist_url”:”http://localhost:3000/artist/1”
Perhaps add a url() method to all of our
objects
75. LPW
8th November 2014
Other GET Actions
Getting lists of objects is easy
Other things to consider
Searching
Sorting
Paging
Filtering
CGI Parameters to DBIC to SQL to JSON
76. LPW
8th November 2014
Other Actions
We will want to to other things to our data
Add objects
Update objects
Delete objects
CRUD operations
77. LPW
8th November 2014
Other Actions
Use HTTP methods
POST /mp3
Create
GET /mp3/:id
Read
PUT /mp3/:id
Update
DELETE /mp3/:id
Delete
78. LPW
8th November 2014
Other Actions
Use HTTP methods
POST /mp3
Create
GET /mp3/:id
Read
PUT /mp3/:id
Update
DELETE /mp3/:id
Delete
79. LPW
8th November 2014
Other Actions
Easy to write Dancer handlers for these
delete '/mp3/:id' => sub {
schema->resultset('Mp3')->
find(param('id'))->
delete;
}
But it can be hard to get right
What should we return here?
Is there a better way?
81. Representational State Transfer
Abstraction of web architecture
Dissertation by Roy Fielding, 2000
Particularly applicable to web services
LPW
REST
8th November 2014 81
82. Base URI for service
Defined media type
HTTP methods for interaction
Hypertext links for resources
Hypertext links for related resources
LPW
RESTful Web Services
8th November 2014 82
83. RESTful vs Non-RESTful
Good test
Which HTTP methods does it use?
Web services often use only GET and POST
GET /delete/mp3/1
GET /mp3/1/delete
Not RESTful
DELETE /mp3/1
Might be RESTful
LPW
8th November 2014 83
84. Dancer has a REST plugin
Dancer2::Plugin::REST
Makes our live much easier
LPW
RESTful Dancer
8th November 2014 84
85. Does three things for us
Creates routes
Utility functions for return values
Returns data in different formats
LPW
Dancer2::Plugin::REST
8th November 2014 85
86. resource mp3 =>
LPW
Creates Routes
get => sub { ... },
create => sub { ... },
delete => sub { ... },
update => sub { ... };
8th November 2014 86
87. resource mp3 =>
LPW
Creates Routes
get => sub { ... },
create => sub { ... },
delete => sub { ... },
update => sub { ... };
8th November 2014 87
88. post '/mp3'
get '/mp3/:id'
put '/mp3/:id'
delete '/mp3/:id'
LPW
CRUD Routes
Create
Read
Update
Delete
8th November 2014 88
89. post '/mp3'
get '/mp3/:id'
put '/mp3/:id'
delete '/mp3/:id'
LPW
CRUD Routes
Create
Read
Update
Delete
8th November 2014 89
90. resource mp3 =>
LPW
Creates Routes
get => sub {
my $mp3 =
schema->resultset('Mp3')->
find(params->{id});
if ($mp3) {
status_ok( { $mp3->get_columns } );
} else {
status_not_found('MP3 Not Found');
}
};
8th November 2014 90
91. Note: Still have to create main listing route
/mp3
LPW
Creates Routes
8th November 2014 91
93. Allow user to choose data format
By changing the URL
get '/mp3/:id'
get '/mp3:id.:format'
YAML, JSON, Data::Dumper support built-in
LPW
Format Options
8th November 2014 93
94. Format Options - JSON
$ GET http://localhost:3000/mp3/1.json
{"id":1,"filename":"music/lorde/pure-heroine/
LPW
royals.mp3","title":"Royals","ar
tist":"Lorde"}
8th November 2014 94
95. $ GET http://localhost:3000/mp3/1.yml
---
artist: Lorde
filename: music/lorde/pure-heroine/
LPW
Format Options -YAML
royals.mp3
id: 1
title: Royals
8th November 2014 95
97. Dancer and Dancer2::Plugin::REST make
simple REST easy
But full REST support is more complex
Here's a REST state machine
LPW
More Complex REST
Other frameworks do the same
8th November 2014 97
102. There's a lot to think about when getting
REST right
Can CPAN help?
Web::Machine
WebAPI::DBIC
LPW
CPAN to the Rescue
8th November 2014 102
103. Perl port of Erlang webmachine
You write subclasses of
Web::Machine::Resource
Override methods where necessary
See Stevan Little's YAPC::NA 2012 talk
LPW
Web::Machine
With bits stolen from Ruby and Javascript
versions too
8th November 2014 103
104. use Web::Machine;
LPW
Example
{
package WasteOfTime::Resource;
use parent 'Web::Machine::Resource';
use JSON::XS qw(encode_json);
sub content_types_provided {
[{ 'application/json' => 'to_json' }]
}
sub to_json {
encode_json({ time => scalar localtime })
}
}
Web::Machine->new(
resource => 'WasteOfTime::Resource'
)->to_app;
8th November 2014 104
105. $ plackup time.psgi
$ curl -v http://0:5000
LPW
Example
HTTP::Server::PSGI: Accepting connections at
http://0:5000/
[ ... ]
< HTTP/1.0 200 OK
< Date: Sat, 08 Nov 2014 11:34:02 GMT
< Server: HTTP::Server::PSGI
< Content-Length: 35
< Content-Type: application/json
<
* Closing connection #0
{"time":"Sat Nov 8 11:34:02 2014"}
8th November 2014 105
106. $ curl -v http://0:5000 -H'Accept: text/html'
LPW
Example
[ ... ]
< HTTP/1.0 406 Not Acceptable
< Date: Sat, 08 Nov 2014 11:34:02 GMT
< Server: HTTP::Server::PSGI
< Content-Length: 14
<
* Closing connection #0
Not Acceptable
8th November 2014 106
107. sub content_types_provided { [
LPW
Example
{ 'application/json' => 'to_json' },
{ 'text/html' => 'to_html' },
] }
8th November 2014 107
108. sub content_types_provided { [
LPW
Example
{ 'application/json' => 'to_json' },
{ 'text/html' => 'to_html' },
] }
8th November 2014 108
109. sub content_types_provided { [
sub to_html {
LPW
Example
{ 'application/json' => 'to_json' },
{ 'text/html' => 'to_html' },
] }
my $time = localtime;
return “<html>
<head><title>The Time Now Is:</title></head>
<body>
<h1>$time</h1>
</body>
</html>”;
}
8th November 2014 109
110. $ curl -v http://0:5000 -H'Accept: text/html'
LPW
Example
[ ... ]
< HTTP/1.0 200 OK
< Date: Sun, 09 Dec 2012 02:26:39 GMT
< Server: HTTP::Server::PSGI
< Vary: Accept
< Content-Length: 103
< Content-Type: text/html
<
* Closing connection #0
<html><head><title>The Time Now
Is:</title></head><body><h1>Sat Nov 8 11:34:02
2014</h1></body></html>
8th November 2014 110
111. “WebAPI::DBIC provides the parts you need
to build a feature-rich RESTful JSON web
service API backed by DBIx::Class
schemas.”
REST API in a box
LPW
WebAPI::DBIC
8th November 2014 111
112. Built on top of Web::Machine
And Path::Router
And Plack
Uses JSON+HAL
LPW
WebAPI::DBIC
Hypertext Application Language
8th November 2014 112
113. $ git clone https://github.com/timbunce/WebAPI-DBIC.
$ cd WebAPI-DBIC
$ cpanm Module::CPANfile
$ cpanm --installdeps . # wait ...
$ export WEBAPI_DBIC_SCHEMA=DummyLoadedSchema
$ plackup -Ilib -It/lib webapi-dbic-any.psgi
... open a web browser on port 5000 to
browse the API
LPW
Try It Out
git
8th November 2014 113
114. Try It Out (Your Schema)
$ export WEBAPI_DBIC_SCHEMA=Foo::Bar
$ export WEBAPI_DBIC_HTTP_AUTH_TYPE=none
$ export DBI_DSN=dbi:Driver:...
$ export DBI_USER=...
$ export DBI_PASS=...
$ plackup -Ilib webapi-dbic-any.psgi
... open a web browser on port 5000 to
browse the API
LPW
8th November 2014 114
116. Perl is great for the Internet of Things
Perl is great for text processing
Perl is great for network processing
IoT apps are often mainly HTTP transactions
LPW
Conclusion
And Perl is good at those
8th November 2014 116
117. Perl Training
Central London
Next week
Intermediate Perl
Advanced Perl
See advert in brochure
LPW
Sponsor's Message
11/12 Nov
13/14 Nov
8th November 2014 117
118. Perl Training
Central London
Next week
Intermediate Perl
Advanced Perl
See advert in brochure
LPW
Sponsor's Message
11/12 Nov
13/14 Nov
8th November 2014 118