0
Becoming a Command Line Expert with the AWS
CLI
James Saryerwinnie, Amazon Web Services
November 14, 2013

© 2013 Amazon.c...
AWS Command Line Interface
AWS Command Line Interface
Unified tool to manage your AWS services
One Tool
One Tool
Installation
One Tool
Installation
Configuration
One Tool
Installation
Configuration
Installation
Windows:
32-bit MSI: http://s3.amazonaws.com/aws-cli/AWSCLI32.msi
64-bit MSI: http://s3.amazonaws.com/aws-cli...
Bundled Installer
$
$
$
$

wget http://s3.amazonaws.com/aws-cli/awscli-bundle.zip
unzip awscli-bundle.zip
./install
~/.loc...
Pip
$ pip install --upgrade awscli
One Tool
Installation
Configuration
Credential Configuration
IAM Role
Automatic

Environment
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

Config File: ~/.aws/conf...
Configuring
$ aws configure
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]:
Default ou...
Configuring
[default]
aws_access_key_id = EXAMPLE
aws_secret_access_key = EXAMPLEKEY
region = us-west-2
output = json
One Tool
Installation
Configuration
One Tool
Installation
Configuration
Demo
Let’s run a command
$ aws ec2 describe-instances
{
"Reservations": []
}
$ aws ec2 describe-instances
{
"Reservations": []
}
$ aws ec2 describe-instances
{
"Reservations": []
}
$ aws ec2 describe-instances
{
"Reservations": []
}
$ aws ec2 describe-instances
{
"Reservations": []
}
$ aws ec2 describe-instances
service (command)

operation (subcommand)
Command with Arguments
$ aws ec2 import-key-pair 
--key-name mykey

--public-key-material file:///home/user/.ssh/id_rsa.pu...
Command with Arguments
$ aws ec2 import-key-pair 
--key-name mykey

--public-key-material file:///home/user/.ssh/id_rsa.pu...
Command with Arguments
$ aws ec2 import-key-pair 
--key-name mykey

--public-key-material file:///home/user/.ssh/id_rsa.pu...
For any parameter value

Feature

• file://<filename>
• http://<url>
• https://<url>
How do I know what arguments to
use?
$ aws ec2 create-security-group help
$ aws ec2 create-security-group help
http://docs.aws.amazon.com/cli/latest/reference/ec2/create-security-group.html
Append help to any command

Feature

• aws service operation help
• aws service help
• aws help
Tab Completion
Bash
zsh
tcsh

$ complete -C aws_completer aws
$ source bin/aws_zsh_completer.sh
$ complete aws 'p/*/`aws_completer`/'x
$ aws ec2 describe-instances
{
"Reservations": [
{
"OwnerId": "",
"ReservationId": "r-12345",
"Groups": [
{
"GroupName": "...
Output Formats
JSON

Table

Text

Programmatic processing

Interactive browsing

Piping to text tools

Integrate with JSON...
Multiple output formats

Feature

• --output json
• --output table
• --output text
Demo
Regions[*].RegionName
{
"Regions": [{
"Endpoint": "…",
"RegionName": "eu-west-1"
}, {
"Endpoint": "…",
"RegionName": "us-e...
Reservations[*].Instances[*].[InstanceId,State.Name]
{"Reservations": [
"Instances": [{
"InstanceId": "i-1",
"State": {"Na...
Reservations[*].Instances[*].{ID: InstanceId,State: State.Name}
{"Reservations": [
"Instances": [{
"InstanceId": "i-1",
"S...
Data

Query

Result

{"foo": "bar"}

foo

"bar"

{"foo": {"bar": "baz"}}

foo.bar

baz

{"foo": [0, 1]}

foo[1]

1

{"bar"...
Query response data

Feature

• Use --query to create the
exact output you want.
http://jmespath.readthedocs.org/en/latest...
aws ec2 describe-instances
…
--filters Name=instance-state-name,Values=running
--filters (list)
A list of filters used to match properties for Instances. For a complete reference to the available filte...
--filters (list)
A list of filters used to match properties for Instances. For a complete reference to the available filte...
--filters (list)
A list of filters used to match properties for Instances. For a complete reference to the available filte...
--filters
Name=instance-state-name,Values=running

[{"Name": "instance-state-name",
"Values":["running"]}]
Shorthand Syntax

Feature

• Use Shorthand Syntax to
specify parameter values
Large Responses
1

100000
1

1000

2000
1

100000
Page Size
NextToken
1

1000

2000

--max-items 3500
1

1000

2000

--max-items 3500 --starting-token <blob>
Pagination

Feature

Use --starting-token
and --max-items to
paginate results
We've
Learned

•
•
•
•
•

file://
aws help
Tab Completion
--output
--query

• Shorthand Syntax
• Pagination
Static Blog with Amazon S3
•
•
•
•

Create blog locally
Sync to Amazon S3
AWS Identity and Access Management (IAM)
Amazon ...
Amazon S3
$ aws s3 mb s3://www.reinvent-cli-blog-demo.com/
$ aws s3 website www.reinvent-cli-blog-demo.com 
--index-docume...
Create a user that has access only to the static blog

Full access only to the www.reinvent-cli-blog-demo.com bucket
AWS IAM
$ aws iam create-user --user-name static-blog
{
"User": {
"UserName": "static-blog",
"Path": "/",
"CreateDate": "2...
AWS IAM
$ aws iam create-access-key --user-name static-blog
{
"AccessKey": {
"UserName": "static-blog",
"Status": "Active"...
AWS IAM
$ aws iam put-user-policy --user-name static-blog 
--policy-name static-blog-s3-access 
--policy-document file://s...
AWS IAM
$ aws iam put-user-policy --user-name static-blog 
--policy-name static-blog-s3-access 
--policy-document file://s...
{

AWS IAM
"Statement": [
{
"Sid": "Stmt12345678",
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::ww...
How do I tell the CLI about this new
user?
aws configure --profile staticblog
[default]
…
[profile staticblog]
aws_access_key = ACCESS_KEY
aws_secrete_access_key = SECRET_KEY
region = us-west-2
Pushing Blog Content
aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ 
--acl public-read 
--delete 
--profile staticblog
Pushing Blog Content
aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ 
--acl public-read 
--delete 
--profile staticblog
Pushing Blog Content
aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ 
--acl public-read 
--delete 
--profile staticblog
Pushing Blog Content
aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ 
--acl public-read 
--delete 
--profile staticblog
Pushing Blog Content
aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ 
--acl public-read 
--delete 
--profile staticblog
Amazon S3 Sync
Amazon S3 Sync
Amazon S3 Sync
smallfile

delete-file

largefile-part-1
largefile-part-2

largefile-part-4

largefile-part-3
Amazon S3 Sync

Feature

• Sync new/changed files
• Files uploaded in parallel
• Large files split into
chunks
Amazon Route53
Amazon Route53
$ aws route53 create-hosted-zone --name www.reinvent-cli-blog-demo.com 
--caller-reference reinvent-cli-blo...
Amazon Route53
aws route53 change-resource-record-sets --hosted-zone-id "/hostedzone/Z1TI9W0V4R87XY" 
--change-batch file:...
Amazon Route53
file://changebatch.json
{
"Comment": "Add S3 Bucket",
"Changes": [
{
"Action": "CREATE",
"ResourceRecordSet...
Demo
Additional Topics
Botocore
_regions.json
_retry.json
autoscaling.json
cloudformation.json
cloudfront.json
cloudsearch.json
cloudwatch.json
d...
ec2.json
"api_version": "2013-10-01",
"type": "query",
"signature_version": "v2",
"service_full_name": "Amazon Elastic Com...
_retry.json
"dynamodb": {
"__default__": {
"max_attempts": 10,
"delay": {
"type": "exponential",
"base": 0.05,
"growth_fac...
Plugins
Next Steps
•
•
•

•

Check out the code on Github: http://github.com/aws/aws-cli
Report bugs at Issues: http://github.com/...
Please give us your feedback on this
presentation

TLS304
As a thank you, we will select prize
winners daily for completed...
Upcoming SlideShare
Loading in...5
×

The Best of Both Worlds: Implementing Hybrid IT with AWS (ENT218) | AWS re:Invent 2013

4,103

Published on

(Presented by RightScale) With the increased use of cloud services, organizations are faced with finding the most efficient way to leverage existing IT infrastructure alongside cloud-based compute, storage, and networking resources. This has resulted in the rise of hybrid infrastructure so IT teams can deliver agility and performance with visibility and control.
At RightScale, we’ve implemented some of the world’s largest hybrid IT deployments. In this session, we share implementation approaches, architecture design considerations, and steps for a successful hybrid IT model.

This session covers:
-How to develop a strategy and framework for a successful path to hybrid IT
-How to prioritize applications for public cloud versus on-premises
-How to manage multiple compute resource pools through a unified management framework
-Implementation and continuous improvement of a hybrid IT environment

Examples of enterprise hybrid IT implementations include cloudbursting, high availability, and disaster recovery.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,103
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
54
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "The Best of Both Worlds: Implementing Hybrid IT with AWS (ENT218) | AWS re:Invent 2013"

  1. 1. Becoming a Command Line Expert with the AWS CLI James Saryerwinnie, Amazon Web Services November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. AWS Command Line Interface
  3. 3. AWS Command Line Interface Unified tool to manage your AWS services
  4. 4. One Tool
  5. 5. One Tool Installation
  6. 6. One Tool Installation Configuration
  7. 7. One Tool Installation Configuration
  8. 8. Installation Windows: 32-bit MSI: http://s3.amazonaws.com/aws-cli/AWSCLI32.msi 64-bit MSI: http://s3.amazonaws.com/aws-cli/AWSCLI64.msi Bundled Installer: http://aws-cli.s3.amazonaws.com/awscli-bundle.zip
  9. 9. Bundled Installer $ $ $ $ wget http://s3.amazonaws.com/aws-cli/awscli-bundle.zip unzip awscli-bundle.zip ./install ~/.local/lib/aws/bin/aws --version
  10. 10. Pip $ pip install --upgrade awscli
  11. 11. One Tool Installation Configuration
  12. 12. Credential Configuration IAM Role Automatic Environment AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY Config File: ~/.aws/config aws_access_key_id aws_secret_access_key
  13. 13. Configuring $ aws configure AWS Access Key ID [None]: AWS Secret Access Key [None]: Default region name [None]: Default output format [json]:
  14. 14. Configuring [default] aws_access_key_id = EXAMPLE aws_secret_access_key = EXAMPLEKEY region = us-west-2 output = json
  15. 15. One Tool Installation Configuration
  16. 16. One Tool Installation Configuration
  17. 17. Demo
  18. 18. Let’s run a command
  19. 19. $ aws ec2 describe-instances { "Reservations": [] }
  20. 20. $ aws ec2 describe-instances { "Reservations": [] }
  21. 21. $ aws ec2 describe-instances { "Reservations": [] }
  22. 22. $ aws ec2 describe-instances { "Reservations": [] }
  23. 23. $ aws ec2 describe-instances { "Reservations": [] }
  24. 24. $ aws ec2 describe-instances service (command) operation (subcommand)
  25. 25. Command with Arguments $ aws ec2 import-key-pair --key-name mykey --public-key-material file:///home/user/.ssh/id_rsa.pub
  26. 26. Command with Arguments $ aws ec2 import-key-pair --key-name mykey --public-key-material file:///home/user/.ssh/id_rsa.pub
  27. 27. Command with Arguments $ aws ec2 import-key-pair --key-name mykey --public-key-material file:///home/user/.ssh/id_rsa.pub
  28. 28. For any parameter value Feature • file://<filename> • http://<url> • https://<url>
  29. 29. How do I know what arguments to use?
  30. 30. $ aws ec2 create-security-group help
  31. 31. $ aws ec2 create-security-group help
  32. 32. http://docs.aws.amazon.com/cli/latest/reference/ec2/create-security-group.html
  33. 33. Append help to any command Feature • aws service operation help • aws service help • aws help
  34. 34. Tab Completion
  35. 35. Bash zsh tcsh $ complete -C aws_completer aws $ source bin/aws_zsh_completer.sh $ complete aws 'p/*/`aws_completer`/'x
  36. 36. $ aws ec2 describe-instances { "Reservations": [ { "OwnerId": "", "ReservationId": "r-12345", "Groups": [ { "GroupName": "SSH", "GroupId": "sg-abcdefg" } ], "Instances": [ { "State": { "Code": 16, "Name": "running" }, "KeyName": "mykey", "InstanceType": "t1.micro", } ] }, ] … }
  37. 37. Output Formats JSON Table Text Programmatic processing Interactive browsing Piping to text tools Integrate with JSON tools Easier to visually parse Easy to parse
  38. 38. Multiple output formats Feature • --output json • --output table • --output text
  39. 39. Demo
  40. 40. Regions[*].RegionName { "Regions": [{ "Endpoint": "…", "RegionName": "eu-west-1" }, { "Endpoint": "…", "RegionName": "us-east-1" }] } eu-west-1 us-east-1
  41. 41. Reservations[*].Instances[*].[InstanceId,State.Name] {"Reservations": [ "Instances": [{ "InstanceId": "i-1", "State": {"Name": "running"} }, { "InstanceId": "i-2", "State": {"Name": "stopped"} }]} i-1 i-2 running stopped
  42. 42. Reservations[*].Instances[*].{ID: InstanceId,State: State.Name} {"Reservations": [ "Instances": [{ "InstanceId": "i-1", "State": {"Name": "running"} }, { "InstanceId": "i-2", "State": {"Name": "stopped"} }]} --------------------------| DescribeInstances | +-------------+-----------+ | ID | State | +-------------+-----------+ | i-1 | running | | i-2 | stopped | +-------------+-----------+
  43. 43. Data Query Result {"foo": "bar"} foo "bar" {"foo": {"bar": "baz"}} foo.bar baz {"foo": [0, 1]} foo[1] 1 {"bar": 1, "baz": 2} foo or bar 1 {"a": 1, "b": 2, "c": 3} [a, b] [1, 2] {"a": 1, "b": 2, "c": 3} {a: a, other: b} {"a": 1, "other": 2} [{"a": 1}, {"a": 2}, {"a": 3}, {"a": 4}] [*].a [1, 2, 3, 4]
  44. 44. Query response data Feature • Use --query to create the exact output you want. http://jmespath.readthedocs.org/en/latest/specification.html
  45. 45. aws ec2 describe-instances … --filters Name=instance-state-name,Values=running
  46. 46. --filters (list) A list of filters used to match properties for Instances. For a complete reference to the available filter keys for this operation, see the Amazon EC2 API reference. Shorthand Syntax: Key value pairs, where values are separated by commas. --filters Name=string1,Values=string1,string2 JSON Syntax: [ { "Name": "string", "Values": ["string", ...] } ... ]
  47. 47. --filters (list) A list of filters used to match properties for Instances. For a complete reference to the available filter keys for this operation, see the Amazon EC2 API reference . Shorthand Syntax: Key value pairs, where values are separated by commas. --filters Name=string1,Values=string1,string2 JSON Syntax: [ { "Name": "string", "Values": ["string", ...] } ... ]
  48. 48. --filters (list) A list of filters used to match properties for Instances. For a complete reference to the available filter keys for this operation, see the Amazon EC2 API reference . Shorthand Syntax: Key value pairs, where values are separated by commas. --filters Name=string1,Values=string1,string2 JSON Syntax: [ { "Name": "string", "Values": ["string", ...] } ... ]
  49. 49. --filters Name=instance-state-name,Values=running [{"Name": "instance-state-name", "Values":["running"]}]
  50. 50. Shorthand Syntax Feature • Use Shorthand Syntax to specify parameter values
  51. 51. Large Responses
  52. 52. 1 100000
  53. 53. 1 1000 2000
  54. 54. 1 100000
  55. 55. Page Size NextToken
  56. 56. 1 1000 2000 --max-items 3500
  57. 57. 1 1000 2000 --max-items 3500 --starting-token <blob>
  58. 58. Pagination Feature Use --starting-token and --max-items to paginate results
  59. 59. We've Learned • • • • • file:// aws help Tab Completion --output --query • Shorthand Syntax • Pagination
  60. 60. Static Blog with Amazon S3 • • • • Create blog locally Sync to Amazon S3 AWS Identity and Access Management (IAM) Amazon Route53
  61. 61. Amazon S3 $ aws s3 mb s3://www.reinvent-cli-blog-demo.com/ $ aws s3 website www.reinvent-cli-blog-demo.com --index-document index.html
  62. 62. Create a user that has access only to the static blog Full access only to the www.reinvent-cli-blog-demo.com bucket
  63. 63. AWS IAM $ aws iam create-user --user-name static-blog { "User": { "UserName": "static-blog", "Path": "/", "CreateDate": "2013-10-18T18:46:39.044Z", "UserId": "EXAMPLEUSERID", "Arn": "arn:aws:iam::12345:user/static-blog" } }
  64. 64. AWS IAM $ aws iam create-access-key --user-name static-blog { "AccessKey": { "UserName": "static-blog", "Status": "Active", "CreateDate": "2013-10-18T18:47:38.913Z", "SecretAccessKey": "SECRET_KEY", "AccessKeyId": ”ACCESS_KEY" } }
  65. 65. AWS IAM $ aws iam put-user-policy --user-name static-blog --policy-name static-blog-s3-access --policy-document file://singlebucket.json
  66. 66. AWS IAM $ aws iam put-user-policy --user-name static-blog --policy-name static-blog-s3-access --policy-document file://singlebucket.json
  67. 67. { AWS IAM "Statement": [ { "Sid": "Stmt12345678", "Action": [ "s3:*" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::www.reinvent-cli-blog-demo.com/*", "arn:aws:s3:::www.reinvent-cli-blog-demo.com" ] } ] }
  68. 68. How do I tell the CLI about this new user?
  69. 69. aws configure --profile staticblog
  70. 70. [default] … [profile staticblog] aws_access_key = ACCESS_KEY aws_secrete_access_key = SECRET_KEY region = us-west-2
  71. 71. Pushing Blog Content aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ --acl public-read --delete --profile staticblog
  72. 72. Pushing Blog Content aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ --acl public-read --delete --profile staticblog
  73. 73. Pushing Blog Content aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ --acl public-read --delete --profile staticblog
  74. 74. Pushing Blog Content aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ --acl public-read --delete --profile staticblog
  75. 75. Pushing Blog Content aws s3 sync . s3://www.reinvent-cli-blog-demo.com/ --acl public-read --delete --profile staticblog
  76. 76. Amazon S3 Sync
  77. 77. Amazon S3 Sync
  78. 78. Amazon S3 Sync smallfile delete-file largefile-part-1 largefile-part-2 largefile-part-4 largefile-part-3
  79. 79. Amazon S3 Sync Feature • Sync new/changed files • Files uploaded in parallel • Large files split into chunks
  80. 80. Amazon Route53
  81. 81. Amazon Route53 $ aws route53 create-hosted-zone --name www.reinvent-cli-blog-demo.com --caller-reference reinvent-cli-blog-demo { "HostedZone": { … "Id": "/hostedzone/Z1TI9W0V4R87XY", "Name": "www.reinvent-cli-blog-demo.com" }, "DelegationSet": { "NameServers": [ "ns-abc.awsdns-20.com", "ns-abcd.awsdns-49.org" .. ] } }
  82. 82. Amazon Route53 aws route53 change-resource-record-sets --hosted-zone-id "/hostedzone/Z1TI9W0V4R87XY" --change-batch file://changebatch.json { "ChangeInfo": { "Status": "PENDING", "Comment": "Add S3 Bucket", "SubmittedAt": "2013-10-31T18:37:34.281Z", "Id": "/change/C1AG4RL3JT78JG" } }
  83. 83. Amazon Route53 file://changebatch.json { "Comment": "Add S3 Bucket", "Changes": [ { "Action": "CREATE", "ResourceRecordSet": { "Name": "www.reinvent-cli-blog-demo.com", "Type": "A", "AliasTarget": { "HostedZoneId": "Z3BJ6K6RIION7M", "EvaluateTargetHealth": false, "DNSName": "s3-website-us-west-2.amazonaws.com" } } } ] }
  84. 84. Demo
  85. 85. Additional Topics
  86. 86. Botocore _regions.json _retry.json autoscaling.json cloudformation.json cloudfront.json cloudsearch.json cloudwatch.json datapipeline.json directconnect.json dynamodb.json ec2.json elasticache.json elasticbeanstalk.json elastictranscoder.json elb.json emr.json iam.json importexport.json opsworks.json rds.json redshift.json route53.json s3.json ses.json sns.json sqs.json storagegateway.json sts.json support.json swf.json
  87. 87. ec2.json "api_version": "2013-10-01", "type": "query", "signature_version": "v2", "service_full_name": "Amazon Elastic Compute Cloud", "service_abbreviation": "Amazon EC2", "endpoint_prefix": "ec2", "operations": { "ActivateLicense": { "name": "ActivateLicense", "input": {…}, "output": {…}, } }
  88. 88. _retry.json "dynamodb": { "__default__": { "max_attempts": 10, "delay": { "type": "exponential", "base": 0.05, "growth_factor": 2 }, "policies": { "throughput_exceeded": { "applies_when": { "response": { "service_error_code": "ProvisionedThroughputExceededException", "http_status_code": 400 } } },
  89. 89. Plugins
  90. 90. Next Steps • • • • Check out the code on Github: http://github.com/aws/aws-cli Report bugs at Issues: http://github.com/aws/aws-cli/issues/ Ask questions on our Forums: https://forums.aws.amazon.com/forum.jspa?forumID=150 Chat with us in the Developer Lounge (Boto 1:30pm, CLI 3:30pm)
  91. 91. Please give us your feedback on this presentation TLS304 As a thank you, we will select prize winners daily for completed surveys!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×