Running Coherence Clustered Cache
on Amazon EC2
In The Brain of Daniel Sikar
Skills Matter, London, UK
09.11.2010
Skills Matter supports the Agile and Open Source
developer community, by organising free events,
training courses, confere...
Daniel Sikar
Consultant
Developer
Trainer
Internet & scientific data acquisition applications
Delivers AWS, Oracle Coheren...
Running Coherence Clustered Cache
on Amazon EC2
Why use AWS?
Other PAAS infrastructures:
● GoGrid
● Rackspace Cloud
● Eucalyptus
● PlanetLab
● OpenCirrus
● SARA
EC2 Environment - abridged
Regions
Instances
AMIs
Security Groups
Regions
AP-SOUTHEAST-1
Availability zones: ap-southeast-{1a, 1b}
EU-WEST-1
Availability zones: eu-west-{1a, 1b}
US-EAST-1
...
EC2 Environment - abridged
INSTANCES = Running virtual machines
AMIs = Virtual machine images
SECURITY GROUPS = Firewall c...
Amazon Machine Image (AMI)
$ ec2-run-instances (...)
ec2-api-tools
$ env
$ cd bin
$ ls
$ ec2-describe-instances
$ ec2-describe-images
ec2-api-tools
AWS Public Images - 31.01.2010
REGIONS
● EU-West 1207
● US-East 4521
● US-West 343
Public AMIs
0
500
1000
1500
2000
2500
3...
AWS Public Images - 31.01.2010
Operating Systems
● Flavours of Unix 5707
● Windows 364
Public AMIs
0
1000
2000
3000
4000
5...
AWS Public Images - 31.01.2010
Architecture
● i386 4560
● x86_64 1511
Public AMIs
0
500
1000
1500
2000
2500
3000
3500
4000...
Public AMIs Region/OS/Architecture
eu-w est us-east us-w est
0
500
1000
1500
2000
2500
3000
3500
4000
L32
L64
W32
W64
EU-WEST Top 5 Public AMI Owners
● JumpBox 204
● Alestic 198
● Canonical 177
● Amazon 161
● BitNami 60
Public AMI Ow ners
0...
US-EAST Top 5 Public AMI Owners
● rPath 776
● JumpBox 588
● Alestic 363
● Canonical 319
● Amazon 169
Public AMI Ow ners
0
...
US-WEST Top 5 Public AMI Owners
● Canonical 100
● Amazon 79
● Alestic 53
● Right Scale 14
● Debian Zone 13
Public AMI Ow n...
Public AMIs
$ ec2dim -a > amis.txt
$ cat amis.txt | grep moodle
$ ping <public dns>
$ c:<...>putty.exe
Public AMIs
TURNKEY SOLUTIONS
Why use caching?
Share objects across servers
Lower response time
Increase throughput
Increase availability
Distributed/pa...
Design Patterns
● JSR-107 JCACHE – Java Temporary Caching
API (2001)
● Tangosol (2000 - 2007)
● Oracle Coherence (2007 - 2010)
Coherence t...
Coherence Key Concepts
● Clustering
● Availability
● Reliability
● Scalability
Coherence Key Concepts
● Partitioning
● Replication
● Near Caching
Coherence Key Concepts
● CacheStore
● Write-Behind
● Refresh-Ahead
● Write-Through
● Read-Through
Coherence Networking Protocols
Coherence uses TCMP, a combination of:
● Multicast
● Unicast
● TCP
WKA
Well-Known-Addresses
Used in environments where multicast is not
allowed or not suitable.
WKA Configuration
Command line:
-Dtangosol.coherence.wka=<ip address>
-Dtangosol.coherence.wka.port=<port number>
WKA Configuration
Configuration file:
<?xml version='1.0'?>
<!DOCTYPE coherence SYSTEM "coherence.dtd">
<coherence xml-ove...
EC2AddressProvider
Looks for instances with assigned Elastic IP
EC2AddressProvider
Method Summary
accept()
determineCredentials()
generateWKAList(...)
getNextAddress
reject(Throwable exc...
EC2AddressProvider
Authentication of EC2 calls:
Java system properties:
● tangosol.coherence.ec2addressprovider.accesskey=...
EC2AddressProvider
Libraries:
● AWS SDK for Java
● Coherence
● Coherence Common (Incubator) – class
com.oracle.coherence.c...
Coherence Clustered Cache
+
EC2
=
Running Coherence on virtual machines
Windows EC2 Instance
Configuration
$ ec2auth default -P icmp -t -1:-1
+ Firewall (switch off instance firewall)
Accessing a Windows EC2 Instance running
Coherence Clustered Caching
Access
$ ec2-describe-addresses
$ mstsc
Access
# start cache server
$ cache-server.cmd
# start clients
$ coherence.cmd
# run
# help, cache, put, assert
# maps, list, loc...
Conclusion
Setting up and running
Coherence Clustered Cache is simple on
PAAS environments
such as Amazon EC2
Questions?
Coming up next Nov-Dec 2010:
AWS EC2 S3 Workshop - 1 day
15.11.2010
Coherence Clustered Caching - 2 days
16.12.2010
Cohere...
Thank you for attending!
Running Coherence Clustered Cache
on Amazon EC2
In The Brain of Daniel Sikar
Skills Matter, Londo...
Daniel Sikar: Running Coherence Clustered Cache on Amazon EC2 - 09/11/2010
Daniel Sikar: Running Coherence Clustered Cache on Amazon EC2 - 09/11/2010
Daniel Sikar: Running Coherence Clustered Cache on Amazon EC2 - 09/11/2010
Daniel Sikar: Running Coherence Clustered Cache on Amazon EC2 - 09/11/2010
Upcoming SlideShare
Loading in …5
×

Daniel Sikar: Running Coherence Clustered Cache on Amazon EC2 - 09/11/2010

5,073 views
4,870 views

Published on

In this 'In the brain' session speaker Daniel Sikar talks on the topic titled 'Running Coherence Clustered Cache on Amazon EC2'

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,073
On SlideShare
0
From Embeds
0
Number of Embeds
107
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Daniel Sikar: Running Coherence Clustered Cache on Amazon EC2 - 09/11/2010

  1. 1. Running Coherence Clustered Cache on Amazon EC2 In The Brain of Daniel Sikar Skills Matter, London, UK 09.11.2010
  2. 2. Skills Matter supports the Agile and Open Source developer community, by organising free events, training courses, conferences and by publishing thousands of podcasts on ideas and technologies that drive innovation. 116-120 Goswell Road, London, EC1V 7DP Tel: +44 (0)207 183 9040 info@skillsmatter.com Welcome to Skills Matter
  3. 3. Daniel Sikar Consultant Developer Trainer Internet & scientific data acquisition applications Delivers AWS, Oracle Coherence and Hadoop courses at Skills Matter dsikar@gmail.com
  4. 4. Running Coherence Clustered Cache on Amazon EC2
  5. 5. Why use AWS? Other PAAS infrastructures: ● GoGrid ● Rackspace Cloud ● Eucalyptus ● PlanetLab ● OpenCirrus ● SARA
  6. 6. EC2 Environment - abridged Regions Instances AMIs Security Groups
  7. 7. Regions AP-SOUTHEAST-1 Availability zones: ap-southeast-{1a, 1b} EU-WEST-1 Availability zones: eu-west-{1a, 1b} US-EAST-1 Availability zones: us-east-{1a, 1b, 1c, 1d} US-WEST-1 Availability zones: us-west-{1a, 1b}
  8. 8. EC2 Environment - abridged INSTANCES = Running virtual machines AMIs = Virtual machine images SECURITY GROUPS = Firewall configurations
  9. 9. Amazon Machine Image (AMI)
  10. 10. $ ec2-run-instances (...) ec2-api-tools
  11. 11. $ env $ cd bin $ ls $ ec2-describe-instances $ ec2-describe-images ec2-api-tools
  12. 12. AWS Public Images - 31.01.2010 REGIONS ● EU-West 1207 ● US-East 4521 ● US-West 343 Public AMIs 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 eu-w est us-east us-w est
  13. 13. AWS Public Images - 31.01.2010 Operating Systems ● Flavours of Unix 5707 ● Windows 364 Public AMIs 0 1000 2000 3000 4000 5000 6000 *nix Window s
  14. 14. AWS Public Images - 31.01.2010 Architecture ● i386 4560 ● x86_64 1511 Public AMIs 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 i386 x86_64
  15. 15. Public AMIs Region/OS/Architecture eu-w est us-east us-w est 0 500 1000 1500 2000 2500 3000 3500 4000 L32 L64 W32 W64
  16. 16. EU-WEST Top 5 Public AMI Owners ● JumpBox 204 ● Alestic 198 ● Canonical 177 ● Amazon 161 ● BitNami 60 Public AMI Ow ners 0 50 100 150 200 250 JumpBox Alestic Canonical Amazon BitNami
  17. 17. US-EAST Top 5 Public AMI Owners ● rPath 776 ● JumpBox 588 ● Alestic 363 ● Canonical 319 ● Amazon 169 Public AMI Ow ners 0 100 200 300 400 500 600 700 800 900 rPath JumpBox Alestic Canonical Amazon
  18. 18. US-WEST Top 5 Public AMI Owners ● Canonical 100 ● Amazon 79 ● Alestic 53 ● Right Scale 14 ● Debian Zone 13 Public AMI Ow ners 0 20 40 60 80 100 120 Canonical Amazon Alestic Right Scale Debian Zone
  19. 19. Public AMIs $ ec2dim -a > amis.txt $ cat amis.txt | grep moodle $ ping <public dns> $ c:<...>putty.exe
  20. 20. Public AMIs TURNKEY SOLUTIONS
  21. 21. Why use caching? Share objects across servers Lower response time Increase throughput Increase availability Distributed/parallel processing Failover
  22. 22. Design Patterns
  23. 23. ● JSR-107 JCACHE – Java Temporary Caching API (2001) ● Tangosol (2000 - 2007) ● Oracle Coherence (2007 - 2010) Coherence timeline
  24. 24. Coherence Key Concepts ● Clustering ● Availability ● Reliability ● Scalability
  25. 25. Coherence Key Concepts ● Partitioning ● Replication ● Near Caching
  26. 26. Coherence Key Concepts ● CacheStore ● Write-Behind ● Refresh-Ahead ● Write-Through ● Read-Through
  27. 27. Coherence Networking Protocols Coherence uses TCMP, a combination of: ● Multicast ● Unicast ● TCP
  28. 28. WKA Well-Known-Addresses Used in environments where multicast is not allowed or not suitable.
  29. 29. WKA Configuration Command line: -Dtangosol.coherence.wka=<ip address> -Dtangosol.coherence.wka.port=<port number>
  30. 30. WKA Configuration Configuration file: <?xml version='1.0'?> <!DOCTYPE coherence SYSTEM "coherence.dtd"> <coherence xml-override="/tangosol-coherence-override.xml"> <well-known-addresses> <socket-address id="1"> <address>10.228.22.128</address> <port>8088</port> </socket-address> <socket-address id="2"> <address>10.228.79.164</address> <port>8088</port> </socket-address> </well-known-addresses> </coherence>
  31. 31. EC2AddressProvider Looks for instances with assigned Elastic IP
  32. 32. EC2AddressProvider Method Summary accept() determineCredentials() generateWKAList(...) getNextAddress reject(Throwable exception)
  33. 33. EC2AddressProvider Authentication of EC2 calls: Java system properties: ● tangosol.coherence.ec2addressprovider.accesskey=<access key> ● tangosol.coherence.ec2addressprovider.secretkey=<secret key> Embedded properties file: ● accessKey=<access key> ● secretKey=<secret key>
  34. 34. EC2AddressProvider Libraries: ● AWS SDK for Java ● Coherence ● Coherence Common (Incubator) – class com.oracle.coherence.cloud.amazon.EC2AddressProvider
  35. 35. Coherence Clustered Cache + EC2 = Running Coherence on virtual machines
  36. 36. Windows EC2 Instance Configuration $ ec2auth default -P icmp -t -1:-1 + Firewall (switch off instance firewall)
  37. 37. Accessing a Windows EC2 Instance running Coherence Clustered Caching Access
  38. 38. $ ec2-describe-addresses $ mstsc Access
  39. 39. # start cache server $ cache-server.cmd # start clients $ coherence.cmd # run # help, cache, put, assert # maps, list, lock, unlock, delete Basic Cache Operations
  40. 40. Conclusion Setting up and running Coherence Clustered Cache is simple on PAAS environments such as Amazon EC2
  41. 41. Questions?
  42. 42. Coming up next Nov-Dec 2010: AWS EC2 S3 Workshop - 1 day 15.11.2010 Coherence Clustered Caching - 2 days 16.12.2010 Coherence Clustered Caching - 2 days 13.12.2010 Introduction to Hadoop - 2 days 20.12.2010 Skills Matter – London, UK
  43. 43. Thank you for attending! Running Coherence Clustered Cache on Amazon EC2 In The Brain of Daniel Sikar Skills Matter, London, UK 09.11.2010

×