Puppet for JavadevelopersCarlos Sanchez@csanchezhttp://carlossanchez.euhttp://maestrodev.com
Apache  @csanchez                   Maven                                       Apache   ASF                              ...
DevOps
DevQaOps ?
Agile           planning   iterative development  continuous integrationrelease soon, release often
DevOps addresses       Fear of change     Risky deployments It works on my machine!         SiloisationDev Change vs. Ops ...
Individuals and interactions over processes and tools Working software over comprehensive documentation  Customer collabor...
DevOps is NOT about the tools
Nice, BUThow can I implement IT
Nice, BUThow can I implement IT
Tools can enablechange in behavior  and eventually  change culture     Patrick Debois
DevOps toolseveryone is intelligentenoughevery tool is cloudenabledevery tool is DevOps(tm)
DevOps toolseveryone is intelligentenoughevery tool is cloudenabledevery tool is DevOps(tm)
DevOps tools: infrastructure automation                 infrastructure as code      it’s all invented, now it’s standardized
Infrastructure as CodeNew solutions bring new challengesFollow development best practices              tagging            ...
New solutions bring new problems               DEVOPS  “MY MACHINES ARE BEING PROVISIONED”
Puppet                 exec { "maven-untar":manifests          command => "tar xf /tmp/x.tgz",                   cwd => "/...
Puppetinfrastructure IS code                   package { openssh-server:                     ensure => present,           ...
Puppetdeclarative modelstate vs process no scripting                    service { ntp:                      name      => n...
master - agentarchitecture
Vagrant
Vagrant     Oracle VirtualBox cmdline automation       Easy Puppet and Chef provisioning  Keep VM configuration for differe...
Vagrant base boxes  www.vagrantbox.esanywhere! just (big) files
using Vagrant$ vagrant box add centos-6.0-x86_64       http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box$   vagrant   ...
VagrantVagrant::Config.run do |config|  # Every Vagrant virtual environment requires a box to build off of.  config.vm.box...
manifests/base.pppackage { jdk:  ensure => installed,  name    => $operatingsystem ? {     centOS => "java-1.6.0-openjdk-d...
VeeWee
VeeWee   easily build Vagrant base boxeshttps://github.com/jedi4ever/veewee
using VeeWee$ gem install veewee$ vagrant basebox templates$ vagrant basebox define my-ubuntu-server    ubuntu-11.04-serve...
Editors
Geppetto
http://cloudsmith.github.com/geppetto
TextMate
https://github.com/masterzen/puppet-textmate-bundle
Puppet DSL
Puppet resourcesuser { dave:  ensure     =>   present,  uid        =>   507,  gid        =>   admin,  shell      =>   /bin...
Puppet resourcesfile {testfile:  path    => /tmp/testfile,  ensure => present,  mode    => 0640,  content => "Im a test fi...
Puppet standalone$ puppet apply my_test_manifest.pp
orderingfile {/tmp/test1:  ensure => present,  content => "Hi.",}notify {/tmp/test1 has alreadybeen synced.:  require => F...
orderingfile {/tmp/test1:  ensure => present,  content => "Hi.",} ->notify {/tmp/test1 has alreadybeen synced.:}
package / file /service / subscribepackage { openssh-server:  ensure => present,  before => File[/etc/ssh/sshd_config],}fil...
variables$longthing = "Imagine I have something reallylong in here. Like an SSH key, lets say."file {authorized_keys:  pat...
factshost {self:  ensure         =>   present,  name           =>   $::fqdn,  host_aliases   =>   [puppet, $::hostname],  ...
conditionalsif $is_virtual {  service {ntpd:    ensure => stopped,    enable => false,  }}else {  service { ntpd:    name ...
conditionalscase $operatingsystem {  centos, redhat: { $apache = "httpd" }  debian, ubuntu: { $apache = "apache2" }  defau...
class definitionclass ntp {    package { ntp:      ensure => installed,    }    service { ntp:      name      => ntpd,     ...
class declarationclass {myclass:  param1 => a,  param2 => b,}
class declaration (2)include ntp
parameterized classesclass paramclassexample ($value1, $value2 = "Defaultvalue") {  notify {"Value 1 is ${value1}.":}  not...
definitionsdefine myfile ($path) {  file {$path: content => Something}}myfile {a:  path => /tmp/a,}myfile {b:  path => /tmp...
templatingfile {/etc/foo.conf:  ensure => file,  require => Package[foo],  content => template(foo/foo.conf.erb),}
templates/foo.conf.erbOS is <%= $::operatingsystem %>
node configuration# nodes.ppnode someserver.domain.com inherits basenode {    $web_fqdn = www.domain.com    include generic...
Puppet modules
module tool$ puppet module generate csanchez-mymodule
modules{module}/    files/    lib/    manifests/         init.pp         {class}.pp         {defined type}.pp         {nam...
Modulefilename csanchez-mymoduleversion 0.0.1source UNKNOWNauthor csanchezlicense Apache License,Version 2.0summary UNKNOWN...
building a module$ puppet module build
Puppetlabs Forgehttp://forge.puppetlabs.com   A library of modules
librarian-puppethttp://librarian-puppet.com/$ librarian-puppet init# edit Puppetfile$ librarian-puppet install
Puppetfileforge "http://forge.puppetlabs.com"# mod puppetlabs/stdlib# mod ntp,# :git => git://github.com/puppetlabs/puppetl...
Maven and Puppet
What am I doing to automate deployment             Ant tasks plugin             ssh commands             Assembly plugin  ...
What can I do to automate deployment Handle full deployment including infrastructure             not just webapp deploymen...
Maven-Puppet module         A Maven Puppet modulehttps://github.com/maestrodev/puppet-maven   fetches Maven artifacts from...
Installing Maven$repo1 = {  id => "myrepo",  username => "myuser",  password => "mypassword",  url => "http://repo.acme.co...
New Maven typemaven { "/tmp/maven-core-2.2.1.jar":  id => "org.apache.maven:maven-core:jar:2.2.1",  repos => ["http://repo...
New Maven typemaven { "/tmp/maven-core-2.2.1.jar":  groupId => "org.apache.maven",  artifactId => "maven-core",  version =...
Examples
Tomcat cluster + postgres     postgres database         db.acme.com      tomcat servers       tomcat1.acme.com       tomca...
Modules required  puppetlabs/stdlib   puppetlabs/java puppetlabs/apache inkling/postgresqlcamptocamp/tomcat maestrodev/mav...
Puppetfileforge http://forge.puppetlabs.commod   puppetlabs/stdlib, 3.0.1mod   puppetlabs/java, 0.1.6mod   puppetlabs/apach...
inkling - postgresclass { postgresql::server:  config_hash => {     ip_mask_allow_all_users   =>   192.168.0.0/0,     list...
camptocamp - tomcattomcat::instance { appfuse:  ensure    => present,  http_port => 8080,}$webapp = /srv/tomcat/appfuse/we...
maestrodev - mavenmaven { "${webapp}.war":  id      => org.appfuse:appfuse-spring:2.1.0:war,  require => File[/srv/tomcat/...
Puppet Rspec require spec_helperdescribe db.acme.com do  let(:facts) { {    :osfamily => RedHat,    :operatingsystem => Ce...
Example code and slides                     Available at        http://slideshare.carlossanchez.eu           http://github...
Takk!http://blog.carlossanchez.euhttp://maestrodev.comcsanchez@maestrodev.comcarlos@apache.org@csanchez
Photo Credits               Son of Man Lego - Alex Eylarhttp://www.flickr.com/photos/hoyvinmayvin/4702772452/              ...
Puppet for Java developers - JavaZone NO 2012
Puppet for Java developers - JavaZone NO 2012
Puppet for Java developers - JavaZone NO 2012
Puppet for Java developers - JavaZone NO 2012
Puppet for Java developers - JavaZone NO 2012
Puppet for Java developers - JavaZone NO 2012
Puppet for Java developers - JavaZone NO 2012
Upcoming SlideShare
Loading in...5
×

Puppet for Java developers - JavaZone NO 2012

27,130

Published on

Example code at https://github.com/carlossg/puppet-for-java-devs
More info at http://blog.carlossanchez.eu/tag/devops
Video at http://vimeo.com/49483627

Puppet is an infrastructure-as-code tool that allows easy and automated provisioning of servers, defining the packages, configuration, services,... in code. Enabling DevOps culture, tools like Puppet help drive Agile development all the way to operations and systems administration, and along with continuous integration tools like Jenkins, it is a key piece to accomplish repeatability and continuous delivery, automating the operations side during development, QA or production, and enabling testing of systems configuration.
Traditionally a field for system administrators, Puppet can empower developers, allowing both to collaborate coding the infrastructure needed for their developments, whether it runs in hardware, virtual machines or cloud. Developers and sysadmins can define what JDK version must be installed, application server, version, configuration files, war and jar files,... and easily make changes that propagate across all nodes.
Using Vagrant, a command line automation layer for VirtualBox, they can also spin off virtual machines in their local box, easily from scratch with the same configuration as production servers, do development or testing and tear them down afterwards.
We’ll show how to install and manage Puppet nodes with JDK, multiple application server instances with installed web applications, database, configuration files and all the supporting services. Including getting up and running with Vagrant and VirtualBox for quickstart and Puppet experiments, as well as setting up automated testing of the Puppet code.

Published in: Technology
0 Comments
37 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
27,130
On Slideshare
0
From Embeds
0
Number of Embeds
22
Actions
Shares
0
Downloads
0
Comments
0
Likes
37
Embeds 0
No embeds

No notes for slide
  • \n
  • question? #developers vs #ops vs #devops\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • question? what do you do to specify ops requirements?\n
  • \n
  • question? what do you do to specify ops requirements?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • question? who&amp;#x2019;s able to roll back to a previous version of software? and system?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Puppet for Java developers - JavaZone NO 2012

    1. 1. Puppet for JavadevelopersCarlos Sanchez@csanchezhttp://carlossanchez.euhttp://maestrodev.com
    2. 2. Apache @csanchez Maven Apache ASF Archiva Member ApacheEclipse IAM Continuum blog.carlossanchez.eu
    3. 3. DevOps
    4. 4. DevQaOps ?
    5. 5. Agile planning iterative development continuous integrationrelease soon, release often
    6. 6. DevOps addresses Fear of change Risky deployments It works on my machine! SiloisationDev Change vs. Ops stability
    7. 7. Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
    8. 8. DevOps is NOT about the tools
    9. 9. Nice, BUThow can I implement IT
    10. 10. Nice, BUThow can I implement IT
    11. 11. Tools can enablechange in behavior and eventually change culture Patrick Debois
    12. 12. DevOps toolseveryone is intelligentenoughevery tool is cloudenabledevery tool is DevOps(tm)
    13. 13. DevOps toolseveryone is intelligentenoughevery tool is cloudenabledevery tool is DevOps(tm)
    14. 14. DevOps tools: infrastructure automation infrastructure as code it’s all invented, now it’s standardized
    15. 15. Infrastructure as CodeNew solutions bring new challengesFollow development best practices tagging branching releasing dev, QA, production
    16. 16. New solutions bring new problems DEVOPS “MY MACHINES ARE BEING PROVISIONED”
    17. 17. Puppet exec { "maven-untar":manifests command => "tar xf /tmp/x.tgz", cwd => "/opt", creates => "/opt/apache-maven-${version}",ruby-like path => ["/bin"], } -> ERB templates file { "/usr/bin/mvn": ensure => link, target => "/opt/apache-maven-${version}/bin/mvn", } file { "/usr/local/bin/mvn": ensure => absent, require => Exec["maven-untar"], } file { "$home/.mavenrc": mode => "0600", owner => $user, content => template("maven/mavenrc.erb"), require => User[$user], }
    18. 18. Puppetinfrastructure IS code package { openssh-server: ensure => present, }
    19. 19. Puppetdeclarative modelstate vs process no scripting service { ntp: name => ntpd, ensure => running, }
    20. 20. master - agentarchitecture
    21. 21. Vagrant
    22. 22. Vagrant Oracle VirtualBox cmdline automation Easy Puppet and Chef provisioning Keep VM configuration for different projectsShare boxes and configuration files across teams base box + configuration files http://vagrantup.com/
    23. 23. Vagrant base boxes www.vagrantbox.esanywhere! just (big) files
    24. 24. using Vagrant$ vagrant box add centos-6.0-x86_64 http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box$ vagrant init myproject$ vagrant up$ vagrant ssh$ vagrant suspend$ vagrant resume$ vagrant destroy
    25. 25. VagrantVagrant::Config.run do |config| # Every Vagrant virtual environment requires a box to build off of. config.vm.box = "centos-6.0-x86_64" # The url from where the config.vm.box box will be fetched config.vm.box_url = "http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box" # Boot with a GUI so you can see the screen. (Default is headless) #config.vm.boot_mode = :gui # Assign this VM to a host only network IP, allowing you to access it viathe IP. # config.vm.network "33.33.33.10" # Forward a port from the guest to the host, which allows for outside # computers to access the VM, whereas host only networking does not. config.vm.forward_port "sonar", 9000, 19000 config.vm.provision :puppet do |puppet| puppet.manifest_file = "site.pp" puppet.module_path = "mymodules" endend
    26. 26. manifests/base.pppackage { jdk: ensure => installed, name => $operatingsystem ? { centOS => "java-1.6.0-openjdk-devel", Ubuntu => "openjdk-6-jdk", default => "jdk", },}
    27. 27. VeeWee
    28. 28. VeeWee easily build Vagrant base boxeshttps://github.com/jedi4ever/veewee
    29. 29. using VeeWee$ gem install veewee$ vagrant basebox templates$ vagrant basebox define my-ubuntu-server ubuntu-11.04-server-amd64# customize definitions/my-ubuntu-server$ vagrant basebox build my-ubuntu-server$ vagrant basebox validate my-ubuntu-server$ vagrant basebox export my-ubuntu-server$ vagrant box add my-ubuntu-server my-ubuntu-server.box$ vagrant init my-ubuntu-server
    30. 30. Editors
    31. 31. Geppetto
    32. 32. http://cloudsmith.github.com/geppetto
    33. 33. TextMate
    34. 34. https://github.com/masterzen/puppet-textmate-bundle
    35. 35. Puppet DSL
    36. 36. Puppet resourcesuser { dave: ensure => present, uid => 507, gid => admin, shell => /bin/zsh, home => /home/dave, managehome => true,}
    37. 37. Puppet resourcesfile {testfile: path => /tmp/testfile, ensure => present, mode => 0640, content => "Im a test file.",}notify {"Im notifying you.":}
    38. 38. Puppet standalone$ puppet apply my_test_manifest.pp
    39. 39. orderingfile {/tmp/test1: ensure => present, content => "Hi.",}notify {/tmp/test1 has alreadybeen synced.: require => File[/tmp/test1],}
    40. 40. orderingfile {/tmp/test1: ensure => present, content => "Hi.",} ->notify {/tmp/test1 has alreadybeen synced.:}
    41. 41. package / file /service / subscribepackage { openssh-server: ensure => present, before => File[/etc/ssh/sshd_config],}file { /etc/ssh/sshd_config: ensure => file, mode => 600, source => /root/learning-manifests/sshd_config,}service { sshd: ensure => running, enable => true, hasrestart => true, hasstatus => true, subscribe => File[/etc/ssh/sshd_config],}
    42. 42. variables$longthing = "Imagine I have something reallylong in here. Like an SSH key, lets say."file {authorized_keys: path => /root/.ssh/authorized_keys, content => $longthing,}
    43. 43. factshost {self: ensure => present, name => $::fqdn, host_aliases => [puppet, $::hostname], ip => $::ipaddress,}file {motd: ensure => file, path => /etc/motd, mode => 0644, content => "Welcome to ${::hostname},na ${::operatingsystem} island in the sea of ${::domain}.n",}
    44. 44. conditionalsif $is_virtual { service {ntpd: ensure => stopped, enable => false, }}else { service { ntpd: name => ntpd, ensure => running, enable => true, hasrestart => true, require => Package[ntp], }}
    45. 45. conditionalscase $operatingsystem { centos, redhat: { $apache = "httpd" } debian, ubuntu: { $apache = "apache2" } default: { fail("Unrecognized operating system forwebserver") }}$apache = $operatingsystem ? { centos => httpd, redhat => httpd, /(?i)(ubuntu|debian)/ => "apache2-$1", # (Dont actually use that package name.) default => undef, }
    46. 46. class definitionclass ntp { package { ntp: ensure => installed, } service { ntp: name => ntpd, ensure => running, enable => true, subscribe => File[ntp.conf], }}
    47. 47. class declarationclass {myclass: param1 => a, param2 => b,}
    48. 48. class declaration (2)include ntp
    49. 49. parameterized classesclass paramclassexample ($value1, $value2 = "Defaultvalue") { notify {"Value 1 is ${value1}.":} notify {"Value 2 is ${value2}.":}}class {paramclassexample: value1 => Something, value2 => Something else,}class {paramclassexample: value1 => Something,}
    50. 50. definitionsdefine myfile ($path) { file {$path: content => Something}}myfile {a: path => /tmp/a,}myfile {b: path => /tmp/b,}
    51. 51. templatingfile {/etc/foo.conf: ensure => file, require => Package[foo], content => template(foo/foo.conf.erb),}
    52. 52. templates/foo.conf.erbOS is <%= $::operatingsystem %>
    53. 53. node configuration# nodes.ppnode someserver.domain.com inherits basenode { $web_fqdn = www.domain.com include genericwebserver include some_other_service}node ldapmaster.domain.com inherits basenode { include s_ldap::master}node humanresources.domain.com inherits basenode { include c_humanresources}
    54. 54. Puppet modules
    55. 55. module tool$ puppet module generate csanchez-mymodule
    56. 56. modules{module}/ files/ lib/ manifests/ init.pp {class}.pp {defined type}.pp {namespace}/ {class}.pp {class}.pp templates/ spec/ tests/
    57. 57. Modulefilename csanchez-mymoduleversion 0.0.1source UNKNOWNauthor csanchezlicense Apache License,Version 2.0summary UNKNOWNdescription UNKNOWNproject_page UNKNOWN## Add dependencies, if any:# dependency username/name, >= 1.2.0
    58. 58. building a module$ puppet module build
    59. 59. Puppetlabs Forgehttp://forge.puppetlabs.com A library of modules
    60. 60. librarian-puppethttp://librarian-puppet.com/$ librarian-puppet init# edit Puppetfile$ librarian-puppet install
    61. 61. Puppetfileforge "http://forge.puppetlabs.com"# mod puppetlabs/stdlib# mod ntp,# :git => git://github.com/puppetlabs/puppetlabs-ntp.git# mod apt,# :git => https://github.com/puppetlabs/puppetlabs-apt.git,# :ref => feature/master/dans_refactor
    62. 62. Maven and Puppet
    63. 63. What am I doing to automate deployment Ant tasks plugin ssh commands Assembly plugin Cargo Capistrano
    64. 64. What can I do to automate deployment Handle full deployment including infrastructure not just webapp deployment Help Ops with clear, automated manifests Ability to reproduce production environments in local box using Vagrant / VirtualBox / VMWare Use the right tool for the right job
    65. 65. Maven-Puppet module A Maven Puppet modulehttps://github.com/maestrodev/puppet-maven fetches Maven artifacts from the repo manages them with Puppet no more extra packaging
    66. 66. Installing Maven$repo1 = { id => "myrepo", username => "myuser", password => "mypassword", url => "http://repo.acme.com",}# Install Mavenclass { "maven::maven": version => "2.2.1",} -># Create a settings.xml with the repo credentialsclass { "maven::settings" : servers => [$repo1],}
    67. 67. New Maven typemaven { "/tmp/maven-core-2.2.1.jar": id => "org.apache.maven:maven-core:jar:2.2.1", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"],}
    68. 68. New Maven typemaven { "/tmp/maven-core-2.2.1.jar": groupId => "org.apache.maven", artifactId => "maven-core", version => "2.2.1", packaging => "jar", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"],}
    69. 69. Examples
    70. 70. Tomcat cluster + postgres postgres database db.acme.com tomcat servers tomcat1.acme.com tomcat2.acme.com ... web server www.acme.com
    71. 71. Modules required puppetlabs/stdlib puppetlabs/java puppetlabs/apache inkling/postgresqlcamptocamp/tomcat maestrodev/maven maestrodev/wget stahnma/epel
    72. 72. Puppetfileforge http://forge.puppetlabs.commod puppetlabs/stdlib, 3.0.1mod puppetlabs/java, 0.1.6mod puppetlabs/apache, 0.2.2mod inkling/postgresqlmod camptocamp/tomcat, :git => git://github.com/camptocamp/puppet-tomcat.git, :ref => 16e498mod maestrodev/maven, ‘0.0.1’mod maestrodev/wget, 0.0.1mod stahnma/epel, 0.0.2
    73. 73. inkling - postgresclass { postgresql::server: config_hash => { ip_mask_allow_all_users => 192.168.0.0/0, listen_addresses => *, manage_redhat_firewall => true, postgres_password => postgres, },}postgresql::db { appfuse: user => appfuse, password => appfuse, grant => all,}
    74. 74. camptocamp - tomcattomcat::instance { appfuse: ensure => present, http_port => 8080,}$webapp = /srv/tomcat/appfuse/webapps/ROOT
    75. 75. maestrodev - mavenmaven { "${webapp}.war": id => org.appfuse:appfuse-spring:2.1.0:war, require => File[/srv/tomcat/appfuse/webapps], notify => Service[tomcat-centrepoint],}
    76. 76. Puppet Rspec require spec_helperdescribe db.acme.com do let(:facts) { { :osfamily => RedHat, :operatingsystem => CentOS, :operatingsystemrelease => 6.3} } it { should_not contain_class(java) } it { should contain_class(postgresql::server)}end
    77. 77. Example code and slides Available at http://slideshare.carlossanchez.eu http://github.carlossanchez.euhttps://github.com/carlossg/puppet-for-java-devs http://blog.carlossanchez.euhttps://github.com/maestrodev/puppet-modules
    78. 78. Takk!http://blog.carlossanchez.euhttp://maestrodev.comcsanchez@maestrodev.comcarlos@apache.org@csanchez
    79. 79. Photo Credits Son of Man Lego - Alex Eylarhttp://www.flickr.com/photos/hoyvinmayvin/4702772452/ Brick wall - Luis Argerich http://www.flickr.com/photos/lrargerich/4353397797/ Agile vs. Iterative flow - Christopher Littlehttp://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg DevOps - Rajiv.Pant http://en.wikipedia.org/wiki/File:Devops.png Pimientos de Padron - Howard Walfish http://www.flickr.com/photos/h-bomb/4868400647/ Compiling - XKCD http://xkcd.com/303/ Printer in 1568 - Meggs, Philip B http://en.wikipedia.org/wiki/File:Printer_in_1568-ce.png Relativity - M. C. Escherhttp://en.wikipedia.org/wiki/File:Escher%27s_Relativity.jpg Teacher and class - Herald Post http://www.flickr.com/photos/heraldpost/5169295832/

    ×