Main areas of expertise
DevOps
Version management
 GIT based on local GIT server
 GITHub (public repositories, private repositories, team working)
 Bitbucket (public repositories, private repositories, team working)
 CVS based on local CVS server
 SVN based on local SVN server
Backgroundknow-how
Build automation tools
 Apache ANT
 Maven
 Dependency management, different repositories and plugins
 Gradle – used just within a research phase
Development tools
 Commonly IDEs used: Eclipse, Netbeans
 Continuous Integration
 Jenkins CI along with custom plugins implementation
 Travis for GitHub
 Bug tracking and documentation tools:
 Jira (having Confluence integrated)
 Mantis (customized installation, DokuWiki integrated)
Backgroundknow-how
Web Servers and Application Servers
 Apache web server
 Apache Tomcat web server
 Jetty web server
 JBoss application server
 Glassfish application server
 Nginx web server
Relational databases
 Postgres database
 Experience custom installations, configuration and backup
procedures
 Mysql database
 Used within several Java based projects
 Experience with Percona integration
 Sybase database
 Widely used within one warehouse system implementation
(java based)
 IBM DB2 database
 Widely used within one warehouse system implementation
(java and C/C++ based)
 SQLite database
 Experienced within quick demo projects
Programmingknow-how
Web services
 Apache Axis 1.x and 2.0 as SOAP protocol implementation
 Apache CXF (JAX-WS)
 Apache JAX- RX – Restful web services implementation
Web based front-end development
 Apache Struts 1.x and 2.x framework
 Django framework
 JSF based frameworks (Primefaces and IFaces)
 Zkoss framework
 Java script related: Node.js, Io.js, Angular.js, Ember.js, Backbone.js
 Spring framework
Programmingknow-how
Core Java technologies
 Java networking – TCP/IP, UDP, advanced client server protocols
implemented
 Java Advanced Imaging, openCV, advanced image processing
 Java Contexts and Dependency Injection
 JDBC under Postgres, IBM DB2 and sysbase
 Hibernate persistence layer
 Data nucleus persistence layer
 EJB 2.x and EJB 3.x
 Spring framework
 JBoss JBPM & Kie Workbench
Programmingknow-how
Automation on testing web applications
 Selenium IDE - used for regress testing on Web Applications
 It is a complete integrated development environment (IDE) for
Selenium tests.
 It is implemented as a Firefox Add-On, and allows recording,
editing, and debugging tests.
 We used in different web based projects (like online shopping)
 Selenium WebDrive - used for regress testing on WebApplications
 Selenium provides a test domain-specific language (Selenese)
to write tests in a number of popular programming languages,
including Java.
 CI Jenkins
 Integrate automated testing with Jenkins CI
Qualityassuranceknow-how
Qualityassurance
know-how Automation for e-commerce testing
 Cross browsers testing
 Firefox.
 Chrome
 Opera
 Internet Explorer
 Main software involved in this automation
 Junit Test
 Junit Suite
 Selenium
 Web Driver
 XPath
 JSoup
 Log4j
 Ant builder
Qualityassurance
know-how Configurable ecommerce workflows
 User registration
 Simple user navigation on different pages (dynamically created)
 Order workflow - the most important workflow
 Administration area usage
 Notifications workflows
Qualityassurance
know-how Usage of Continuous integration for this automation
 Checkout entire source code of the automation process
 Checkout entire source code of a e-commerce version
 Jenkins based deployment of e-commerce
 Jenkins Master starts Jenkins slaves
 Each slave could control a set of browsers, a set of e-commerce workfl
ows
Advanced usage of Apache Ofbiz framework
 Java based framework
 Its own persistence layer
 Its own transaction manager (Geronimo)
 Integration of Bean shell scripting and groovy
 Customized web front-end templates – Free-marker integration
 Its own Content management system
 Handling over 50.000 orders in 24 hours, by a farm of 20 servers
 Integration of Opscode Chef for the automation of servers
deployment
 Investigation of security issues within an Ofbiz based
e-commerce platform (PCI-DSS 2.0 compliance)
E-commerceknow-how
Advanced usage of Magento framework
 Php based framework
 Extended usage of configuration management for high scalability
 Auto scaling capabilities based on Opscode Chef and Amazon EC2
E-commerceknow-how
Voltnet Software Brasov
Full-text search engine
Large experience in a customized search engine, based on Apache Lucene and SolR
 Apache Lucene
 Indexing large amounts of data (over 4 GB)
 Fast queries against the index
 Used as non-sql database
 Used for special features not available inside Solr
 Apache Solr
 Remote queries to the index
 User friendly interface
 Faster development, easier queries
 Extended usage of configuration management for high scalability
 Automated load balancing based on Opscode Chef and Amazon EC2
Full-text search engine
Large experience in a customized search engine, based on Apache Lucene and SolR
 SolrCloud
 Lucene indexes spread on different machines
 Used as transport layer between different LANs
 Data exchange and data synchronization between different
relational databases
 ZooKeeper
 Manager for SolrCloud instances
 RestEasy
 Communication to Solr from an external application using REST
services
 Elastic search
 Search engine based on Apache Lucene
No-sql databases
 Experience with MongoDB, CouchDB, Redis
 Hadoop/HBase
Openfornewtechnologies
Monitoring tools
 Nagios & Icinga
 IT infrastructure monitoring, custom plugins, chef based
automations for Nagios installations
 Munin
 Networked resource monitoring, custom plugins, chef based
automations for Munin installations
 Zabbix
 Enterprise monitoring platform, chef based automations for
Zabbix installations
Centralize logs, processing large volumes of logs
 ElasticSearch & Logstash & Kibana
 ElasticSearch – non-sql solution of logs storage
 Logstash - receiving, processing and outputting logs
 Kibana - analytics and visualization platform
 Splunk
 captures, indexes and correlates real-time data, graphs,
reports, alerts, dashboards and visualizations
Openfornewtechnologies
Pioneers
 Handling virtual machines within:
 Open VMWare
 VirtualBox
 VMWare workstation
 iESX
Virtualization
Tiny solutions
 Vagrant - offers different work environments
 Machines provisioned on top of VirtualBox, VMWare, AWS as
providers
 Provisioning tools: shell scripts, Opscode Chef
 Docker
 Build, ship and run distributed applications
 Docker and Chef working together
 Docker and Jenkins working together
Amazon Web Services - AWS
 Amazon EC2
 Extended usage of the core EC2 service of AWS
 Storage and content delivery
 Amazon S3 – store and retrieve large volumes of data
 Amazon Glacier – low-cost solution for archiving and backup
solutions
 Private isolated cloud
 AWS VPC
 DNS service
 AWS Route 53
Virtualization
Amazon Web Services - AWS
Virtualization
 Resources and applications monitoring
 AWS CloudWatch
 Template based resources handling
 CloudFormation extended usage
 Good experience with troposphere - library that allows easy
creation of AWS Cloudformation files
 Amazon Spot Instances and Fleet API
 Basic experience with Fleet API to manage large number of spot
instances
 Amazon SQS
 Basic usage of SQS queues
 AWS marketplace
 Basic experience creating customized AMIs
Configuration management
Opscode Chef
 Usage of standard Opscode cookbooks
 “Inheritance” of standard cookbooks
 “Role” cookbooks
 Usage of berkshelf for more efficient cookbooks management
 Own installation of a Chef server
 Customized LWRP written in Ruby
 Usage of data bags and encrypted data bags
 Testing and simulating using chef-zero
Processing of big data
Bigdatatopics
 Apache Spark
 Cookbooks for installing and configuring master-slave Apache Spark.
 Additional cookbooks for real demos with Apache Spark:
 Real time log processing – redirect apache access logs to Spark
streaming for real time big data processing; the concept could be used within ap
plication security regarding fast identification of specific
tags/items within access log files.
 Real time Twitter stream processing – filter Twitter streams for specific
topics
 Instagram image processing – complex system getting Instagram
pictures from a predefined set of geo locations, Apache Spark used to
process the pictures based on image recognition pieces of software
Continuous education
Training materials
 A large set of training materials available – teaching in English language
 Introduction to Ruby programming
 Advanced Ruby programming
 Amazon EC2 training module
 Amazon IAM training module
 Amazon CloudFormation training module
 Amazon Route53 training module
 Introduction to Opscode Chef
 Advanced Opscode Chef training module
 Amazon EBT training module
 Amazon RDS training module
 Advanced Docker containers usage
“increase security” – introduce ssl for an existing web stack
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Changes to run postgres DB under ssl certificates
 Changes to run nginx web server under ssl certificates
 Ensure nginx to postgres communication through ssl
 Installing ssl certificate to an AWS Load Balancer (using boto)
 Ensure LB to nginx communication through ssl
 Project duration: 30 man hours
“monitoring capabilities” for a web stack
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Cookbooks for client side and server side of Munin framework
 Cookbooks for client side and server side of Nagios & Icinga
framework
 Cookbooks for client side and server side of Splunk framework
 Project duration: 320 man hours
“splunk inside AWS” – extended usage of splunk
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Research of how Splunk manages its data
 Check if Splunk starts to delete data after a while when disk
space becomes to small
 Plan is to have for one year all access log info in Splunk
 Data will be kept forever (until it is deleted) in S3.
 Investigate how data can exported out of Splunk into S3
(lets say on a daily basis).
 Project duration: 60 man hours
“advanced postgres usage”
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Adjust the standard postgres cookbooks to support Percona
wrapper
 A proxy server which can act in front of a Postgres Master and n slaves.
(like similar solutions of Mysql - MariaDB Maxscale and
MySQL Proxy)
 The proxy should send all insert/update/delete statements to the mast
er and the select statements to the slaves. Everything should be finally
based on cookbooks.
 Project duration: 220 man hours
“auto scaling magento” implement auto scale for a
production magento stack
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Adjustments to magentostack cookbook
 Automatic load of large demo data
 Load testing plan to be able to test auto scaling feature
 Implementation and testing of auto scaling feature based on
Amazon CloudWatch metrics
 Project duration: 40 man hours
“auto scaling wordpress”
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Adjustments to wordpress related cookbook
 Load testing plan to be able to test auto scaling feature
 Implementation and testing of auto scaling feature without
CloudWatch metrics
 Project duration: 20 man hours
“postgres disaster recovery” disaster recovery based on
volume snapshots
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 postgres data directory is defined on a separate volume
 a snapshot is made at regular time intervals for that volume.
 In case of a crash, another machine is started and the data
volume is created from latest snapshot at start time.
 Project duration: 60 man hours
“AWS CloudTrail integration with Splunk”
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Cookbooks for Splunk Enterprise installation
 Cookbook for Splunk App plugin installation and configuration
 Configure Cloudtrail logging to send SNS topic that should point
to Amazon SQS queue
 Testing the integration regarding different EC2 API calls
 Project duration: 40 man hours
“PCI DSS Compliance” Implementation of PCI security
standards (PCI DSS version 1.2)
Samplesofprojects
 Virtualization environment:
Amazon EC2
 Configuration management:
Opscode Chef
 Requirements overview:
Tasks: PCI Reference:
Migration to an operating system with current security
patches (Ubuntu 10.04).
PCI-DSS requires installing critical security patches within
one month of release.
PCI-DSS 6.1
Installation of a minimal operating system.
De-installation of not required services and drivers.
PCI-DSS: 2.2.2, 2.2.4
Installation and configuration of an access control system l
ike Radius or OpenLDAP for all EC2 instances.
PCI-DSS 7.1.4, 7.2, 8.3
Configuration of an access control system:
Only named account usage. No root account. No account
sharing.
PCI-DSS: 8.1, 8.5
Configuration of Amazon Identity and Access
Management (IAM) rules.
Limit access to EC2 security groups (PCI-DSS terminology:
firewall) and EBS snapshots. Separation of EC2 access
keys.
PCI-DSS 1.1.1, 6.3.2
Review and re-configuration of EC2 security groups. PCI-DSS 1.2, 1.3, 6.3.2
Configuration of host based firewall with iptables. PCI-DSS 1.2.1, 1.3.5
Installation of anti-virus software (like F-PROT) on all
EC2 instances.
PCI-DSS 5.1
Review of MySQL security settings and reconfiguration.
Preferably: Security configuration with Radius/LDAP server
.
PCI-DSS: 8.5.16
Installation and configuration of an audit logging system. PCI-DSS 10.2, 10.3, 10.5, 10.7
Installation and configuration of a reliable time service. PCI-DSS 10.4
Installation and configuration of an intrusion detection
system like Tripwire or AIDE.
PCI-DSS: 10.5.5, 11.5
Installation and configuration of an intrusion prevention
system like Snort.
PCI-DSS 11.4
Internal vulnerability scans with Nessus. PCI-DSS 11.2, 11.3
Installation and configuration of an application firewall like
AppArmor/SELinux for all web-server EC2 instances.
PCI-DSS 6.6
Review VPN configuration to ensure a two-factor
authentication.
PCI-DSS 8.3
“PCI DSS Compliance” Implementation of PCI security
standards (PCI DSS version 1.2)
Samplesofprojects
 Project details:
 Customizing the iptables cookbook to automatically
set up rules based on machines' roles
 Cookbook to remove unneeded packages from a
freshly installed machine (part of PCI standardization)
 Cookbook for vtun setup
 Cookbook for splunk setup
 Cookbook for snort, openlda, openssh
 Cookbook for Ossec setup
 Cookbook that would set up a daily ubuntu dist
repository
 Cookbook that would backup rotated apache logs
both on s3 and glacier
 Cookbook to create local users using databags
 Implementation of different testing scenario
 Project duration: 120 man hours
“improve reports of full-text search engine” search-engine
reporting
Samplesofprojects
 Virtualization environment: local servers, Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Cookbooks installing virtualbox, vagrant, test-kitchen and Jenkins on
Ubuntu 12.04 under VMWare
 Usage of vagrant as underlying virtualization plugin for test-kitchen
 Usage of test-kitchen as integration testing tool (check if a node
converges)
 Vagrant could run either VirtualBox or VMWare or directly EC2 machines
 Vagrant could provision either with Chef Solo or with a real Chef Server
 Assertions using minitest and vagrant
 ChefSpec based assertions for cookbook recipies
 Jenkins jobs to start test-kitchen and vagrant – configured through cookbooks
 Project duration: 160 man hours
“enlarge processing power” export scanned documents into tex
t having large but cheap processing power
Samplesofprojects
 Virtualization environment: Amazon EC2
 Configuration management: Opscode Chef
 Project details:
 Cookbooks installing customized OCR software
 Usage of Amazon Spot Instances and FleetAPI
 Usage of Amazon SQS
 Project duration: 120 man hours
“quality assurance in cookbooks world” different testing
procedures for Chef configuration management
Samplesofprojects
 Virtualization environment: local servers,
 Configuration management: Opscode Chef
 Project details:
 Cookbooks for elastic search installation and configuration
 Cookbooks for logstash installation and configuration
 Cookbooks for kibana installation and configuration
 Shells scripts for a primitive solution of self learning search system
 Project duration: 16 man hours
“Jenkins runs Docker” Docker based containers, managed
by Jenkins, for testing purposes
Samplesofprojects
 Virtualization environment: local servers,
 Docker account
 Project details:
 Even Jenkins runs under a Docker container so in the end we
have Docker inside Docker
 Docker file to install Jenkins in a Docker container
 Shell scripting called by Jenkins jobs to start Docker containers
 Integration of magento web stack for testing under
Jenkins & Docker
 Project duration: 60 man hours
Real live demo to prove the reality behind our experience
 Our demo consists in doing a complete Magento installation on an EC2
machine inside AWS
 The machine will be launched inside an autoscaling group
 The autoscaling group has an Elastic Load Balancer in front
 We define scaling policies inside the autoscaling group, based on the
average CPU load, that will start up new Magento machines when load is too
high or will stop some of the existing machines when load is under defined limits
 To test the scaling policies we use Apache Benchmark (ab) utility to create
load on the existing machines
Reallivedemotoprove
thereality
behindourexperience
Reallivedemotoprove
thereality
behindourexperience
 Using the Apache Bench tool (ab) we perform load tests by sending a large num
ber of concurrent requests
 Based on the configuration below, a new instance was started and an
alarm email was sent, notifying about the started instance
Reallivedemotoprove
thereality
behindourexperience
 Here is the notification email which confirms the custom scaling feature imple
mented.
Looking forward for a long term cooperation
Thank you for your
attention and interest !

devops_

  • 1.
    Main areas ofexpertise DevOps
  • 2.
    Version management  GITbased on local GIT server  GITHub (public repositories, private repositories, team working)  Bitbucket (public repositories, private repositories, team working)  CVS based on local CVS server  SVN based on local SVN server Backgroundknow-how Build automation tools  Apache ANT  Maven  Dependency management, different repositories and plugins  Gradle – used just within a research phase
  • 3.
    Development tools  CommonlyIDEs used: Eclipse, Netbeans  Continuous Integration  Jenkins CI along with custom plugins implementation  Travis for GitHub  Bug tracking and documentation tools:  Jira (having Confluence integrated)  Mantis (customized installation, DokuWiki integrated) Backgroundknow-how Web Servers and Application Servers  Apache web server  Apache Tomcat web server  Jetty web server  JBoss application server  Glassfish application server  Nginx web server
  • 4.
    Relational databases  Postgresdatabase  Experience custom installations, configuration and backup procedures  Mysql database  Used within several Java based projects  Experience with Percona integration  Sybase database  Widely used within one warehouse system implementation (java based)  IBM DB2 database  Widely used within one warehouse system implementation (java and C/C++ based)  SQLite database  Experienced within quick demo projects Programmingknow-how
  • 5.
    Web services  ApacheAxis 1.x and 2.0 as SOAP protocol implementation  Apache CXF (JAX-WS)  Apache JAX- RX – Restful web services implementation Web based front-end development  Apache Struts 1.x and 2.x framework  Django framework  JSF based frameworks (Primefaces and IFaces)  Zkoss framework  Java script related: Node.js, Io.js, Angular.js, Ember.js, Backbone.js  Spring framework Programmingknow-how
  • 6.
    Core Java technologies Java networking – TCP/IP, UDP, advanced client server protocols implemented  Java Advanced Imaging, openCV, advanced image processing  Java Contexts and Dependency Injection  JDBC under Postgres, IBM DB2 and sysbase  Hibernate persistence layer  Data nucleus persistence layer  EJB 2.x and EJB 3.x  Spring framework  JBoss JBPM & Kie Workbench Programmingknow-how
  • 7.
    Automation on testingweb applications  Selenium IDE - used for regress testing on Web Applications  It is a complete integrated development environment (IDE) for Selenium tests.  It is implemented as a Firefox Add-On, and allows recording, editing, and debugging tests.  We used in different web based projects (like online shopping)  Selenium WebDrive - used for regress testing on WebApplications  Selenium provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages, including Java.  CI Jenkins  Integrate automated testing with Jenkins CI Qualityassuranceknow-how
  • 8.
    Qualityassurance know-how Automation fore-commerce testing  Cross browsers testing  Firefox.  Chrome  Opera  Internet Explorer  Main software involved in this automation  Junit Test  Junit Suite  Selenium  Web Driver  XPath  JSoup  Log4j  Ant builder
  • 9.
    Qualityassurance know-how Configurable ecommerceworkflows  User registration  Simple user navigation on different pages (dynamically created)  Order workflow - the most important workflow  Administration area usage  Notifications workflows
  • 10.
    Qualityassurance know-how Usage ofContinuous integration for this automation  Checkout entire source code of the automation process  Checkout entire source code of a e-commerce version  Jenkins based deployment of e-commerce  Jenkins Master starts Jenkins slaves  Each slave could control a set of browsers, a set of e-commerce workfl ows
  • 11.
    Advanced usage ofApache Ofbiz framework  Java based framework  Its own persistence layer  Its own transaction manager (Geronimo)  Integration of Bean shell scripting and groovy  Customized web front-end templates – Free-marker integration  Its own Content management system  Handling over 50.000 orders in 24 hours, by a farm of 20 servers  Integration of Opscode Chef for the automation of servers deployment  Investigation of security issues within an Ofbiz based e-commerce platform (PCI-DSS 2.0 compliance) E-commerceknow-how
  • 12.
    Advanced usage ofMagento framework  Php based framework  Extended usage of configuration management for high scalability  Auto scaling capabilities based on Opscode Chef and Amazon EC2 E-commerceknow-how
  • 13.
    Voltnet Software Brasov Full-textsearch engine Large experience in a customized search engine, based on Apache Lucene and SolR  Apache Lucene  Indexing large amounts of data (over 4 GB)  Fast queries against the index  Used as non-sql database  Used for special features not available inside Solr  Apache Solr  Remote queries to the index  User friendly interface  Faster development, easier queries  Extended usage of configuration management for high scalability  Automated load balancing based on Opscode Chef and Amazon EC2
  • 14.
    Full-text search engine Largeexperience in a customized search engine, based on Apache Lucene and SolR  SolrCloud  Lucene indexes spread on different machines  Used as transport layer between different LANs  Data exchange and data synchronization between different relational databases  ZooKeeper  Manager for SolrCloud instances  RestEasy  Communication to Solr from an external application using REST services  Elastic search  Search engine based on Apache Lucene
  • 15.
    No-sql databases  Experiencewith MongoDB, CouchDB, Redis  Hadoop/HBase Openfornewtechnologies Monitoring tools  Nagios & Icinga  IT infrastructure monitoring, custom plugins, chef based automations for Nagios installations  Munin  Networked resource monitoring, custom plugins, chef based automations for Munin installations  Zabbix  Enterprise monitoring platform, chef based automations for Zabbix installations
  • 16.
    Centralize logs, processinglarge volumes of logs  ElasticSearch & Logstash & Kibana  ElasticSearch – non-sql solution of logs storage  Logstash - receiving, processing and outputting logs  Kibana - analytics and visualization platform  Splunk  captures, indexes and correlates real-time data, graphs, reports, alerts, dashboards and visualizations Openfornewtechnologies
  • 17.
    Pioneers  Handling virtualmachines within:  Open VMWare  VirtualBox  VMWare workstation  iESX Virtualization Tiny solutions  Vagrant - offers different work environments  Machines provisioned on top of VirtualBox, VMWare, AWS as providers  Provisioning tools: shell scripts, Opscode Chef  Docker  Build, ship and run distributed applications  Docker and Chef working together  Docker and Jenkins working together
  • 18.
    Amazon Web Services- AWS  Amazon EC2  Extended usage of the core EC2 service of AWS  Storage and content delivery  Amazon S3 – store and retrieve large volumes of data  Amazon Glacier – low-cost solution for archiving and backup solutions  Private isolated cloud  AWS VPC  DNS service  AWS Route 53 Virtualization
  • 19.
    Amazon Web Services- AWS Virtualization  Resources and applications monitoring  AWS CloudWatch  Template based resources handling  CloudFormation extended usage  Good experience with troposphere - library that allows easy creation of AWS Cloudformation files  Amazon Spot Instances and Fleet API  Basic experience with Fleet API to manage large number of spot instances  Amazon SQS  Basic usage of SQS queues  AWS marketplace  Basic experience creating customized AMIs
  • 20.
    Configuration management Opscode Chef Usage of standard Opscode cookbooks  “Inheritance” of standard cookbooks  “Role” cookbooks  Usage of berkshelf for more efficient cookbooks management  Own installation of a Chef server  Customized LWRP written in Ruby  Usage of data bags and encrypted data bags  Testing and simulating using chef-zero
  • 21.
    Processing of bigdata Bigdatatopics  Apache Spark  Cookbooks for installing and configuring master-slave Apache Spark.  Additional cookbooks for real demos with Apache Spark:  Real time log processing – redirect apache access logs to Spark streaming for real time big data processing; the concept could be used within ap plication security regarding fast identification of specific tags/items within access log files.  Real time Twitter stream processing – filter Twitter streams for specific topics  Instagram image processing – complex system getting Instagram pictures from a predefined set of geo locations, Apache Spark used to process the pictures based on image recognition pieces of software
  • 22.
    Continuous education Training materials A large set of training materials available – teaching in English language  Introduction to Ruby programming  Advanced Ruby programming  Amazon EC2 training module  Amazon IAM training module  Amazon CloudFormation training module  Amazon Route53 training module  Introduction to Opscode Chef  Advanced Opscode Chef training module  Amazon EBT training module  Amazon RDS training module  Advanced Docker containers usage
  • 23.
    “increase security” –introduce ssl for an existing web stack Samplesofprojects  Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  Changes to run postgres DB under ssl certificates  Changes to run nginx web server under ssl certificates  Ensure nginx to postgres communication through ssl  Installing ssl certificate to an AWS Load Balancer (using boto)  Ensure LB to nginx communication through ssl  Project duration: 30 man hours
  • 24.
    “monitoring capabilities” fora web stack Samplesofprojects  Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  Cookbooks for client side and server side of Munin framework  Cookbooks for client side and server side of Nagios & Icinga framework  Cookbooks for client side and server side of Splunk framework  Project duration: 320 man hours
  • 25.
    “splunk inside AWS”– extended usage of splunk Samplesofprojects  Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  Research of how Splunk manages its data  Check if Splunk starts to delete data after a while when disk space becomes to small  Plan is to have for one year all access log info in Splunk  Data will be kept forever (until it is deleted) in S3.  Investigate how data can exported out of Splunk into S3 (lets say on a daily basis).  Project duration: 60 man hours
  • 26.
    “advanced postgres usage” Samplesofprojects Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  Adjust the standard postgres cookbooks to support Percona wrapper  A proxy server which can act in front of a Postgres Master and n slaves. (like similar solutions of Mysql - MariaDB Maxscale and MySQL Proxy)  The proxy should send all insert/update/delete statements to the mast er and the select statements to the slaves. Everything should be finally based on cookbooks.  Project duration: 220 man hours
  • 27.
    “auto scaling magento”implement auto scale for a production magento stack Samplesofprojects  Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  Adjustments to magentostack cookbook  Automatic load of large demo data  Load testing plan to be able to test auto scaling feature  Implementation and testing of auto scaling feature based on Amazon CloudWatch metrics  Project duration: 40 man hours
  • 28.
    “auto scaling wordpress” Samplesofprojects Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  Adjustments to wordpress related cookbook  Load testing plan to be able to test auto scaling feature  Implementation and testing of auto scaling feature without CloudWatch metrics  Project duration: 20 man hours
  • 29.
    “postgres disaster recovery”disaster recovery based on volume snapshots Samplesofprojects  Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  postgres data directory is defined on a separate volume  a snapshot is made at regular time intervals for that volume.  In case of a crash, another machine is started and the data volume is created from latest snapshot at start time.  Project duration: 60 man hours
  • 30.
    “AWS CloudTrail integrationwith Splunk” Samplesofprojects  Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  Cookbooks for Splunk Enterprise installation  Cookbook for Splunk App plugin installation and configuration  Configure Cloudtrail logging to send SNS topic that should point to Amazon SQS queue  Testing the integration regarding different EC2 API calls  Project duration: 40 man hours
  • 31.
    “PCI DSS Compliance”Implementation of PCI security standards (PCI DSS version 1.2) Samplesofprojects  Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Requirements overview: Tasks: PCI Reference: Migration to an operating system with current security patches (Ubuntu 10.04). PCI-DSS requires installing critical security patches within one month of release. PCI-DSS 6.1 Installation of a minimal operating system. De-installation of not required services and drivers. PCI-DSS: 2.2.2, 2.2.4 Installation and configuration of an access control system l ike Radius or OpenLDAP for all EC2 instances. PCI-DSS 7.1.4, 7.2, 8.3 Configuration of an access control system: Only named account usage. No root account. No account sharing. PCI-DSS: 8.1, 8.5 Configuration of Amazon Identity and Access Management (IAM) rules. Limit access to EC2 security groups (PCI-DSS terminology: firewall) and EBS snapshots. Separation of EC2 access keys. PCI-DSS 1.1.1, 6.3.2 Review and re-configuration of EC2 security groups. PCI-DSS 1.2, 1.3, 6.3.2 Configuration of host based firewall with iptables. PCI-DSS 1.2.1, 1.3.5 Installation of anti-virus software (like F-PROT) on all EC2 instances. PCI-DSS 5.1 Review of MySQL security settings and reconfiguration. Preferably: Security configuration with Radius/LDAP server . PCI-DSS: 8.5.16 Installation and configuration of an audit logging system. PCI-DSS 10.2, 10.3, 10.5, 10.7 Installation and configuration of a reliable time service. PCI-DSS 10.4 Installation and configuration of an intrusion detection system like Tripwire or AIDE. PCI-DSS: 10.5.5, 11.5 Installation and configuration of an intrusion prevention system like Snort. PCI-DSS 11.4 Internal vulnerability scans with Nessus. PCI-DSS 11.2, 11.3 Installation and configuration of an application firewall like AppArmor/SELinux for all web-server EC2 instances. PCI-DSS 6.6 Review VPN configuration to ensure a two-factor authentication. PCI-DSS 8.3
  • 32.
    “PCI DSS Compliance”Implementation of PCI security standards (PCI DSS version 1.2) Samplesofprojects  Project details:  Customizing the iptables cookbook to automatically set up rules based on machines' roles  Cookbook to remove unneeded packages from a freshly installed machine (part of PCI standardization)  Cookbook for vtun setup  Cookbook for splunk setup  Cookbook for snort, openlda, openssh  Cookbook for Ossec setup  Cookbook that would set up a daily ubuntu dist repository  Cookbook that would backup rotated apache logs both on s3 and glacier  Cookbook to create local users using databags  Implementation of different testing scenario  Project duration: 120 man hours
  • 33.
    “improve reports offull-text search engine” search-engine reporting Samplesofprojects  Virtualization environment: local servers, Amazon EC2  Configuration management: Opscode Chef  Project details:  Cookbooks installing virtualbox, vagrant, test-kitchen and Jenkins on Ubuntu 12.04 under VMWare  Usage of vagrant as underlying virtualization plugin for test-kitchen  Usage of test-kitchen as integration testing tool (check if a node converges)  Vagrant could run either VirtualBox or VMWare or directly EC2 machines  Vagrant could provision either with Chef Solo or with a real Chef Server  Assertions using minitest and vagrant  ChefSpec based assertions for cookbook recipies  Jenkins jobs to start test-kitchen and vagrant – configured through cookbooks  Project duration: 160 man hours
  • 34.
    “enlarge processing power”export scanned documents into tex t having large but cheap processing power Samplesofprojects  Virtualization environment: Amazon EC2  Configuration management: Opscode Chef  Project details:  Cookbooks installing customized OCR software  Usage of Amazon Spot Instances and FleetAPI  Usage of Amazon SQS  Project duration: 120 man hours
  • 35.
    “quality assurance incookbooks world” different testing procedures for Chef configuration management Samplesofprojects  Virtualization environment: local servers,  Configuration management: Opscode Chef  Project details:  Cookbooks for elastic search installation and configuration  Cookbooks for logstash installation and configuration  Cookbooks for kibana installation and configuration  Shells scripts for a primitive solution of self learning search system  Project duration: 16 man hours
  • 36.
    “Jenkins runs Docker”Docker based containers, managed by Jenkins, for testing purposes Samplesofprojects  Virtualization environment: local servers,  Docker account  Project details:  Even Jenkins runs under a Docker container so in the end we have Docker inside Docker  Docker file to install Jenkins in a Docker container  Shell scripting called by Jenkins jobs to start Docker containers  Integration of magento web stack for testing under Jenkins & Docker  Project duration: 60 man hours
  • 37.
    Real live demoto prove the reality behind our experience  Our demo consists in doing a complete Magento installation on an EC2 machine inside AWS  The machine will be launched inside an autoscaling group  The autoscaling group has an Elastic Load Balancer in front  We define scaling policies inside the autoscaling group, based on the average CPU load, that will start up new Magento machines when load is too high or will stop some of the existing machines when load is under defined limits  To test the scaling policies we use Apache Benchmark (ab) utility to create load on the existing machines
  • 38.
  • 39.
    Reallivedemotoprove thereality behindourexperience  Using theApache Bench tool (ab) we perform load tests by sending a large num ber of concurrent requests  Based on the configuration below, a new instance was started and an alarm email was sent, notifying about the started instance
  • 40.
    Reallivedemotoprove thereality behindourexperience  Here isthe notification email which confirms the custom scaling feature imple mented.
  • 41.
    Looking forward fora long term cooperation Thank you for your attention and interest !