OSDConf 2014
“In love with Open Source : Past,
Present and Future…“
~/Piyush
http://piyush.me
In real open source,
you have the right to
control your own
destiny.
~Linus Torvalds
OSDConf 2014 : http://piyush.me 2
Mostly Everyone Uses/Loves Open
Source Directly or Indirectly
• Linux
• Android
• Mozilla Firefox
• LibreOffice/OpenOffice...
4 C’s of Open Source
• Community
• Collaboration
• Collective Ownership of intellectual property
rights
– Distributed Cred...
My entry to open source…
• I started with using Linux
• then started using a classic LAMP stack as Dev
• And then in sea o...
Journey….
• 2003-2004 -> Linux User back there in my
college days
• 2005 -> became first Red Hat Certified
Engineer of col...
2006 -> 2008
• Started as QE – Quality Engineer at
NexTag.com (27 months) part of QA-
Infrastructure team having worked on...
Key Learning's
• COMMUNITY : IRC #centos channel freenode
– Inspired by awesome z00dax : Karanbir Singh
• HABIT : IRC #noc...
2008 -> till date
• I played various roles in Web Operations
(which demanded multidisciplinary skills )
– System Engineer
...
DDoS Protection
Firewall - ASA
IPS – Intrusion Detection / Prevention System
WAF – Web Application Firewall
Load Balancer
...
MMT OSS powered Tech Stack
• Programming Languages
– JAVA / PHP / Python
• Operating System – CentOS 5/6
• Caching proxy
–...
• NoSQL
– SOLR / ElasticSearch (Lucene) – Search
– Cassandra
– Couchbase
• Monitoring
– Graphite / Diamond
– Nagios
– NOCM...
• Cheetah
– Selenium , TestNG , Pyvnc2swf
– Jenkins
• Neerikshan
– Observium (network monitoring)
• Version Control
– GIT,...
• Virtualization
– XenServer
• Configuration Management - Puppet
• Application Frameworks
– Spring (JAVA)
– Laravel (PHP)
...
Cheetah
• cheetah : Test automation and business
service monitoring tool
– uses Selenium and Jenkins(Hudson) integration f...
Few other hacks
• BizEye SynC Tool
• MyClient
• Image Thumbnail Processing
– ImageMagick
• MyIdeas
– Drupal
OSDConf 2014 :...
LBManager
• Managing LTM and GTM using Django /
PyControl
• Single interface for managing the
LoadBalancers hosted in mult...
Graphite/Diamond & NOCMATE-Rx
• Monitoring for Operational Excellence
– Easy interface to collect data / Metrics
– Trends,...
Graphite : Trends / Analysis / Dashboards
OSDConf 2014 : http://piyush.me 19
Lmetric (Logging Metric)
• Centralized Log and event storage and search !
• Powered by :
– Logstash
– ElasticSearch
– Kiba...
Personare : BigData Stack
• To collect valuable user interaction information for
subsequent use in personalization,
recomm...
Personare : Architecture
OSDConf 2014 : http://piyush.me 22
OSS powered BigData Stack
• Log Collection: Logstash, Syslog-NG
• Distributed messaging system : Kafka , RabbitMQ
• ETL : ...
My next hope ! …goal ?
• Would I be able to apply what all I have
learned in technology through OSS to solve
some issues f...
Thanks!
Questions (If any ) ?
Contact:
@piykumar
http://piyush.me
OSDConf 2014 : http://piyush.me 25
Upcoming SlideShare
Loading in...5
×

"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014

2,798

Published on

OPEN SOURCE DEVELOPERS CONFERENCE http://osdconf.in/
★ April 26-27th, Noida ★
Keynote Session By Piyush Kumar (Lead of Infrastructure and Website Operations at MakeMyTrip)

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,798
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
19
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014

  1. 1. OSDConf 2014 “In love with Open Source : Past, Present and Future…“ ~/Piyush http://piyush.me
  2. 2. In real open source, you have the right to control your own destiny. ~Linus Torvalds OSDConf 2014 : http://piyush.me 2
  3. 3. Mostly Everyone Uses/Loves Open Source Directly or Indirectly • Linux • Android • Mozilla Firefox • LibreOffice/OpenOffice • Apache HTTP server • Asterisk (PBX) • VLC media player • Wordpress OSDConf 2014 : http://piyush.me 3
  4. 4. 4 C’s of Open Source • Community • Collaboration • Collective Ownership of intellectual property rights – Distributed Credit • Continuous, public process – Software releases part of the process – Open peer review – Public Bug Tracking – Testing OSDConf 2014 : http://piyush.me 4
  5. 5. My entry to open source… • I started with using Linux • then started using a classic LAMP stack as Dev • And then in sea of Infrastructure (WebOps) it was open source all around • Now playing with BigData Stack again … its OSS … making our life easier ! OSDConf 2014 : http://piyush.me 5
  6. 6. Journey…. • 2003-2004 -> Linux User back there in my college days • 2005 -> became first Red Hat Certified Engineer of college • 2006-2008 -> NexTag Journey as QE • 2008-2013 -> MakeMyTrip, Website Operations Team • 2013-till now -> BI / Lead Data Platform Team OSDConf 2014 : http://piyush.me 6
  7. 7. 2006 -> 2008 • Started as QE – Quality Engineer at NexTag.com (27 months) part of QA- Infrastructure team having worked on managing QA Servers, Testing of NexTag Application (Search Indexes, Cache System, Monitoring Applications) and Automation • Performance Testing of OS, Apache HTTP server, Tomcat + Every New Release • Managing F5 BigIP LTM in QA & Google Mini (Search Engine) OSDConf 2014 : http://piyush.me 7
  8. 8. Key Learning's • COMMUNITY : IRC #centos channel freenode – Inspired by awesome z00dax : Karanbir Singh • HABIT : IRC #noc channel for production updates @ NexTag Infra • Learned the art of – ./configure && make && make install • Love for BASH #!/bin/bash • SOA – Service Oriented Architecture • Performance Testing - Jmeter/Replay ( Load / Stress ) • Automated Testing – Sahi / Selenium • Code Coverage - Cobertura • Importance of managing Infrastructure like your web applications – Continuous updates / improvements – Automation – Monitoring OSDConf 2014 : http://piyush.me 8
  9. 9. 2008 -> till date • I played various roles in Web Operations (which demanded multidisciplinary skills ) – System Engineer – Security Operations – Lead Infrastructure – ISM – Information Security Manager – Lead DataPlatformTeam OSDConf 2014 : http://piyush.me 9 hacker@makemytrip
  10. 10. DDoS Protection Firewall - ASA IPS – Intrusion Detection / Prevention System WAF – Web Application Firewall Load Balancer Web Server Engine / SOA / APIs RDBMS - Transactional Database NoSQL Caching Technologies Virtualization / Cloud Stack Email / DNS / AUTH / Config Management / CI/Deployment CDN – content delivery network Log Management - SIEM Typical Web Infra Stack Obviously including Hardware, Network & Power  OSDConf 2014 : http://piyush.me 10
  11. 11. MMT OSS powered Tech Stack • Programming Languages – JAVA / PHP / Python • Operating System – CentOS 5/6 • Caching proxy – ATS – Apache Traffic Server • Webserver – Apache HTTPD – Nginx • Application Server – Apache Tomcat / Jetty – JBoss • RDBMS – MySQL OSDConf 2014 : http://piyush.me 11
  12. 12. • NoSQL – SOLR / ElasticSearch (Lucene) – Search – Cassandra – Couchbase • Monitoring – Graphite / Diamond – Nagios – NOCMATE-Rx • Lmetric – Logging Infrastructure – Syslog – Logstash – ElasticSearch – Kibana – Web Dashboard OSDConf 2014 : http://piyush.me 12
  13. 13. • Cheetah – Selenium , TestNG , Pyvnc2swf – Jenkins • Neerikshan – Observium (network monitoring) • Version Control – GIT, SVN , CVS – GitLab – Git Management • Continuous Integration – Jenkins • Performance Testing – Jmeter / ab / soapui • IDE – Eclipse / vim OSDConf 2014 : http://piyush.me 13
  14. 14. • Virtualization – XenServer • Configuration Management - Puppet • Application Frameworks – Spring (JAVA) – Laravel (PHP) – Django / Flask (Python) • Knowledge Management – Twiki / Wordpress • CMS – Drupal • Security – OSSIM / AlienVault – OWASP ZAP • …..and many more …. OSDConf 2014 : http://piyush.me 14
  15. 15. Cheetah • cheetah : Test automation and business service monitoring tool – uses Selenium and Jenkins(Hudson) integration for Real Time monitoring of website with nice Reporting and Alerting – record sessions using Pyvnc2swf OSDConf 2014 : http://piyush.me 15
  16. 16. Few other hacks • BizEye SynC Tool • MyClient • Image Thumbnail Processing – ImageMagick • MyIdeas – Drupal OSDConf 2014 : http://piyush.me 16
  17. 17. LBManager • Managing LTM and GTM using Django / PyControl • Single interface for managing the LoadBalancers hosted in multi-colocations / IDCs OSDConf 2014 : http://piyush.me 17
  18. 18. Graphite/Diamond & NOCMATE-Rx • Monitoring for Operational Excellence – Easy interface to collect data / Metrics – Trends, Dashboards, Alerting , Reporting – User specific/LOB specific dashboards • Desi principle: If it moves, we track it :) • “You cannot manage what you do not measure and to measure you need to monitor” • 5 Categories of Metrics – System | Network | Security | Business | Application • Poller’s – DBPoller, SNMP, Script etc. OSDConf 2014 : http://piyush.me 18
  19. 19. Graphite : Trends / Analysis / Dashboards OSDConf 2014 : http://piyush.me 19
  20. 20. Lmetric (Logging Metric) • Centralized Log and event storage and search ! • Powered by : – Logstash – ElasticSearch – Kibana – Web Dashboard OSDConf 2014 : http://piyush.me 20
  21. 21. Personare : BigData Stack • To collect valuable user interaction information for subsequent use in personalization, recommendations, analysis and reporting. • Layered Architecture (Lambda): – Batch Layer : Hadoop M/R – Speed Layer : Storm NRT Computation – Serving Layer OSDConf 2014 : http://piyush.me 21
  22. 22. Personare : Architecture OSDConf 2014 : http://piyush.me 22
  23. 23. OSS powered BigData Stack • Log Collection: Logstash, Syslog-NG • Distributed messaging system : Kafka , RabbitMQ • ETL : Pentaho/ Talend • Hadoop Ecosystem • NRT Computation: Twitter Storm • Document-Oriented NoSQL DB : Couchbase • Distributed Search: ElasticSearch • Distributed messaging system : Kafka , RabbitMQ • NoSQL : Cassandra, Redis, Neo4J (Graph) • API Management : WSO2 API Manager • Programming Languages : Java , Python, R OSDConf 2014 : http://piyush.me 23
  24. 24. My next hope ! …goal ? • Would I be able to apply what all I have learned in technology through OSS to solve some issues faced by masses around the world ..? – Using technology to make world a better place for everyone ! and I am sure OSS will have major role to play in it !! OSDConf 2014 : http://piyush.me 24
  25. 25. Thanks! Questions (If any ) ? Contact: @piykumar http://piyush.me OSDConf 2014 : http://piyush.me 25
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×