SlideShare a Scribd company logo
1 of 38
@michaelneale
www.CloudBees.com
@michaelneale
github.com/michaelneale
currently: trying to make less reliable things more
reliable.
Some things we use to get it done:
- scala (back end apps)
- ruby (chef, rails, proxymachine, lots)
- java (jenkins and plugins)
- erlang (“agents” and lots more..)
Exceptions. Errors.
What The Hell Is That Process Doing?
and how it relates to..
On Call PTSD
Exception handling is wrong...
But it seems harmless...
Erlang says no no no...
Question: a process on server not responding
Answer:
1) panic
2) take time thinking about underlying cause
3) bounce the bastard and pray **
FACT: Answer#3 will often work so well, you
often don’t really spend time to think about
doing #2**



** FibreError?
Imagine a programming environment:
 - unlimited process spawning
 - concurrency someone else’s problem
 - don’t try to handle errors - let it fail
Erlang
Built for stability, and not bothering me.
Anything else is gravy


I wasn’t an erlang programmer
Process isolation
- it works so well
- nothing shared
- has worked so well for so long with OSes
- in erlang - each ‘process’ is like a process !
nginx    Example: My Host OS

     haproxy               Erlang process

                         supervisor
                            ...
                       p1         p999



Each: own heap, garbage collector
Erlang is:
- A vm based runtime (light !)
- language with prolog heritage (!)
- compiler
- interpreter
- debug tools - batteries included
- OTP - an application (server?) platform
Erlang is:
- Not OO
- no shared state **
- state kept in “term stores” - mini DBs
- processes easy to hot swap
- Quasi functional programming
    (immutable variables, higher order)
- distributed (if you want)
OTP
- Open Telecommunications Platform
- not about telco !
- network stacks/apis, supervisor trees
- toolkit for reliable servers (middleware)
- behaviours
State in erlang




files
         Host + real kernel
                        Erlang process

DETS                supervisor
             ...
ETS                p1                    p999



              Send                       Receive
Erlang processes, like OS processes


- Can only send messages to each other
- Can fail independently
- Can be restarted, safely, independently
- Easy to write apps for - Old School !
Better


- Tiny overhead (can have 100K+ of them !)
- perfect for supervisory agents
- distribute: run on every box if you like !
- lots of processes == lots of concurrency
- if you need to block, block !
Have used erlang for


- Supervisory agents - controllers
- Autoscaling services (realtime data analytics)
- misc backend systems
- fun command line tools !
- is “lite” VM (startup, footprint)
Supervisor configuration


- Declare rules for restarting, failure
- So you don’t have to intervene !
- can hot reload changes
- can have dependent processes
Restart just busted process
Program 1 (module with behaviour)
Program 2 (module with behaviour)
Anyway, that is why we use Erlang, also...
- RabbitMQ
- Riak (nosql DB)
- CouchDB (nosql DB)
- Probably your telephone exchange
Alternative approaches:
- Use “native” with daemontools/monit etc
- Akka (Scala) has similar supervisor
- library for your favourite language
- ephemeral “state less” apps
- Write software without bugs
- Or just restart everything 400 times a day
What is not awesome:
- OTP boiler plate
- VM not as fast as some
- No one seems to like the language
- tools? IDEs?
Thank you

More Related Content

What's hot

Cassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQLCassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQL
Eric Evans
 
Spark intro by Adform Research
Spark intro by Adform ResearchSpark intro by Adform Research
Spark intro by Adform Research
Vasil Remeniuk
 

What's hot (19)

Application Profiling for Memory and Performance
Application Profiling for Memory and PerformanceApplication Profiling for Memory and Performance
Application Profiling for Memory and Performance
 
Async and Await on the Server
Async and Await on the ServerAsync and Await on the Server
Async and Await on the Server
 
Task parallel library presentation
Task parallel library presentationTask parallel library presentation
Task parallel library presentation
 
Debugging ansible modules
Debugging ansible modulesDebugging ansible modules
Debugging ansible modules
 
Cassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQLCassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQL
 
Celery introduction
Celery introductionCelery introduction
Celery introduction
 
Testing in Scala. Adform Research
Testing in Scala. Adform ResearchTesting in Scala. Adform Research
Testing in Scala. Adform Research
 
Ansible: What, Why & How
Ansible: What, Why & HowAnsible: What, Why & How
Ansible: What, Why & How
 
Spark intro by Adform Research
Spark intro by Adform ResearchSpark intro by Adform Research
Spark intro by Adform Research
 
Play Framework
Play FrameworkPlay Framework
Play Framework
 
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
 
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
 
Automating Perl deployments with Hudson
Automating Perl deployments with HudsonAutomating Perl deployments with Hudson
Automating Perl deployments with Hudson
 
Async/Await
Async/AwaitAsync/Await
Async/Await
 
Open stack and_vagrant-os-meetup-2015
Open stack and_vagrant-os-meetup-2015Open stack and_vagrant-os-meetup-2015
Open stack and_vagrant-os-meetup-2015
 
Coroutines in Kotlin
Coroutines in KotlinCoroutines in Kotlin
Coroutines in Kotlin
 
The art of concurrent programming
The art of concurrent programmingThe art of concurrent programming
The art of concurrent programming
 
Gatling @ Scala.Io 2013
Gatling @ Scala.Io 2013Gatling @ Scala.Io 2013
Gatling @ Scala.Io 2013
 
Async await
Async awaitAsync await
Async await
 

Viewers also liked (7)

裝飾設計
裝飾設計裝飾設計
裝飾設計
 
Errors and handling them. YOW nights Sydney 2011
Errors and handling them. YOW nights Sydney 2011Errors and handling them. YOW nights Sydney 2011
Errors and handling them. YOW nights Sydney 2011
 
Devoxx 2014 michael_neale
Devoxx 2014 michael_nealeDevoxx 2014 michael_neale
Devoxx 2014 michael_neale
 
Scala sydoct2011
Scala sydoct2011Scala sydoct2011
Scala sydoct2011
 
Cors michael
Cors michaelCors michael
Cors michael
 
Legal counsel it_cv
Legal counsel it_cvLegal counsel it_cv
Legal counsel it_cv
 
Cross site calls with javascript - the right way with CORS
Cross site calls with javascript - the right way with CORSCross site calls with javascript - the right way with CORS
Cross site calls with javascript - the right way with CORS
 

Similar to Osdc 2011 michael_neale

0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
Maxim Kharchenko
 
Joe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsJoe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystems
Sentifi
 
PuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into OperationsPuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into Operations
grim_radical
 
Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018
Holden Karau
 
Chelberg ptcuser 2010
Chelberg ptcuser 2010Chelberg ptcuser 2010
Chelberg ptcuser 2010
Clay Helberg
 
DevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 SlidesDevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 Slides
Alex Cruise
 
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB
 

Similar to Osdc 2011 michael_neale (20)

Repeating History...On Purpose...with Elixir
Repeating History...On Purpose...with ElixirRepeating History...On Purpose...with Elixir
Repeating History...On Purpose...with Elixir
 
Erlang
ErlangErlang
Erlang
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
 
An introduction to erlang
An introduction to erlangAn introduction to erlang
An introduction to erlang
 
Joe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsJoe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystems
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys Admins
 
REX about installing Kubernetes on Premise
REX about installing Kubernetes on PremiseREX about installing Kubernetes on Premise
REX about installing Kubernetes on Premise
 
PuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into OperationsPuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into Operations
 
You shall not get excited
You shall not get excitedYou shall not get excited
You shall not get excited
 
Understanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New YorkUnderstanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New York
 
Spark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New YorkSpark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New York
 
Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018
 
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
 
Chelberg ptcuser 2010
Chelberg ptcuser 2010Chelberg ptcuser 2010
Chelberg ptcuser 2010
 
DevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 SlidesDevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 Slides
 
Introduction to Erlang/(Elixir) at a Webilea Hands-On Session
Introduction to Erlang/(Elixir) at a Webilea Hands-On SessionIntroduction to Erlang/(Elixir) at a Webilea Hands-On Session
Introduction to Erlang/(Elixir) at a Webilea Hands-On Session
 
Concurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple SpacesConcurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple Spaces
 
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, Analytics
 
Elixir
ElixirElixir
Elixir
 
Beam me up, Scotty
Beam me up, ScottyBeam me up, Scotty
Beam me up, Scotty
 

More from Michael Neale

More from Michael Neale (11)

Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)
 
Cd syd
Cd sydCd syd
Cd syd
 
Microservices and functional programming
Microservices and functional programmingMicroservices and functional programming
Microservices and functional programming
 
Java one 2011_michaelneale
Java one 2011_michaelnealeJava one 2011_michaelneale
Java one 2011_michaelneale
 
Sjug aug 2010_cloud
Sjug aug 2010_cloudSjug aug 2010_cloud
Sjug aug 2010_cloud
 
SJUG March 2010 Restful design
SJUG March 2010 Restful designSJUG March 2010 Restful design
SJUG March 2010 Restful design
 
On Scala Slides - OSDC 2009
On Scala Slides - OSDC 2009On Scala Slides - OSDC 2009
On Scala Slides - OSDC 2009
 
Osdc Complex Event Processing
Osdc Complex Event ProcessingOsdc Complex Event Processing
Osdc Complex Event Processing
 
Scala Sjug 09
Scala Sjug 09Scala Sjug 09
Scala Sjug 09
 
Jaoo Michael Neale 09
Jaoo Michael Neale 09Jaoo Michael Neale 09
Jaoo Michael Neale 09
 
Osdc Michael Neale 2008
Osdc Michael Neale 2008Osdc Michael Neale 2008
Osdc Michael Neale 2008
 

Recently uploaded

Recently uploaded (20)

How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024Top 10 Most Downloaded Games on Play Store in 2024
Top 10 Most Downloaded Games on Play Store in 2024
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 

Osdc 2011 michael_neale

  • 2. @michaelneale github.com/michaelneale currently: trying to make less reliable things more reliable.
  • 3. Some things we use to get it done: - scala (back end apps) - ruby (chef, rails, proxymachine, lots) - java (jenkins and plugins) - erlang (“agents” and lots more..)
  • 4. Exceptions. Errors. What The Hell Is That Process Doing? and how it relates to..
  • 7. But it seems harmless...
  • 8.
  • 9.
  • 10. Erlang says no no no...
  • 11. Question: a process on server not responding Answer: 1) panic 2) take time thinking about underlying cause 3) bounce the bastard and pray **
  • 12. FACT: Answer#3 will often work so well, you often don’t really spend time to think about doing #2** ** FibreError?
  • 13.
  • 14. Imagine a programming environment: - unlimited process spawning - concurrency someone else’s problem - don’t try to handle errors - let it fail
  • 15. Erlang Built for stability, and not bothering me. Anything else is gravy I wasn’t an erlang programmer
  • 16. Process isolation - it works so well - nothing shared - has worked so well for so long with OSes - in erlang - each ‘process’ is like a process !
  • 17. nginx Example: My Host OS haproxy Erlang process supervisor ... p1 p999 Each: own heap, garbage collector
  • 18. Erlang is: - A vm based runtime (light !) - language with prolog heritage (!) - compiler - interpreter - debug tools - batteries included - OTP - an application (server?) platform
  • 19. Erlang is: - Not OO - no shared state ** - state kept in “term stores” - mini DBs - processes easy to hot swap - Quasi functional programming (immutable variables, higher order) - distributed (if you want)
  • 20.
  • 21.
  • 22.
  • 23. OTP - Open Telecommunications Platform - not about telco ! - network stacks/apis, supervisor trees - toolkit for reliable servers (middleware) - behaviours
  • 24.
  • 25.
  • 26.
  • 27. State in erlang files Host + real kernel Erlang process DETS supervisor ... ETS p1 p999 Send Receive
  • 28. Erlang processes, like OS processes - Can only send messages to each other - Can fail independently - Can be restarted, safely, independently - Easy to write apps for - Old School !
  • 29. Better - Tiny overhead (can have 100K+ of them !) - perfect for supervisory agents - distribute: run on every box if you like ! - lots of processes == lots of concurrency - if you need to block, block !
  • 30. Have used erlang for - Supervisory agents - controllers - Autoscaling services (realtime data analytics) - misc backend systems - fun command line tools ! - is “lite” VM (startup, footprint)
  • 31. Supervisor configuration - Declare rules for restarting, failure - So you don’t have to intervene ! - can hot reload changes - can have dependent processes
  • 33. Program 1 (module with behaviour)
  • 34. Program 2 (module with behaviour)
  • 35. Anyway, that is why we use Erlang, also... - RabbitMQ - Riak (nosql DB) - CouchDB (nosql DB) - Probably your telephone exchange
  • 36. Alternative approaches: - Use “native” with daemontools/monit etc - Akka (Scala) has similar supervisor - library for your favourite language - ephemeral “state less” apps - Write software without bugs - Or just restart everything 400 times a day
  • 37. What is not awesome: - OTP boiler plate - VM not as fast as some - No one seems to like the language - tools? IDEs?

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n