Erlang for Five Nines bar camp Cork III Tamas Nagy [email_address] © 1999 – 2009 Erlang Training and Consulting Ltd
Contents Erlang History Erlang Highlights Products Questions
Erlang History: The Telecom Industry Complex Reliable Scalable Distributed Maintainable Highly  Available versus Time To Market!   Access transport and switching networks Cellular PLMN PSTN/ ISDN Data/ IP Networks CATV Services Past  Single-service networks Clients/applications Present Multiservice networks/client server Backbone Network Access Access Access Content Content Control Communication applications Media Gateways
Erlang History 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 “ Find the right methods -  design by prototyping.” Mike Willliams, CS LAB Make mistakes on a small scale, not in a  production project. Mike Willliams It’s not good enough to have ideas – you must also be able to implement them to know  that they work Mike Willliams And the rest is history... Prototypes of Telecom applications First projects launched First products launched Major projects started OTP R1  released Released as Open Source Experiments started at the Computer Science Lab First Erlang Book Published First C-Based Virtual Machine
Erlang History And this is  just the beginning... “ Erlang is going to be a very important language. It could be the next Java.” Ralph Johnson Co-author, “Design Patterns” (the “Gang-of-Four book”) To build a large scale message handling system that really had to be up all the time, I would unhesitatingly choose Erlang…. Tim Bray Sun Microsystems … if we had to start again we  would probably use Erlang… Mike Shaver Mozilla Foundation 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 The Future January, 36,000 hits on erlang.org Bluetail AB  acquired for 152M USD August, 1 million hits on erlang.org Erlang R11 goes  multicore  The first  Erlang Factory Major new projects started within Ericsson ETC is  founded
Requests to erlang.org
Contents Erlang History Erlang Highlights Products Questions
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Functional programming  language High abstraction level Pattern matching Concise readable programs
Erlang Highlights: Factorial Factorial using Recursion n! = 1 n*(n-1)! n = 0 n   1 Definition -module(ex1). -export([factorial/1]). factorial(0) -> 1; factorial(N) when N >= 1 -> N * factorial(N-1). Implementation Eshell V5.0.1  (abort with ^G) 1> c(ex1). {ok,ex1} 2> ex1:factorial(6). 720
Erlang Highlights: High-level Constructs -define(IP_VERSION, 4). -define(IP_MIN_HDR_LEN, 5).  …… DgramSize = size(Dgram),  <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16, ID:16,    Flgs:3, FragOff:13, TTL:8, Proto:8,  HdrChkSum:16,    SrcIP:32, DestIP:32, Body/binary>> = Dgram, if (HLen >= 5) and (4*HLen =< DgramSize) -> OptsLen = 4*(HLen - ?IP_MIN_HDR_LEN), <<Opts:OptsLen/binary, Data/binary>> = Body, … ..  end. Parsing an IP Datagram using the Bit Syntax
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Either transparent or  explicit concurrency Light-weight processes Highly scalable
Erlang Highlights: Concurrency Creating a new process using spawn -module(ex3). -export([activity/3]). activity(Name,Pos,Size) -> ………… Pid = spawn(ex3,activity,[Joe,75,1024]) activity(Joe,75,1024)
Erlang Highlights: Concurrency 10 100 1,000 10,000 100,000 Number of processes 1 10 100 1,000 Microseconds/process erlang java C# Source: Joe Armstrong SICS Process creation time
Erlang Highlights: Concurrency Processes communicate by asynchronous  message passing Pid ! {data,12,13} receive {start} -> ……… {stop} -> ……… {data,X,Y} -> ……… end receive {start} -> ……… {stop} -> ……… {data,X,Y} -> ……… end
Erlang Highlights: Concurrency 10 100 1,000 10,000 100,000 Number of processes 1 10 1,000 100,000 Microseconds/message erlang java C# 10,000 100 1 Source: Joe Armstrong SICS Message passing times
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Response times in the order of milliseconds Per-process garbage collection
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Simple and consistent error recovery Supervision hierarchies &quot;Program for the correct case&quot;
Erlang Highlights: Robustness Robust systems can be built by layering “ Supervisors” “ Workers”
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Explicit or transparent  distribution Network-aware runtime system
Erlang Highlights: Distribution Erlang Run-Time System Erlang Run-Time System B ! Msg network C ! Msg
Erlang Highlights: Distribution loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. {rex, Node} ! {self(), {apply, M, F, A}}, receive {rex, Node, What} -> What end {rex, Node} ! {self(), {apply, M, F, A}}, receive {rex, Node, What} -> What end {rex, Node} ! {self(), {apply, M, F, A}}, receive {rex, Node, What} -> What end loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. Simple Remote Procedure Call
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Easily change code in a  running system Enables non-stop operation Simplifies testing
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support &quot;Ports&quot; to the outside world behave as Erlang processes
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Erlang runs on a Virtual  Machine ported to UNIX, Windows, VxWorks, OS X, … Supports heterogeneous networks.
Erlang Highlights Declarative Concurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Symmetric multiprocessing  support. Takes full advantage  of multiple CPU  architectures.
Erlang SMP Erlang VM Scheduler run queue non-SMP VM Erlang VM Scheduler #2 run queue run queue run queue SMP VM Scheduler #1 Scheduler #N migration logic
Contents Erlang History Erlang Highlights Products Questions
Products: Ericsson AXD301 Switch A Telephony-Class,  scalable  (10 –160 GBps)  ATM switch  Designed from scratch in less than 3 years Proof of concept in three months 1.5 million lines of Erlang code in the OM 500,000 lines of C/C++ in the switch core 13,000 lines of Java in the GUI AXD 301 Success factors: Competent organisation and people Efficient process Excellent technology (Erlang!)
Products: Ericsson AXD301 Switch Using Erlang in Complex Systems Fits very well with the incremental design method High programmer satisfaction Outstanding support for robustness and concurrency Very few side-effects    easier to add/change single components Small directed teams can achieve impressive results Productivity estimates Similar line/hour programmer productivity 4-10 fewer lines of source code (compared to C/C++, Java, PLEX) Similar number of faults per 1000 lines of source code
Erlang: It’s Happening! Amazon  Simple DB Yahoo!  Delicious T-Mobile  WAP, SMS, IN services Facebook  Chat channel servers   Powerset/Microsoft  Key/Value store 37 Signals Backend servers
Erlang: It’s Happening! CouchDB   Distributed Robust  document database Wings 3D   3D modeller based on Nendo  YAWS   Yet Another Web Server RabbitMQ   High performance enterprise messaging Ejabberd   XMPP  instant messaging server
Questions
www.planeterlang.org www.trapexit.org www.erlang-factory.com www.erlang-consulting.com www.erlang.org

Erlang For Five Nines

  • 1.
    Erlang for FiveNines bar camp Cork III Tamas Nagy [email_address] © 1999 – 2009 Erlang Training and Consulting Ltd
  • 2.
    Contents Erlang HistoryErlang Highlights Products Questions
  • 3.
    Erlang History: TheTelecom Industry Complex Reliable Scalable Distributed Maintainable Highly Available versus Time To Market! Access transport and switching networks Cellular PLMN PSTN/ ISDN Data/ IP Networks CATV Services Past Single-service networks Clients/applications Present Multiservice networks/client server Backbone Network Access Access Access Content Content Control Communication applications Media Gateways
  • 4.
    Erlang History 19871988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 “ Find the right methods - design by prototyping.” Mike Willliams, CS LAB Make mistakes on a small scale, not in a production project. Mike Willliams It’s not good enough to have ideas – you must also be able to implement them to know that they work Mike Willliams And the rest is history... Prototypes of Telecom applications First projects launched First products launched Major projects started OTP R1 released Released as Open Source Experiments started at the Computer Science Lab First Erlang Book Published First C-Based Virtual Machine
  • 5.
    Erlang History Andthis is just the beginning... “ Erlang is going to be a very important language. It could be the next Java.” Ralph Johnson Co-author, “Design Patterns” (the “Gang-of-Four book”) To build a large scale message handling system that really had to be up all the time, I would unhesitatingly choose Erlang…. Tim Bray Sun Microsystems … if we had to start again we would probably use Erlang… Mike Shaver Mozilla Foundation 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 The Future January, 36,000 hits on erlang.org Bluetail AB acquired for 152M USD August, 1 million hits on erlang.org Erlang R11 goes multicore The first Erlang Factory Major new projects started within Ericsson ETC is founded
  • 6.
  • 7.
    Contents Erlang HistoryErlang Highlights Products Questions
  • 8.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Functional programming language High abstraction level Pattern matching Concise readable programs
  • 9.
    Erlang Highlights: FactorialFactorial using Recursion n! = 1 n*(n-1)! n = 0 n  1 Definition -module(ex1). -export([factorial/1]). factorial(0) -> 1; factorial(N) when N >= 1 -> N * factorial(N-1). Implementation Eshell V5.0.1 (abort with ^G) 1> c(ex1). {ok,ex1} 2> ex1:factorial(6). 720
  • 10.
    Erlang Highlights: High-levelConstructs -define(IP_VERSION, 4). -define(IP_MIN_HDR_LEN, 5). …… DgramSize = size(Dgram), <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16, ID:16, Flgs:3, FragOff:13, TTL:8, Proto:8, HdrChkSum:16, SrcIP:32, DestIP:32, Body/binary>> = Dgram, if (HLen >= 5) and (4*HLen =< DgramSize) -> OptsLen = 4*(HLen - ?IP_MIN_HDR_LEN), <<Opts:OptsLen/binary, Data/binary>> = Body, … .. end. Parsing an IP Datagram using the Bit Syntax
  • 11.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Either transparent or explicit concurrency Light-weight processes Highly scalable
  • 12.
    Erlang Highlights: ConcurrencyCreating a new process using spawn -module(ex3). -export([activity/3]). activity(Name,Pos,Size) -> ………… Pid = spawn(ex3,activity,[Joe,75,1024]) activity(Joe,75,1024)
  • 13.
    Erlang Highlights: Concurrency10 100 1,000 10,000 100,000 Number of processes 1 10 100 1,000 Microseconds/process erlang java C# Source: Joe Armstrong SICS Process creation time
  • 14.
    Erlang Highlights: ConcurrencyProcesses communicate by asynchronous message passing Pid ! {data,12,13} receive {start} -> ……… {stop} -> ……… {data,X,Y} -> ……… end receive {start} -> ……… {stop} -> ……… {data,X,Y} -> ……… end
  • 15.
    Erlang Highlights: Concurrency10 100 1,000 10,000 100,000 Number of processes 1 10 1,000 100,000 Microseconds/message erlang java C# 10,000 100 1 Source: Joe Armstrong SICS Message passing times
  • 16.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Response times in the order of milliseconds Per-process garbage collection
  • 17.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Simple and consistent error recovery Supervision hierarchies &quot;Program for the correct case&quot;
  • 18.
    Erlang Highlights: RobustnessRobust systems can be built by layering “ Supervisors” “ Workers”
  • 19.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Explicit or transparent distribution Network-aware runtime system
  • 20.
    Erlang Highlights: DistributionErlang Run-Time System Erlang Run-Time System B ! Msg network C ! Msg
  • 21.
    Erlang Highlights: Distributionloop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. {rex, Node} ! {self(), {apply, M, F, A}}, receive {rex, Node, What} -> What end {rex, Node} ! {self(), {apply, M, F, A}}, receive {rex, Node, What} -> What end {rex, Node} ! {self(), {apply, M, F, A}}, receive {rex, Node, What} -> What end loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. Simple Remote Procedure Call
  • 22.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Easily change code in a running system Enables non-stop operation Simplifies testing
  • 23.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support &quot;Ports&quot; to the outside world behave as Erlang processes
  • 24.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Erlang runs on a Virtual Machine ported to UNIX, Windows, VxWorks, OS X, … Supports heterogeneous networks.
  • 25.
    Erlang Highlights DeclarativeConcurrency Soft real-time Robustness Distribution Hot code loading External interfaces Portability SMP Support Symmetric multiprocessing support. Takes full advantage of multiple CPU architectures.
  • 26.
    Erlang SMP ErlangVM Scheduler run queue non-SMP VM Erlang VM Scheduler #2 run queue run queue run queue SMP VM Scheduler #1 Scheduler #N migration logic
  • 27.
    Contents Erlang HistoryErlang Highlights Products Questions
  • 28.
    Products: Ericsson AXD301Switch A Telephony-Class, scalable (10 –160 GBps) ATM switch Designed from scratch in less than 3 years Proof of concept in three months 1.5 million lines of Erlang code in the OM 500,000 lines of C/C++ in the switch core 13,000 lines of Java in the GUI AXD 301 Success factors: Competent organisation and people Efficient process Excellent technology (Erlang!)
  • 29.
    Products: Ericsson AXD301Switch Using Erlang in Complex Systems Fits very well with the incremental design method High programmer satisfaction Outstanding support for robustness and concurrency Very few side-effects  easier to add/change single components Small directed teams can achieve impressive results Productivity estimates Similar line/hour programmer productivity 4-10 fewer lines of source code (compared to C/C++, Java, PLEX) Similar number of faults per 1000 lines of source code
  • 30.
    Erlang: It’s Happening!Amazon Simple DB Yahoo! Delicious T-Mobile WAP, SMS, IN services Facebook Chat channel servers Powerset/Microsoft Key/Value store 37 Signals Backend servers
  • 31.
    Erlang: It’s Happening!CouchDB Distributed Robust document database Wings 3D 3D modeller based on Nendo YAWS Yet Another Web Server RabbitMQ High performance enterprise messaging Ejabberd XMPP instant messaging server
  • 32.
  • 33.
    www.planeterlang.org www.trapexit.org www.erlang-factory.comwww.erlang-consulting.com www.erlang.org

Editor's Notes

  • #22 This is the essential loop needed for an RPC server. The server receives requests from clients, uses apply to evaluate the requests and then returns the answer to the originator. This is a simplified solution where no error handling is done. The BIF node/0 returns the node name.
  • #27 Ulf Wiger, Ericsson AB 2009-01-16 ProTest - Property-Based Testing The non-SMP version of the Erlang virtual machine uses a cooperative reduction-counting scheduler with a single run queue (the run queue itself is partitioned according to priorities – ’low’, ’normal’, ’high’ and ’max’). Each runnable process gets a ”time slice” of 1000 reductions (1 reduction is basically one function call), but the time slice can be cut short if the process pauses to wait for a message (i.e. whenever a receive statement would block). There is also a yield() function, allowing processes to deliberately end the time slice. From an Erlang programmer’s perspective, the scheduling is preemptive, since it cannot be certain that it won’t be scheduled out at any given moment. In practice, it has been possible to stack the deck by calling yield() and then performing a sequence of calls that one wants to be ”atomic”. This is obviously questionable, and the practice has always been discouraged. Still, some programs have relied on the fact that the non-SMP scheduler was highly predictable – it was possible to ”know” that preemption wouldn’t occur at certain points in the code. (Another ”feature” was that the priorities ’high’ and ’max’ were strict – as long as there were runnable jobs on these priority levels, ’normal’ and ’low’ priority jobs would never get to execute. Partly for this reason, use of priorities is discourages in Erlang.)