Erlang at Nu Echo
Some lessons learned
Dominique BoucherDominique Boucher
Nu Echo
Agenda
Who we are, what we doWho we are, what we do
NuGram Hosted Server
Why Erlang
Architecture
Some lessons
Who we are / what we do
• Speech applicationsSpeech applications
experts
Lots of R&D• Lots of R&D
• Many tools to
support ...
Some of our tools
NuBot
NuGram IDE
NuGram Hosted Server
Hosted infrastructure for deploying andHosted infrastructure for deploying and
sharing speech grammar...
NuGram Hosted Server
Components
RESTf l API– RESTful API
– www.grammarserver.com/api
C it b it– Community website
– www.gr...
NuGram Hosted Server
NuGram
Hosted Server
Your application
(Java Ruby PythonHosted Server (Java, Ruby, Python,
JavaScript,...
NuGram Hosted Server
An example in Ruby
dyngrammar = "#ABNF 1.0 UTF-8;
...
"
server = GrammarServer.new()
session = server...
Why Erlang?
• Initial requirementsInitial requirements
• High-performance HTTP server
• Scalability Fault tolerance• Scala...
Components
• Behaviours for building robust
applications
• Behaviours for building robust
applications
OTP
(Open Telecom
P...
High-level architecture
JInterface JInterface JInterface
www.grammarserver.com
(http + https)
Yaws
nugram.nuecho.com
(http...
Session handling
Java/Kawa Worker
Java/Kawa Worker
Java Thread
Erlang Node
Ja a ead
Session
Dispatcher
Mnesia
Yaws
Dispatc...
Session handling
Java/Kawa Worker
Java/Kawa Worker
Java Thread
Erlang NodeErlang Node
Ja a ead
cated)
Session
cated)
Dispa...
Session handling
• Sending messages to remote nodes isSending messages to remote nodes is
completely transparent
E l ( i l...
Session handling
• Session timeouts handled by theSession timeouts handled by the
session process
E lExample
receive
…
aft...
Mnesia
• StoresStores…
• User accounts
• All grammars
(templates and instantiated ones)
• Node IDs of all Java workers
• A...
What we’ve developed
Account management
Registration, email confirmation, etc.
BBcode validation & conversion for comments...
What we’ve developed
Code-based representationp
of the web site structure
Tree like structureTree-like structure
Documents...
Web site structure
-define(SITE_CONFIGURATION,
{ i d{section, "product_app", "Home",
[{module, "welcome",
"Welcome - NuGra...
Lessons learned
• Erlang strengths
• Glueing things togetherg g g
• Data transformations
• bbcode XML• bbcode, XML
• Proto...
Lessons learned
E l k• Erlang weaknesses
• Sequential computations
harder to express
• A matter of style
• String manipula...
Lessons learned
• Learn and use OTP
from day one
• Servers,
supervisors,
applicationsapplications
• Worth the investment
E...
Lessons learned
• Dynamically-typed languages makeDynamically typed languages make
non-trivial refactorings difficult and
...
Lessons learned
• Know your toolsKnow your tools
• Toolbar, tv,
appmon, pmanpp , p
D 't i t th h l• Don't reinvent the whe...
We learned a lot!
And we had a lot of fun()!()
halt()halt().
Upcoming SlideShare
Loading in …5
×

Erlang at Nu Echo

2,179 views
2,026 views

Published on

Slides from a talk at the Erlang Montreal user group.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,179
On SlideShare
0
From Embeds
0
Number of Embeds
401
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Erlang at Nu Echo

  1. 1. Erlang at Nu Echo Some lessons learned Dominique BoucherDominique Boucher Nu Echo
  2. 2. Agenda Who we are, what we doWho we are, what we do NuGram Hosted Server Why Erlang Architecture Some lessons
  3. 3. Who we are / what we do • Speech applicationsSpeech applications experts Lots of R&D• Lots of R&D • Many tools to support our own practice • Grammar tools • Benchmarking/tuning tools • Testing/monitoring toolsTesting/monitoring tools
  4. 4. Some of our tools NuBot NuGram IDE
  5. 5. NuGram Hosted Server Hosted infrastructure for deploying andHosted infrastructure for deploying and sharing speech grammars – Supports static AND dynamic grammars– Supports static AND dynamic grammars – Embeds the NuGram technologies • Dynamic grammar instantiation• Dynamic grammar instantiation • Grammar conversion to ABNF/GrXML/GSL • Text-based semantic interpretationp • To complement telephony in the cloudTo complement telephony in the cloud (eg. Tropo)
  6. 6. NuGram Hosted Server Components RESTf l API– RESTful API – www.grammarserver.com/api C it b it– Community website – www.grammarserver.com – NuGram product website – nugram.nuecho.com
  7. 7. NuGram Hosted Server NuGram Hosted Server Your application (Java Ruby PythonHosted Server (Java, Ruby, Python, JavaScript, Erlang, etc.) Telephony Platform
  8. 8. NuGram Hosted Server An example in Ruby dyngrammar = "#ABNF 1.0 UTF-8; ... " server = GrammarServer.new() session = server.create session("username", "passwrd")session server.create_session( username , passwrd ) session.upload("digits2.abnf", dyngrammar); grammar = session.instantiate("digits2.abnf", {}) interpretation i t t(' t th f ') t j= grammar.interpret('one two three four').to_json session.disconnect
  9. 9. Why Erlang? • Initial requirementsInitial requirements • High-performance HTTP server • Scalability Fault tolerance• Scalability, Fault-tolerance • Hot code swapping Di t ib t d d t b• Distributed database • The Erlang appeal • Single language, single platform • Ideal projectp j • for experimenting with a different technology
  10. 10. Components • Behaviours for building robust applications • Behaviours for building robust applications OTP (Open Telecom Pl tf ) applicationsapplicationsPlatform) • High performance HTTP Server• High performance HTTP ServerYaws • High-performance HTTP Server• High-performance HTTP ServerYaws • Distributed, fault-tolerant, soft realtime DB • Distributed, fault-tolerant, soft realtime DB Mnesia • Java API to expose Java/Kawa code as a regular Erlang node • Java API to expose Java/Kawa code as a regular Erlang node Jinterface
  11. 11. High-level architecture JInterface JInterface JInterface www.grammarserver.com (http + https) Yaws nugram.nuecho.com (http + https) Y MnesiaRESTful API (http + https)
  12. 12. Session handling Java/Kawa Worker Java/Kawa Worker Java Thread Erlang Node Ja a ead Session Dispatcher Mnesia Yaws Dispatcher Request Response
  13. 13. Session handling Java/Kawa Worker Java/Kawa Worker Java Thread Erlang NodeErlang Node Ja a ead cated) Session cated) Dispatcher sia(replic DispatcherDispatcher sia(replic Yaws Dispatcher Mnes Yaws Dispatcher Yaws Dispatcher Mnes Request Response
  14. 14. Session handling • Sending messages to remote nodes isSending messages to remote nodes is completely transparent E l ( i lifi d)Example (simplified) Process = db:get session(SessionID)Process = db:get_session(SessionID), Process ! Request.
  15. 15. Session handling • Session timeouts handled by theSession timeouts handled by the session process E lExample receive … after Timeout -> db:expire_session(self()) end
  16. 16. Mnesia • StoresStores… • User accounts • All grammars (templates and instantiated ones) • Node IDs of all Java workers • All session data
  17. 17. What we’ve developed Account management Registration, email confirmation, etc. BBcode validation & conversion for comments Twitter widget A process caches the tweetsA process caches the tweets and updates it every 5 minutes Rendering of tweetse de g o ee s RSS reader / widget A process caches the feed contentA process caches the feed content and updates it every hour
  18. 18. What we’ve developed Code-based representationp of the web site structure Tree like structureTree-like structure Documents, modules, sections Supports the addition on new “modules” Lots of unit tests!
  19. 19. Web site structure -define(SITE_CONFIGURATION, { i d{section, "product_app", "Home", [{module, "welcome", "Welcome - NuGram Platform", {nugram_site, welcome_with_news, []}}, {document, "system_error", "Warning - Request not completed", "system_error.html"}, {section, "nugram_platform", "NuGram Platform", [{document, "overview", "Overview", "nugram_overview.html"}, {document, "faq", "FAQ", "faq.html"},{ , q , , q }, {document, "abnf_tutorial", "Grammar Language Tutorial", "abnf_tutorial.html"}, ……
  20. 20. Lessons learned • Erlang strengths • Glueing things togetherg g g • Data transformations • bbcode XML• bbcode, XML • Protocol handling / processing byte streamsstreams • Like encoding detection G t d i d l t i t• Great dynamic development environment • REPL + connecting to a running node
  21. 21. Lessons learned E l k• Erlang weaknesses • Sequential computations harder to express • A matter of style • String manipulationg p • Somewhat better now (support for encodings)
  22. 22. Lessons learned • Learn and use OTP from day one • Servers, supervisors, applicationsapplications • Worth the investment E l t itt f d• Example: twitter feed server • First version without OTP crashed way too often making the platform unreliable• crashed way too often, making the platform unreliable • Second version using gen_server
  23. 23. Lessons learned • Dynamically-typed languages makeDynamically typed languages make non-trivial refactorings difficult and error-proneerror-prone • Not specific to Erlang, b t litbut a reality • Dializer can help • Hot code loading is greatHot code loading is great • But tricky to do right
  24. 24. Lessons learned • Know your toolsKnow your tools • Toolbar, tv, appmon, pmanpp , p D 't i t th h l• Don't reinvent the wheel • There are a lot more libraries now • we use Yaws, oauth, smtp_fsm • rebar - great build tool
  25. 25. We learned a lot! And we had a lot of fun()!()
  26. 26. halt()halt().

×