Bhasker V Kode, co-founder and CTO of hover.in, gave a presentation at Erlang Factory in London in June 2009 about lessons learned from applying Erlang at hover.in. He discussed how hover.in uses Erlang processes and message passing to model behaviors seen in bacteria colonies and the human brain. Key aspects of the Erlang implementation include spawning worker processes, pattern matching, and scaling to handle large amounts of in-memory data through data partitioning and priority queues.
Parsing binaries and protocols with erlangBhasker Kode
Delivered by Bhasker V Kode at foss.in/2009
Official talk page at http://foss.in/2009/schedules/talkdetailspub.php?talkid=17
Erlang 's support for handling binaries and pattern matching make it a great choice for parsing everything from IPv4 packets, to payloads from the Memcached protocol, SWF files, or databases like Tokyo Cabinet. From a functional programming perspective, there are various ways of building these parsers, taking advantage of the concurrent and recursive nature that is inherent to the language and other challenges which have been gathered while validating the storage & retrieval options for our distributed crawler, and submitting patches to projects like Medici & Tora ( erlang based Tokyo Cabinet clients). The talk will also touch upon Tokyo cabinet's support for mapreduce with Lua, and notes from building your own custom formats & our internal mapreduce'esque and caching frameworks used in building a multi-million impression platform utilizing under a gig of RAM per node.
Notes on:
- trends in disk/memory/bandwidth
- why erlang, RAM, binaries
- garbage collection in the erlang VM
- message passing
- use-cases
The document discusses how Erlang is well-suited for building concurrent and distributed systems like Hover.in's platform. Some key points covered include:
- Hover.in uses Erlang's lightweight processes and message passing model to build a concurrent, fault-tolerant architecture.
- Erlang learns from nature's concurrent systems like bacteria colonies and the human brain to address challenges like scaling and fault tolerance.
- Examples of how Erlang is used at Hover.in include building a clustered architecture, handling high volumes of requests, and implementing services like caching, queues and supervision of worker processes.
Package manages and Puppet - PuppetConf 2015ice799
This talk will begin by explaining what a package manager is and how package managers work, at a high level. Next, we'll observe the common patterns seen on the internet of compiling software in a Puppet manifest and discuss why this not ideal. This talk will conclude by showing how you can add package repositories to your infrastructure using Puppet and what settings are important for ensuring secure access to remote package repositories.
This document discusses customizing the Bash shell prompt. It covers incorporating standard escape sequences to display information like the username and directory in the prompt. It provides suggestions for modifying prompts and xterm title bars, using external functions and ANSI colors. The document includes many code snippets and examples for incorporating features like the date, number of processes, and power status into prompts.
Open Source Tools for Leveling Up Operations FOSSET 2014Mandi Walls
This document discusses using open source tools to improve operations workflows and processes. It introduces various tools including Git for version control, packaging tools like FPM, and testing tools like Nagios plugins. The document advocates applying principles from development like testing, version control, and automation to make operations processes more reliable, transparent and reduce risk.
This document provides instructions for installing Hubot on OSX and writing custom scripts for it in JavaScript to create a bot for Slack. It outlines installing Hubot and its dependencies, deploying it to Heroku, configuring it for the Slack adapter, and publishing scripts to it. Tips are provided like checking Heroku logs for debugging and referring to CoffeeScript code since documentation for the JavaScript version is limited. The overall process allows one to make a fun bot for Slack without an extensive programming background.
Parsing binaries and protocols with erlangBhasker Kode
Delivered by Bhasker V Kode at foss.in/2009
Official talk page at http://foss.in/2009/schedules/talkdetailspub.php?talkid=17
Erlang 's support for handling binaries and pattern matching make it a great choice for parsing everything from IPv4 packets, to payloads from the Memcached protocol, SWF files, or databases like Tokyo Cabinet. From a functional programming perspective, there are various ways of building these parsers, taking advantage of the concurrent and recursive nature that is inherent to the language and other challenges which have been gathered while validating the storage & retrieval options for our distributed crawler, and submitting patches to projects like Medici & Tora ( erlang based Tokyo Cabinet clients). The talk will also touch upon Tokyo cabinet's support for mapreduce with Lua, and notes from building your own custom formats & our internal mapreduce'esque and caching frameworks used in building a multi-million impression platform utilizing under a gig of RAM per node.
Notes on:
- trends in disk/memory/bandwidth
- why erlang, RAM, binaries
- garbage collection in the erlang VM
- message passing
- use-cases
The document discusses how Erlang is well-suited for building concurrent and distributed systems like Hover.in's platform. Some key points covered include:
- Hover.in uses Erlang's lightweight processes and message passing model to build a concurrent, fault-tolerant architecture.
- Erlang learns from nature's concurrent systems like bacteria colonies and the human brain to address challenges like scaling and fault tolerance.
- Examples of how Erlang is used at Hover.in include building a clustered architecture, handling high volumes of requests, and implementing services like caching, queues and supervision of worker processes.
Package manages and Puppet - PuppetConf 2015ice799
This talk will begin by explaining what a package manager is and how package managers work, at a high level. Next, we'll observe the common patterns seen on the internet of compiling software in a Puppet manifest and discuss why this not ideal. This talk will conclude by showing how you can add package repositories to your infrastructure using Puppet and what settings are important for ensuring secure access to remote package repositories.
This document discusses customizing the Bash shell prompt. It covers incorporating standard escape sequences to display information like the username and directory in the prompt. It provides suggestions for modifying prompts and xterm title bars, using external functions and ANSI colors. The document includes many code snippets and examples for incorporating features like the date, number of processes, and power status into prompts.
Open Source Tools for Leveling Up Operations FOSSET 2014Mandi Walls
This document discusses using open source tools to improve operations workflows and processes. It introduces various tools including Git for version control, packaging tools like FPM, and testing tools like Nagios plugins. The document advocates applying principles from development like testing, version control, and automation to make operations processes more reliable, transparent and reduce risk.
This document provides instructions for installing Hubot on OSX and writing custom scripts for it in JavaScript to create a bot for Slack. It outlines installing Hubot and its dependencies, deploying it to Heroku, configuring it for the Slack adapter, and publishing scripts to it. Tips are provided like checking Heroku logs for debugging and referring to CoffeeScript code since documentation for the JavaScript version is limited. The overall process allows one to make a fun bot for Slack without an extensive programming background.
Hubot is an open source chatbot framework written in CoffeeScript. It can be customized and installed on platforms like Heroku. Hubot scripts allow it to perform tasks like translating languages, integrating with maps and images. The document provides instructions on setting up Hubot with the Telegram adapter, including creating a Telegram bot and running Hubot with the Telegram token. It also suggests experiments like editing external scripts and writing custom scripts.
In the world of social gaming, the classic 2-tier of web application does not cut it anymore. We need new and better solutions.
Follow along the evolution of game servers at Wooga and get an in-depth look into the next-generation backend putting the combined forces of Erlang and Ruby to work. Learn how scalability, reliability, concurrency control and beautiful code do not need to be mutually exclusive.
Puppet getting started will show the different components used in puppet environments, starting with facter and puppet to different webinterfaces like puppet enterprise console and foreman. It will also cover an exemplary design for scaling the puppet master and for development livecycle of modules. Furthermore an example for design of modules will be given.
The document discusses the polyglot capabilities of JBoss AS 7 through projects like TorqueBox and Immutant. TorqueBox allows Ruby applications to run on JBoss AS 7, providing services like web applications, background jobs, messaging, and more through integration with Java libraries. Similarly, Immutant aims to enable Clojure applications to run on JBoss AS 7 with analogous services. The goal is to leverage JBoss AS 7 as a platform while allowing developers to choose the best programming language for the task.
Bob McWhirter is a JBoss Fellow and Chief Architect of Middleware Cloud Computing. He founded The Codehaus, Drools, and TorqueBox. The document discusses BoxGrinder, a tool that can create virtual machine appliances from definition files in order to simplify deploying software to infrastructure platforms like Amazon EC2 or VMware. It describes how BoxGrinder supports both "baking" and "frying" approaches to creating VMs and walks through an example of using BoxGrinder to build a JBoss application server appliance.
Bob McWhirter gave a presentation on TorqueBox, an open-source Ruby application server built on the Java Virtual Machine. Some key points:
- TorqueBox allows Ruby applications like Rails to take advantage of features traditionally provided by Java application servers like scalability, messaging, jobs, and telephony.
- It provides queues for asynchronous processing and scheduling jobs to run on a cron-like schedule directly from Ruby classes.
- The use of the Java VM allows clustering and high availability of Ruby applications in the same way achieved with Java applications.
- All components like queues, jobs, and clustering work seamlessly together since everything is integrated within TorqueBox.
TorqueBox allows mixing Java and Ruby by running Ruby code on the Java Virtual Machine (JVM). It provides Ruby applications access to enterprise Java features like JNDI, JMS, Quartz, and more. TorqueBox applications can be deployed to JBoss Application Server with these Java services and capabilities.
TorqueBox allows Ruby applications to leverage the Java Application Server and Java EE services by running Ruby code on JRuby and integrating it with the underlying Java platform. It provides services like messaging, caching, background jobs, and more to Ruby applications. TorqueBox handles the integration between Ruby frameworks/APIs and the corresponding Java APIs and services, allowing Ruby developers to build enterprise applications that scale.
Under the Dome (of failure driven pipeline)Maciej Lasyk
The document discusses various topics related to DevOps including:
1. Different types of shells (login, non-login, interactive, non-interactive, su, sudo su, sudo -i, sudo /bin/bash, sudo -s) and how they affect environment variables and profile files.
2. Stories of organizational "anti-types" that go against DevOps principles like not seeing the need for operations teams.
3. How automation, consistency, and reducing errors leads to stable environments and less unplanned work, allowing teams to focus on delivery.
TorqueBox: The beauty of Ruby with the power of JBoss. Presented at Devnexus...bobmcwhirter
- Bob McWhirter is the project lead of TorqueBox and a JBoss Fellow.
- TorqueBox allows Ruby web applications to run on the JBoss Application Server using JRuby.
- It provides tight integration with JBoss and allows Ruby applications to take advantage of features like messaging, jobs, and services that are traditionally Java-based.
Dennis Matotek, Technical Lead Platforms at Experian Hitwise Australia, gave an excellent presentation on setting up puppet using vagrant, puppet and testing, including a full demo of rspec-puppet and Jenkins.
Every day, sysadmins find the need to build tools that get the job done in a fast and efficient way. Go is a new and powerful—yet simple—language that is an excellent fit for the needs of the harried sysadmin. This tutorial provides an introduction to Go with a focus on using it for everday tooling.
Topics include:
Understanding of the applicability of Go to everyday work
The Go environment and toolset
Go language fundamentals:
Control Structures
Functions
Data Types
Methods and Interfaces
Goroutines
Channels
Examples in Go:
Email
Web Server
Directory Tools
The document discusses a Python tutorial presentation on systems programming. It describes building programs to analyze Firefox browser cache files, including a findcache.py program that recursively searches a directory for Firefox cache folders. The goal is to demonstrate Python for practical system tasks like file parsing and processing. Disclaimers note the code only uses standard Python and is intended for educational purposes.
Gofer is a scalable stateless proxy architecture for DBI that is transport independent, highly configurable, efficient, well tested, scalable, and simple. It consists of a simple request/response protocol, a DBI proxy driver called DBD::Gofer, a request executor module, pluggable transport modules like HTTP, SSH, and Gearman, and an extensible client configuration mechanism. It aims to minimize round trips and supports connection pooling to improve performance and scalability.
JRuby is a great way to use native Java libraries and get around the project overhead of Java, but how do you actually use Java from JRuby? This talk explores building a JRuby application, backed by the portable Java version of Couchbase Mobile.
While JRuby is build to interface with Java, when calling out to JVM land there are all those little hurdles to overcome. Handling Strings correctly, using native Collection types, and interfacing with libraries which expect those native types instead of the ones provided by JRuby, and last but not least Implementing native interfaces to pass around.
Oh and by the way all of this runs on a little Raspberry Pi!
This document provides an overview of Linux, Apache, MySQL, and PHP (LAMP) including:
- Client-server architecture with Apache as a web server and browsers as clients
- Modules that extend Apache's functionality like PHP for server-side scripting
- Using MySQL for databases and administering users
- Integrating PHP, Apache, and MySQL to dynamically generate web pages.
Complex Made Simple: Sleep Better with TorqueBoxbobmcwhirter
The document discusses using TorqueBox, a Ruby application server based on JRuby and JBoss AS7, to deploy a Rails application in production. It compares various deployment options from rolling your own infrastructure to using a platform as a service like Heroku. TorqueBox provides a middle ground where it handles services like caching, background jobs, scheduling, and clustering but still allows customization. The document walks through migrating an existing Rails app's Delayed::Job and caching implementations to use TorqueBox equivalents to simplify the deployment.
Kafka & Storm - FifthElephant 2015 by @bhaskerkode, HelpshiftBhasker Kode
The document discusses how Kafka's key distinguishing feature is its published protocol specification that defines how clients communicate with Kafka brokers. This allows different clients to integrate with Kafka by simply implementing the protocol over TCP, without relying on a specific client library. It also enables the ecosystem to develop rapidly due to wide adoption. The protocol focuses on efficiency through techniques like zero-copy transfer of message data directly from kernel space to sockets.
Bhasker V Kode , (Co-Founder & CTO - Hover.in ) talking about erlang + engineering efforts at the Commercial Users of Functional Programming 2009, Edinburgh
Hubot is an open source chatbot framework written in CoffeeScript. It can be customized and installed on platforms like Heroku. Hubot scripts allow it to perform tasks like translating languages, integrating with maps and images. The document provides instructions on setting up Hubot with the Telegram adapter, including creating a Telegram bot and running Hubot with the Telegram token. It also suggests experiments like editing external scripts and writing custom scripts.
In the world of social gaming, the classic 2-tier of web application does not cut it anymore. We need new and better solutions.
Follow along the evolution of game servers at Wooga and get an in-depth look into the next-generation backend putting the combined forces of Erlang and Ruby to work. Learn how scalability, reliability, concurrency control and beautiful code do not need to be mutually exclusive.
Puppet getting started will show the different components used in puppet environments, starting with facter and puppet to different webinterfaces like puppet enterprise console and foreman. It will also cover an exemplary design for scaling the puppet master and for development livecycle of modules. Furthermore an example for design of modules will be given.
The document discusses the polyglot capabilities of JBoss AS 7 through projects like TorqueBox and Immutant. TorqueBox allows Ruby applications to run on JBoss AS 7, providing services like web applications, background jobs, messaging, and more through integration with Java libraries. Similarly, Immutant aims to enable Clojure applications to run on JBoss AS 7 with analogous services. The goal is to leverage JBoss AS 7 as a platform while allowing developers to choose the best programming language for the task.
Bob McWhirter is a JBoss Fellow and Chief Architect of Middleware Cloud Computing. He founded The Codehaus, Drools, and TorqueBox. The document discusses BoxGrinder, a tool that can create virtual machine appliances from definition files in order to simplify deploying software to infrastructure platforms like Amazon EC2 or VMware. It describes how BoxGrinder supports both "baking" and "frying" approaches to creating VMs and walks through an example of using BoxGrinder to build a JBoss application server appliance.
Bob McWhirter gave a presentation on TorqueBox, an open-source Ruby application server built on the Java Virtual Machine. Some key points:
- TorqueBox allows Ruby applications like Rails to take advantage of features traditionally provided by Java application servers like scalability, messaging, jobs, and telephony.
- It provides queues for asynchronous processing and scheduling jobs to run on a cron-like schedule directly from Ruby classes.
- The use of the Java VM allows clustering and high availability of Ruby applications in the same way achieved with Java applications.
- All components like queues, jobs, and clustering work seamlessly together since everything is integrated within TorqueBox.
TorqueBox allows mixing Java and Ruby by running Ruby code on the Java Virtual Machine (JVM). It provides Ruby applications access to enterprise Java features like JNDI, JMS, Quartz, and more. TorqueBox applications can be deployed to JBoss Application Server with these Java services and capabilities.
TorqueBox allows Ruby applications to leverage the Java Application Server and Java EE services by running Ruby code on JRuby and integrating it with the underlying Java platform. It provides services like messaging, caching, background jobs, and more to Ruby applications. TorqueBox handles the integration between Ruby frameworks/APIs and the corresponding Java APIs and services, allowing Ruby developers to build enterprise applications that scale.
Under the Dome (of failure driven pipeline)Maciej Lasyk
The document discusses various topics related to DevOps including:
1. Different types of shells (login, non-login, interactive, non-interactive, su, sudo su, sudo -i, sudo /bin/bash, sudo -s) and how they affect environment variables and profile files.
2. Stories of organizational "anti-types" that go against DevOps principles like not seeing the need for operations teams.
3. How automation, consistency, and reducing errors leads to stable environments and less unplanned work, allowing teams to focus on delivery.
TorqueBox: The beauty of Ruby with the power of JBoss. Presented at Devnexus...bobmcwhirter
- Bob McWhirter is the project lead of TorqueBox and a JBoss Fellow.
- TorqueBox allows Ruby web applications to run on the JBoss Application Server using JRuby.
- It provides tight integration with JBoss and allows Ruby applications to take advantage of features like messaging, jobs, and services that are traditionally Java-based.
Dennis Matotek, Technical Lead Platforms at Experian Hitwise Australia, gave an excellent presentation on setting up puppet using vagrant, puppet and testing, including a full demo of rspec-puppet and Jenkins.
Every day, sysadmins find the need to build tools that get the job done in a fast and efficient way. Go is a new and powerful—yet simple—language that is an excellent fit for the needs of the harried sysadmin. This tutorial provides an introduction to Go with a focus on using it for everday tooling.
Topics include:
Understanding of the applicability of Go to everyday work
The Go environment and toolset
Go language fundamentals:
Control Structures
Functions
Data Types
Methods and Interfaces
Goroutines
Channels
Examples in Go:
Email
Web Server
Directory Tools
The document discusses a Python tutorial presentation on systems programming. It describes building programs to analyze Firefox browser cache files, including a findcache.py program that recursively searches a directory for Firefox cache folders. The goal is to demonstrate Python for practical system tasks like file parsing and processing. Disclaimers note the code only uses standard Python and is intended for educational purposes.
Gofer is a scalable stateless proxy architecture for DBI that is transport independent, highly configurable, efficient, well tested, scalable, and simple. It consists of a simple request/response protocol, a DBI proxy driver called DBD::Gofer, a request executor module, pluggable transport modules like HTTP, SSH, and Gearman, and an extensible client configuration mechanism. It aims to minimize round trips and supports connection pooling to improve performance and scalability.
JRuby is a great way to use native Java libraries and get around the project overhead of Java, but how do you actually use Java from JRuby? This talk explores building a JRuby application, backed by the portable Java version of Couchbase Mobile.
While JRuby is build to interface with Java, when calling out to JVM land there are all those little hurdles to overcome. Handling Strings correctly, using native Collection types, and interfacing with libraries which expect those native types instead of the ones provided by JRuby, and last but not least Implementing native interfaces to pass around.
Oh and by the way all of this runs on a little Raspberry Pi!
This document provides an overview of Linux, Apache, MySQL, and PHP (LAMP) including:
- Client-server architecture with Apache as a web server and browsers as clients
- Modules that extend Apache's functionality like PHP for server-side scripting
- Using MySQL for databases and administering users
- Integrating PHP, Apache, and MySQL to dynamically generate web pages.
Complex Made Simple: Sleep Better with TorqueBoxbobmcwhirter
The document discusses using TorqueBox, a Ruby application server based on JRuby and JBoss AS7, to deploy a Rails application in production. It compares various deployment options from rolling your own infrastructure to using a platform as a service like Heroku. TorqueBox provides a middle ground where it handles services like caching, background jobs, scheduling, and clustering but still allows customization. The document walks through migrating an existing Rails app's Delayed::Job and caching implementations to use TorqueBox equivalents to simplify the deployment.
Kafka & Storm - FifthElephant 2015 by @bhaskerkode, HelpshiftBhasker Kode
The document discusses how Kafka's key distinguishing feature is its published protocol specification that defines how clients communicate with Kafka brokers. This allows different clients to integrate with Kafka by simply implementing the protocol over TCP, without relying on a specific client library. It also enables the ecosystem to develop rapidly due to wide adoption. The protocol focuses on efficiency through techniques like zero-copy transfer of message data directly from kernel space to sockets.
Bhasker V Kode , (Co-Founder & CTO - Hover.in ) talking about erlang + engineering efforts at the Commercial Users of Functional Programming 2009, Edinburgh
Recursion & Erlang, FunctionalConf 14, BangaloreBhasker Kode
The document discusses the history and design of the Erlang programming language. Some key points:
1) Erlang was designed in 1986 at Ericsson for writing concurrent programs that "run forever." It was created by Joe Armstrong to address the needs of building telephony systems.
2) Concurrency was the primary goal in designing Erlang. This influenced decisions like message passing between processes instead of shared memory, and copying data between processes for isolation.
3) Tail recursion and the actor model were incorporated due to their suitability for implementing concurrent processes and distributed systems. Tail recursion allows processes to be spawned efficiently while preserving state.
QCON SP 2016 - Elixir: Tolerância a Falhas para AdultosFabio Akita
Elixir é uma das novas linguagens que está despontando principalmente pela reputação de capacidade de altíssima concorrência e paralelismo que sua VM permite. Mas o principal elemento é que Erlang e por consequência, Elixir, é uma das poucas linguagens criadas assumindo que vamos cometer erros. Ele garante alta disponibilidade nos dando primitivas que permitam que nosso software continue funcionando, mesmo que às vezes ele falhe. E agora vamos ver como Erlang faz isso como Elixir expõe esse potencial a todos nós.
There Are Literally Thousands of Erlang ProjectsPierre Fenoll
Present state of erldocs.com and its future: API search engine, erldocs skins & possibility to run jobs on every Erlang projects, all for the benefit of the larger Erlang community
High Performance Erlang - Pitfalls and SolutionsYinghai Lu
Presented at Erlang Factory 2016, San Francisco, CA.
Erlang is widely used for building concurrent applications. However, when we push the performance of our Erlang based application to handle millions of concurrent clients, some Erlang scalability issues begin to show and some conventional programming paradigm of Erlang no longer hold. We would like to share some of these issue and how we address them. In addition, we share some of our experience on how to profile an Erlang application to identify bottlenecks.
We will take a deep look at some of the basic mechanisms of Erlang and show how they behave under high load and parallelism, which includes message delivery, process management and shared data structures such as maps and ETS tables. We will demonstrate their limitations and propose techniques to alleviate the issues.
We will also share profiling techniques on how to find those bottlenecks in Erlang applications across different levels. We will share techniques for writing highly performant Erlang applications.
Erlang supervision trees helps bring fault tolerance, recovery and robustness to erlang applications. The presentation goes through basic and advanced examples along with supervision strategies.
Erlang is a functional programming language with built-in support for concurrency, distribution, and fault tolerance. It uses lightweight processes to enable highly scalable concurrent programming. Processes communicate asynchronously by message passing and are monitored to detect and prevent failures. Erlang was designed for building distributed, soft real-time systems with requirements for high availability such as telecom applications.
Elixir - Easy fun for busy developers @ Devoxx 2016David Schmitz
The document is a presentation about Elixir and its features. It introduces Elixir and how it combines Erlang's robustness with Ruby's elegance. It demonstrates building a distributed pizza ordering application in Elixir to showcase features like processes, messaging, and supervision. It also discusses Elixir's macros, polymorphism with protocols, hot code reloading, and real-time capabilities with Phoenix. While promising for many tasks, the presenter notes Elixir may not be suitable for all projects and emphasizes its strengths in distributed, cloud-native microservices.
This document discusses the history and development of container networking and service discovery solutions. It describes how Mesosphere developed DC/OS to provide networking features like load balancing and service discovery using Erlang microservices including Spartan, Minuteman, and Lashup. Spartan provides high availability DNS, Minuteman provides distributed load balancing, and Lashup uses HyParView to maintain global network state across the cluster. The document outlines how these services were developed to enable dynamic container networking and service discovery.
FunctionalConf '16 Robert Virding Erlang EcosystemRobert Virding
The document discusses the background and origins of the Erlang ecosystem. It describes how Erlang was originally developed at Ericsson to address the challenges of programming telephone switching systems, which required handling a large number of concurrent processes, distributed systems, continuous operation, and fault tolerance. It outlines the principles that guided the design of Erlang, including lightweight concurrency, asynchronous messaging, and error handling through process supervision. Finally, it discusses how the Erlang ecosystem has expanded through additional languages like Elixir and LFE that maintain Erlang's principles, as well as integrations with other languages like Lua.
This document provides an overview of Elixir and functional programming presented by Anton Mishchuk at an Elixir meetup. It discusses the software crisis driven by increasing hardware capabilities, the resurgence of functional programming like Erlang to address this, and how Elixir builds upon Erlang with a Ruby-like syntax and metaprogramming capabilities. While sharing a "programmer happiness" philosophy with Ruby, Elixir differs in being functional and using Erlang's actor model for concurrency over mutable state and threads. The presentation encourages trying Elixir to build software for modern multicore systems and IoT given these advantages.
Repeating History...On Purpose...with ElixirBarry Jones
A dive into the highlights of Elixir that make it the ideal platform for the web...and how all these questions were answered figured out 30 years ago. Presented to Upstate Elixir in Greenville, SC on Nov 16.
"RabbitMQ: Message queuing that works" by Gabriele Santomaggio.
In this talk I will show how to use RabbitMQ for two different applications. For the first one, how to setup a simple messaging application using different clients as Java, Java-Script, Python, Erlang, .Net, etc. For the second one, how to distribute long batch processes between more consumers in high-availability configuration. In both examples, we will see how to scale the application in terms of clients and brokers, we will see a few standard configurations for local cluster and geographics one. At the end of this talk, you will have walked through all the basics of RabbitMQ.
Work common problems out via a functional approach. Think in terms of functions, be lazy, use pipelines and handle concurrency with immutable data borrowing concepts from Haskell/Scala/Erlang. All of this in Python.
PHP Mega Meetup, Sep, 2020, Anti patterns in phpAhmed Abdou
PHP is one of the easiest programming languages to use ever and powers more than half of the internet.
With this ease of use, certain common patterns emerge that become harmful. This is especially true when your product or service is not expected to die soon. Some anti-patterns are coding, others are related to operating your service, especially with new docker stacks. We will go over some of the most common pitfalls with a focus on enterprise development.
The document discusses programming concepts including programming languages, switch case statements, and looping. It provides examples of how to write code using switch case statements and different types of loops (for, while, do-while). The examples demonstrate how to get user input, perform calculations, and repeat blocks of code multiple times.
Switch case and looping statements are programming constructs that allow for conditional execution of code blocks. A switch case statement allows a variable's value to control program flow via multiple branches, improving clarity over long if statements. Looping statements like while and for loops execute a block of code repeatedly, allowing programs to perform repetitive tasks or process collections of data by running the same code multiple times. These programming structures are essential for many applications to make complex decisions and process large amounts of information.
Dégraissons le mammouth ou Darwin a encore frappé - La théorie de l'évolution...Arnauld Loyer
Dégraissons le mammouth ou Darwin a encore frappé
La théorie de l'évolution appliquée au développement informatique - cas pratique de l'architecture du site PMU.fr
Depuis 1980, Lehman nous avertit: un programme doit évoluer ou péricliter, mais alors qu'il devient de plus en plus gros, la complexité résultante tend à limiter son évolution. Comment remédier à cela? Quelle architecture adopter pour un site à fort trafic comme celui du PMU?
Après avoir abordé les problématiques d'évolution et de maintenance d'une application monolithique, nous verrons pourquoi et surtout comment séparer les composants et les comportements de notre application.
Du monolithe aux micro services, du distribué, des messages, du publish/subscribe, du REST, une approche polyglotte, ... au cours de cet exposé, nous verrons quelques uns des choix retenus pour garantir la survie et l'évolution de notre application. Nous verrons comment nous avons construit un socle solide permettant de répondre aux nouvelles manières de faire du Web, d'être adapté aux applications mobiles et aux télés connectées. Ce sera l'occasion d'aborder aussi bien les principes architecturaux et les principes organisationnels qui nous ont permis d'atteindre cet objectif.
Fundamentals of programming final santosAbie Santos
This document discusses programming concepts like switch case statements, looping statements, and if-else statements. It provides examples of code and explanations of how to write and run different statements in C++. It also includes links to external websites that contain additional examples of programming concepts and how they appear when compiled and executed.
This document discusses programming concepts like switch case statements, looping statements, and programming languages. It provides examples of different types of control structures like if-else statements, for loops, while loops, and do-while loops. It also gives examples of how to write code using these structures and control flows in programming languages like C++.
The document discusses different types of looping statements in programming languages. It describes while, do-while, for, and foreach loops. It explains that while and do-while loops check a condition each iteration, for loops allow initializing and changing a counter, and foreach loops iterate over array elements. It also mentions break and continue statements that alter loop execution flow. Examples of each looping statement are provided with links to online code samples.
Red Team operations require substantial efforts to both create implants and a resilient C2 infrastructure. SiestaTime aims to merge these ideas into a tool with an easy-to-use GUI, which facilitates implant and infrastructure automation alongside its actors reporting.
SiestaTime allows operators to provide registrar, SaaS and VPS credentials in order to deploy a resilient and ready to use Red Team infrastructure. The generated implants will blend-in as legitimate traffic by communicating to the infrastructure using SaaS channels and/or common network methods.
Use your VPS/Domains battery to deploy staging servers and inject your favorite shellcode for interactive sessions, clone sites and hide your implants ready to be downloaded, deploy more redirectors if needed. All this jobs/interactions will be saved and reported to help the team members with documentation process.
SiestaTime is built entirely in Golang, with the ability to generate Implants for multiple platforms, interact with different OS resources, and perform efficient C2 communications. Terraform used to deploy/destroy different Infrastructure.
This will help increase companies red teams efficiency, improving industry security standards and make the defenders to catch-up , being ready for real threats.
The document discusses switch case statements and looping in programming. It provides examples of switch case statements that check the value of a variable and execute different code blocks depending on the value. It also discusses the different types of loops - for, while, and do-while loops - and provides examples of each. The examples demonstrate how to use switch cases and loops to repeat blocks of code or make decisions based on variable values.
The document discusses joining the nanite cloud by installing Erlang via a provided script. It provides an overview of Erlang and Ruby, describing their pros and cons. The document proposes combining the strengths of Erlang and Ruby via Nanite, which uses Erlang for messaging, networking, scalability, and distribution, and Ruby for rapid development and ease of use. It describes Nanite's use of RabbitMQ, AMQP, and state storage. Security methods are briefly outlined, and future directions are proposed, including directly integrating Ruby as an Erlang node.
A talk to introduce Singularity Registry HPC, which allows you to install Singularity, Podman, or Docker containers (and others) as modules on an HPC system (e.g., LMOD or environment modules). Presented 2021.
The document provides an overview of switch case and looping statements in programming. It discusses key concepts like if/else statements, switch cases, while loops, for loops, foreach loops, and break and continue statements. Examples are given for each concept along with links to online resources that demonstrate them. The document is intended to teach fundamental programming concepts for educational purposes.
The document discusses the pros and cons of using open source software in projects. It notes that open source code comes with various licenses like GPL, BSD, and MIT that differ in how they can be used. While open source lacks perceived accountability, it actually has governance structures. The document aims to provide context on open source licensing and use within the scope of incorporating external code into a project. It addresses common concerns about open source like licensing complexity but clarifies the intent is to protect author rights, not hinder code use.
In this talk, I'll explain how the current web applications are no more simple DB projections but more complex and distributed applications that require technologies and architectures different from the past. We start with a simple & classic application to move to a more complex and feature-rich application that need a more advanced architecture.
This presentation has been presented at the Flex User Group in Berlin [1] on July 5th, 2012. I basically tried to cover the current state of Apache Flex, its possible future role in 2050 and compared Apache Flex with other Web technologies. I also tried to summarise my current work at Apache Flex. Hopefully, you'll find this presentation inspiring, too ;)
[1] http://www.flash-kiez.de
In 2015, while I was on my first year in WebMD, one of the main problems our team was facing was the amount of time needed to perform acceptance testing for new content and features. At the time, our QA team was using Java based scripts that typically took weeks to develop, hours to execute, and didn’t output a final report that could measure the results for each scenario.
In this talk, I’ll share how we combined open source tools, such as Codeception, Selenium, and Docker, and services like BrowserStack to automate our testing process, and how we reduced the time needed to develop testing code from 80 hours to 2 hours.
XOOPS 2.5.x Debugging with FirePHP/FireBugxoopsproject
FirePHP is a Mozilla Firefox plugin/extension that merges with Firebug and enables you to log to your Firebug Console using a simple PHP method call. All data is sent via response headers and will not interfere with the content on your page, therefore it is ideally suited for AJAX development where clean JSON and XML responses are required.
In this tutorial we'll show you how to use FirePHP/FireBug to debug XOOPS.
Kubernetes is Hard! Lessons Learned Taking Our Apps to Kubernetes by Eldad AssisAgileSparks
This document contains slides from a presentation on lessons learned from taking applications to Kubernetes. It discusses why the speaker's company wanted to move to Kubernetes, including needing a continuous integration/continuous delivery (CI/CD) pipeline for branches and better utilizing resources. It covers ensuring applications are ready for Kubernetes by following best practices around logging, data persistence, and handling signals. The document provides tips on starting small with Kubernetes, using Helm to manage application lifecycles, and setting proper resource limits and health probes.
Similar to in-memory capacity planning, Erlang Factory London 09 (20)
Nunit vs XUnit vs MSTest Differences Between These Unit Testing Frameworks.pdfflufftailshop
When it comes to unit testing in the .NET ecosystem, developers have a wide range of options available. Among the most popular choices are NUnit, XUnit, and MSTest. These unit testing frameworks provide essential tools and features to help ensure the quality and reliability of code. However, understanding the differences between these frameworks is crucial for selecting the most suitable one for your projects.
This presentation provides valuable insights into effective cost-saving techniques on AWS. Learn how to optimize your AWS resources by rightsizing, increasing elasticity, picking the right storage class, and choosing the best pricing model. Additionally, discover essential governance mechanisms to ensure continuous cost efficiency. Whether you are new to AWS or an experienced user, this presentation provides clear and practical tips to help you reduce your cloud costs and get the most out of your budget.
Trusted Execution Environment for Decentralized Process MiningLucaBarbaro3
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
How to Interpret Trends in the Kalyan Rajdhani Mix Chart.pdfChart Kalyan
A Mix Chart displays historical data of numbers in a graphical or tabular form. The Kalyan Rajdhani Mix Chart specifically shows the results of a sequence of numbers over different periods.
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
Have you ever been confused by the myriad of choices offered by AWS for hosting a website or an API?
Lambda, Elastic Beanstalk, Lightsail, Amplify, S3 (and more!) can each host websites + APIs. But which one should we choose?
Which one is cheapest? Which one is fastest? Which one will scale to meet our needs?
Join me in this session as we dive into each AWS hosting service to determine which one is best for your scenario and explain why!
Dive into the realm of operating systems (OS) with Pravash Chandra Das, a seasoned Digital Forensic Analyst, as your guide. 🚀 This comprehensive presentation illuminates the core concepts, types, and evolution of OS, essential for understanding modern computing landscapes.
Beginning with the foundational definition, Das clarifies the pivotal role of OS as system software orchestrating hardware resources, software applications, and user interactions. Through succinct descriptions, he delineates the diverse types of OS, from single-user, single-task environments like early MS-DOS iterations, to multi-user, multi-tasking systems exemplified by modern Linux distributions.
Crucial components like the kernel and shell are dissected, highlighting their indispensable functions in resource management and user interface interaction. Das elucidates how the kernel acts as the central nervous system, orchestrating process scheduling, memory allocation, and device management. Meanwhile, the shell serves as the gateway for user commands, bridging the gap between human input and machine execution. 💻
The narrative then shifts to a captivating exploration of prominent desktop OSs, Windows, macOS, and Linux. Windows, with its globally ubiquitous presence and user-friendly interface, emerges as a cornerstone in personal computing history. macOS, lauded for its sleek design and seamless integration with Apple's ecosystem, stands as a beacon of stability and creativity. Linux, an open-source marvel, offers unparalleled flexibility and security, revolutionizing the computing landscape. 🖥️
Moving to the realm of mobile devices, Das unravels the dominance of Android and iOS. Android's open-source ethos fosters a vibrant ecosystem of customization and innovation, while iOS boasts a seamless user experience and robust security infrastructure. Meanwhile, discontinued platforms like Symbian and Palm OS evoke nostalgia for their pioneering roles in the smartphone revolution.
The journey concludes with a reflection on the ever-evolving landscape of OS, underscored by the emergence of real-time operating systems (RTOS) and the persistent quest for innovation and efficiency. As technology continues to shape our world, understanding the foundations and evolution of operating systems remains paramount. Join Pravash Chandra Das on this illuminating journey through the heart of computing. 🌟
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on automated letter generation for Bonterra Impact Management using Google Workspace or Microsoft 365.
Interested in deploying letter generation automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
A Comprehensive Guide to DeFi Development Services in 2024Intelisync
DeFi represents a paradigm shift in the financial industry. Instead of relying on traditional, centralized institutions like banks, DeFi leverages blockchain technology to create a decentralized network of financial services. This means that financial transactions can occur directly between parties, without intermediaries, using smart contracts on platforms like Ethereum.
In 2024, we are witnessing an explosion of new DeFi projects and protocols, each pushing the boundaries of what’s possible in finance.
In summary, DeFi in 2024 is not just a trend; it’s a revolution that democratizes finance, enhances security and transparency, and fosters continuous innovation. As we proceed through this presentation, we'll explore the various components and services of DeFi in detail, shedding light on how they are transforming the financial landscape.
At Intelisync, we specialize in providing comprehensive DeFi development services tailored to meet the unique needs of our clients. From smart contract development to dApp creation and security audits, we ensure that your DeFi project is built with innovation, security, and scalability in mind. Trust Intelisync to guide you through the intricate landscape of decentralized finance and unlock the full potential of blockchain technology.
Ready to take your DeFi project to the next level? Partner with Intelisync for expert DeFi development services today!
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
in-memory capacity planning, Erlang Factory London 09
1. erlang at hover.in
Bhasker V Kode
cofounder & CTO at hover.in
at Erlang Factory, London
June 26, 2009
http://developers.hover.in
2. brief introduction to hover.in
choose words from your blog, & decide what content / ad
you want when you hover* over it
* or other events like click,right click,etc
or...
the worlds first publisher driven
intext content & ad delivery platform...
or
lets web publishers push clientside event handling to the
cloud, to run various rich applications called hoverlets
demo at http://start.hover.in/ and http://hover.in/demo
more at http://hover.in , http://developers.hover.in/blog/
http://developers.hover.in
4. ➔
hover.in founded late 2007
http://developers.hover.in
5. ➔
hover.in founded late 2007
➔
the web ~ 10 20 years old
http://developers.hover.in
6. ➔
hover.in founded late 2007
➔
the web ~ 10 20 years old
➔
humans 100's of thousands of years
http://developers.hover.in
7. ➔
hover.in founded late 2007
➔
the web ~ 10 20 years old
➔
humans 100's of thousands of years
➔
but bacteria.... around for millions of years
... so this talk is going to be about what we can
learn from bacteria, the brain, and memory in
a concurrent world followed by hover.in's erlang
setup and lessons learnt
http://developers.hover.in
9. should_we_glow(Ctr, Max, Acc)>
receive
{in,Cell}>
One_or_zero = should_i_glow(Cell),
case Ctr of
Max > done;
_ >
case Acc of
?SOME_VAL >
glow_for_8_hours(), done;
_ >
NewAcc = One_or_zero + Acc,
should_we_glow(Ctr+1,Max,NewAcc)
end;
_ > error
end http://developers.hover.in
10. some traits of bacteria
●
each bacteria cell spawns its own proteins
●
All bacteria have some sort of some presence
& replies associated, (asynchronous comm.)
●
group dynamics exhibits 'list fold' ish operation
●
only when the Accumulator is > some guard
clause, will groupdynamics of making light
(bioluminiscence) work (eg: in deep sea)
http://developers.hover.in
11. spawning, in practice
●
for a single google search result, the same
requests are sent to multiple machines( ~1000
as of 09), which ever replies the quickest wins.
●
in amazon's dynamo architecture that powers
S3, use a (3,2,2) rule . ie Maintain 3 copies of
the same data, reads/writes are succesful only
when 2 concurrent requests succeed. This ratio
varies based on SLA, internal vs public service.
( more on conflict resolution... )
http://developers.hover.in
12. pattern matching behaviour
●
each molecule connects to its specific receptor
protein to complete the missing piece,to trigger
the group behaviour that are only succesful
when all of the cells participate in unison.
●
Type = case UserType of
user > true;
admin > true;
_Else > false
end
http://developers.hover.in
13. supervisors, workers
●
as bacteria grow, they split into two. when
muscle tears, it knows exactly what to replace.
●
erlang supervisors can decide restart policies: if
one worker fails, restart all .... or if one worker
fails, restart just that worker, more tweaks.
●
can spawn multiple workers on the fly, much
like the need for launching a new ec2 instant
http://developers.hover.in
14. interspecies communication
●
if you look at your skin – consists of very many
different species, but all bacteria found to
communicate using one common chemical
language.
http://developers.hover.in
15. interspecies communication
●
if you look at your skin – consists of very many
different species, but all bacteria found to
communicate using one common chemical
language.
hmmmmmmmmmmmmmmmmmmm..............
....serialization ?!
....a common protein interpretor ?!
....or perhaps justintime protein compilation?!
http://developers.hover.in
16. interspecies comm. in practice
➔
attempts at serialization , cross language
communication include:
➔
thrift ( by facebook)
➔
protocol buffers ( by google)
➔
en/decoding , port based communication ( erlang<
>python at hover.in )
➔
rabbitMQ shows speeds of several thousands of
msgs/sec between python <> erlang (by using...?)
http://developers.hover.in
17. talking about scaling
The brain of the worker honeybee weighs about
1mg ( ~ 950,000 neurons )
●
Flies acrobatically , recognizes patterns,
navigates , communicates, etc
●
Energy consumption: 10−15 J/op, at least 106
more efficient than digital silicon neurons
http://developers.hover.in
18. the human brain
●
100 billion neurons, stores ~100 TB
●
Differential analysis e.g., we compute color
●
Multiple inputs: sight, sound, taste, smell, touch
●
Facial recognition subcircuits, peripheral vision
●
in essence the left & right brain vary in:
left > persistent disk , handles past/future
right > temporal caches! , handles present
http://developers.hover.in
19. inmemory is the new embedded
●
servers as of '09 typically have 4 16 GB RAM
●
stats of how companies are adding nodes
http://developers.hover.in
20. inmemory is the new embedded
●
caching systems avoid disk/db makes sense
●
caching systems for processing tasks makes
sense
●
but ....
●
keeping your entire data inmemory by having N
number of nodes ?
http://developers.hover.in
21. inmemory is the new embedded
●
keeping your entire data inmemory by having N
number of nodes , ( where N = total data in gb /
max ram per node ) is like ...
– building a billion dollar company with 999 milion
dollars of funding!
or
– having only a right brain !
●
surely we can do better than that!
http://developers.hover.in
22. inmemory capacity planning
●
No matter how many machines you have, and
how many cores, in production level – your
product could be defined by how well you
design your inmemory / RAM strategies.
●
alternatives to avoid swapping could be – just
leaving results partioned on diff nodes, or
additional tasks to reduce the dataload further
until they can fit in memory
http://developers.hover.in
23. inmemory capacity planning
●
parallizing jobs inmemory is a lot of fun...
●
but...
●
more often bottleneck will not be how well you
can paralliize, but how much you need to
parallize so that memory does'nt swap (eg: || db
reads)
http://developers.hover.in
24. (1)#1 shard thy data to make it sufficiently unrelated
●
typical web backends – all user data in one
table – then clustering just splits that on artibary
basis. eg: query user table where id=user1,
●
what if you have N concurrent process's
accessing N diff user tables – no locks, you can
||'ze & results can come back asynchronously
since sufficiently unrelated.
●
Warning: but more atoms ( list_to_atom atoms
aren't garbage collected ) http://developers.hover.in
25. (1)#2 implementing flowcontrol
●
great to handle both bursts or silent traffic & to
determine bottlenecks.(eg ur own,rabbitmq,etc )
●
eg1: when we addjobs to the queue, if it takes
greater than X consistently we move it to high
traffic bracket, do things differently, possibly
add workers or ignore based on the task.
●
eg2: amazon shopping carts, are known to be
extra resilient to write failures, (dont mind
multiple versions of them over time) http://developers.hover.in
26. (1)#3 all data is important, but some less important
●
priority queue used to built heatseeking algo
( priority to crawl webpages that get more hits
rather than depthfirst or breadthfirst)
●
can configure max number of buckets
●
can configure max number of urls per bucket
●
can configure pyramid like queue. ( moving
from lower buckets to higher is easier than
moving from high to higher )
http://developers.hover.in
28. erlang in a crawler architecture ?
●
each time a hit repeats for a URL , it moves from
bucket N to bucket N+1
●
crawls happen from top down (priority queue)
http://developers.hover.in
29. erlang in a crawler architecture ?
●
each time a hit repeats for a URL , it moves from
bucket N to bucket N+1
●
crawls happen from top down (priority queue)
●
so the bucket is locked, so that locked urls dont keep
move up anymore
http://developers.hover.in
30. erlang in a crawler architecture ?
●
each time a hit repeats for a URL , it moves from
bucket N to bucket N+1
●
crawls happen from top down (priority queue)
●
so the bucket is locked, so that locked urls dont keep
move up anymore
●
each user/site has their own priority queues, which
keep shifting roundrobin after every X urls crawled
per user/site
http://developers.hover.in
31. erlang in a crawler architecture ?
●
each time a hit repeats for a URL , it moves from
bucket N to bucket N+1
●
crawls happen from top down (priority queue)
●
so the bucket is locked, so that locked urls dont keep
move up anymore
●
each user/site has their own priority queues, which
keep shifting roundrobin after every X urls crawled
per user/site
●
python crawler leaves text files which dirty loaded into
fragmented mnesia
http://developers.hover.in
32. (1)#3 time spent x RAM utilization = a constant
(2) eg: of || db reads
(3)
(4)#4 before every succesful persistent write & after
every succesful persistent read is an inmemory one
(5) eg: hi_cache_worker's used to build
(6) most recent queue's
http://developers.hover.in
33. (1)#5 before every succesful persistent write & after
every succesful persistent read is an inmemory one
●
you listen to a phone number in batch's of 3 or 4
digits. the part that absorbs just before writing
(temporal), until you write into your contact book
or memorize it ( persistent)
●
eg: if LRU cache exists inmemory, like 100
most recent url's or tags, then no need to parse
server logs for computation, try during writes
itself . No logs, no files. live buzz analytics!
http://developers.hover.in
34. (1)#6 know thy RAM, trial/error to find ideal dataload
●
eg: || db reads if || happens so fast, mem probs
●
replication vs location transparency, are they
fragmented, are some nodes readonly ? (rpc...)
●
need metadata for which node to acess for user1,
(or use hashing fn like memcache)
●
are tables inmemory (right brain ), cached from
disk , or on disk alone ( left brain )
●
fortunately mnesia allows highly granular choices
http://developers.hover.in
35. (1)#7 what cannot be measured cannot be improved
●
you can't improve what you can't measure. an
investment in debugging utilities is a good
investment
●
looking forward to debugging with dtrace,gproc
etc but until then – just a set/get away!
●
using tsung (written in erlang again ) – load
performance testing tool, for simulating 100's of
concurrent users/requests , and great for
analysing bottlenecks of your system ,CDN's ) ,
http://developers.hover.in
36. hi_cache_worker
●
a circular queue implemented via gen_server
●
set ( ID , Key , Value , OptionsList)
Options are {purge, <true| false>}
{ size , <integer> }
{ set_callback , <Function> }
{ delete_callback , <Function> }
{ get_callback , <Function> }
{ timeout, <int>, <Function> }
ID is usually a siteid or “global”
http://developers.hover.in
37. ●
C = hi_cache_worker,
C:set ( User1, “recent_saved” , Value)
C:set ( “global”, “recent_hits” , Value
[{size,1000}] )
C:get (“global”,”recent_voted”)
C:get (User1,”recenthits”)
C:get (User1,”recent_cron_times”)
●
( Note: initially used in debugging internally >
then reporting > next in public community stats)
http://developers.hover.in
38. 7 rules of inmemory capacity planning
(1) shard thy data to make it sufficiently unrelated
(2) implementing flowcontrol
(3) all data is important, but some less important
(4) time spent x RAM utilization = a constant
(5) before every succesful persistent write & after
every succesful persistent read is an inmemory one
(6) know thy RAM, trial/error to find ideal dataload
(7) what cannot be measured cannot be improved
http://developers.hover.in
39. summary of erlang at hover.in
●
LYME stack since ~dec 07 , 3 nodes (64bit 4gb )
●
python crawler, associated NLP parsers, cpu time
splicing algo's for cron's app, configurable priority
queue's for heatseeking algo's app, flowcontrol
app , caching app , pagination app for memoizing
●
remote node debugger, cyclic queue workers, lru
cache workers , headlessfirefox for thumbnails
●
touched 1 million hovers/month in May'09 after
launching closed beta to publishers in Jan 09
http://developers.hover.in
41. references
●
http://developers.hover.in
●
http://erlang.org
●
http://memcached.org , http://rabbitmq.org/ http://highscalability.com/
●
http://www.allthingsdistributed.com/files/amazondynamososp2007.pdf
●
shoutout to everyone at #erlang !
●
amazing brainrelated talks at http://ted.com ,
●
go read more about the brain and hack on
erlang NOW!
http://developers.hover.in