Delivered by Bhasker V Kode at foss.in/2009
Official talk page at http://foss.in/2009/schedules/talkdetailspub.php?talkid=17
Erlang 's support for handling binaries and pattern matching make it a great choice for parsing everything from IPv4 packets, to payloads from the Memcached protocol, SWF files, or databases like Tokyo Cabinet. From a functional programming perspective, there are various ways of building these parsers, taking advantage of the concurrent and recursive nature that is inherent to the language and other challenges which have been gathered while validating the storage & retrieval options for our distributed crawler, and submitting patches to projects like Medici & Tora ( erlang based Tokyo Cabinet clients). The talk will also touch upon Tokyo cabinet's support for mapreduce with Lua, and notes from building your own custom formats & our internal mapreduce'esque and caching frameworks used in building a multi-million impression platform utilizing under a gig of RAM per node.
Notes on:
- trends in disk/memory/bandwidth
- why erlang, RAM, binaries
- garbage collection in the erlang VM
- message passing
- use-cases
Bhasker V Kode , (Co-Founder & CTO - Hover.in ) talking about erlang + engineering efforts at the Commercial Users of Functional Programming 2009, Edinburgh
when erlang makes sense. this talk tries to draw simple metaphors from what we can learn from bacteria ,the brain ,memory & concurrency .
the talk was presented at devcamp bangalore, by bhasker v kode, co-founder & CTO at hover.in
also check out the hover.in devblog at http://developers.hover.in
Package manages and Puppet - PuppetConf 2015ice799
This talk will begin by explaining what a package manager is and how package managers work, at a high level. Next, we'll observe the common patterns seen on the internet of compiling software in a Puppet manifest and discuss why this not ideal. This talk will conclude by showing how you can add package repositories to your infrastructure using Puppet and what settings are important for ensuring secure access to remote package repositories.
Bhasker V Kode , (Co-Founder & CTO - Hover.in ) talking about erlang + engineering efforts at the Commercial Users of Functional Programming 2009, Edinburgh
when erlang makes sense. this talk tries to draw simple metaphors from what we can learn from bacteria ,the brain ,memory & concurrency .
the talk was presented at devcamp bangalore, by bhasker v kode, co-founder & CTO at hover.in
also check out the hover.in devblog at http://developers.hover.in
Package manages and Puppet - PuppetConf 2015ice799
This talk will begin by explaining what a package manager is and how package managers work, at a high level. Next, we'll observe the common patterns seen on the internet of compiling software in a Puppet manifest and discuss why this not ideal. This talk will conclude by showing how you can add package repositories to your infrastructure using Puppet and what settings are important for ensuring secure access to remote package repositories.
In the world of social gaming, the classic 2-tier of web application does not cut it anymore. We need new and better solutions.
Follow along the evolution of game servers at Wooga and get an in-depth look into the next-generation backend putting the combined forces of Erlang and Ruby to work. Learn how scalability, reliability, concurrency control and beautiful code do not need to be mutually exclusive.
Every day, sysadmins find the need to build tools that get the job done in a fast and efficient way. Go is a new and powerful—yet simple—language that is an excellent fit for the needs of the harried sysadmin. This tutorial provides an introduction to Go with a focus on using it for everday tooling.
Topics include:
Understanding of the applicability of Go to everyday work
The Go environment and toolset
Go language fundamentals:
Control Structures
Functions
Data Types
Methods and Interfaces
Goroutines
Channels
Examples in Go:
Email
Web Server
Directory Tools
JRuby is a great way to use native Java libraries and get around the project overhead of Java, but how do you actually use Java from JRuby? This talk explores building a JRuby application, backed by the portable Java version of Couchbase Mobile.
While JRuby is build to interface with Java, when calling out to JVM land there are all those little hurdles to overcome. Handling Strings correctly, using native Collection types, and interfacing with libraries which expect those native types instead of the ones provided by JRuby, and last but not least Implementing native interfaces to pass around.
Oh and by the way all of this runs on a little Raspberry Pi!
An overview of Spacebrew, introducing the basic inner-workings, libraries involved, and its capabilities.
Usually accompanied by me, hence some of the "let's try..." language!
Ice Age melting down: Intel features considered usefull!Peter Hlavaty
Decades history of kernel exploitation, however still most used techniques are such as ROP. Software based approaches comes finally challenge this technique, one more successful than the others. Those approaches usually trying to solve far more than ROP only problem, and need to handle not only security but almost more importantly performance issues. Another common attacker vector for redirecting control flow is stack what comes from design of today’s architectures, and once again some software approaches lately tackling this as well. Although this software based methods are piece of nice work and effective to big extent, new game changing approach seems coming to the light. Methodology closing this attack vector coming right from hardware - intel. We will compare this way to its software alternatives, how one interleaving another and how they can benefit from each other to challenge attacker by breaking his most fundamental technologies. However same time we go further, to challenge those approaches and show that even with those technologies in place attackers is not yet in the corner.
In the world of social gaming, the classic 2-tier of web application does not cut it anymore. We need new and better solutions.
Follow along the evolution of game servers at Wooga and get an in-depth look into the next-generation backend putting the combined forces of Erlang and Ruby to work. Learn how scalability, reliability, concurrency control and beautiful code do not need to be mutually exclusive.
Every day, sysadmins find the need to build tools that get the job done in a fast and efficient way. Go is a new and powerful—yet simple—language that is an excellent fit for the needs of the harried sysadmin. This tutorial provides an introduction to Go with a focus on using it for everday tooling.
Topics include:
Understanding of the applicability of Go to everyday work
The Go environment and toolset
Go language fundamentals:
Control Structures
Functions
Data Types
Methods and Interfaces
Goroutines
Channels
Examples in Go:
Email
Web Server
Directory Tools
JRuby is a great way to use native Java libraries and get around the project overhead of Java, but how do you actually use Java from JRuby? This talk explores building a JRuby application, backed by the portable Java version of Couchbase Mobile.
While JRuby is build to interface with Java, when calling out to JVM land there are all those little hurdles to overcome. Handling Strings correctly, using native Collection types, and interfacing with libraries which expect those native types instead of the ones provided by JRuby, and last but not least Implementing native interfaces to pass around.
Oh and by the way all of this runs on a little Raspberry Pi!
An overview of Spacebrew, introducing the basic inner-workings, libraries involved, and its capabilities.
Usually accompanied by me, hence some of the "let's try..." language!
Ice Age melting down: Intel features considered usefull!Peter Hlavaty
Decades history of kernel exploitation, however still most used techniques are such as ROP. Software based approaches comes finally challenge this technique, one more successful than the others. Those approaches usually trying to solve far more than ROP only problem, and need to handle not only security but almost more importantly performance issues. Another common attacker vector for redirecting control flow is stack what comes from design of today’s architectures, and once again some software approaches lately tackling this as well. Although this software based methods are piece of nice work and effective to big extent, new game changing approach seems coming to the light. Methodology closing this attack vector coming right from hardware - intel. We will compare this way to its software alternatives, how one interleaving another and how they can benefit from each other to challenge attacker by breaking his most fundamental technologies. However same time we go further, to challenge those approaches and show that even with those technologies in place attackers is not yet in the corner.
QCON SP 2016 - Elixir: Tolerância a Falhas para AdultosFabio Akita
Elixir é uma das novas linguagens que está despontando principalmente pela reputação de capacidade de altíssima concorrência e paralelismo que sua VM permite. Mas o principal elemento é que Erlang e por consequência, Elixir, é uma das poucas linguagens criadas assumindo que vamos cometer erros. Ele garante alta disponibilidade nos dando primitivas que permitam que nosso software continue funcionando, mesmo que às vezes ele falhe. E agora vamos ver como Erlang faz isso como Elixir expõe esse potencial a todos nós.
There Are Literally Thousands of Erlang ProjectsPierre Fenoll
Present state of erldocs.com and its future: API search engine, erldocs skins & possibility to run jobs on every Erlang projects, all for the benefit of the larger Erlang community
High Performance Erlang - Pitfalls and SolutionsYinghai Lu
Presented at Erlang Factory 2016, San Francisco, CA.
Erlang is widely used for building concurrent applications. However, when we push the performance of our Erlang based application to handle millions of concurrent clients, some Erlang scalability issues begin to show and some conventional programming paradigm of Erlang no longer hold. We would like to share some of these issue and how we address them. In addition, we share some of our experience on how to profile an Erlang application to identify bottlenecks.
We will take a deep look at some of the basic mechanisms of Erlang and show how they behave under high load and parallelism, which includes message delivery, process management and shared data structures such as maps and ETS tables. We will demonstrate their limitations and propose techniques to alleviate the issues.
We will also share profiling techniques on how to find those bottlenecks in Erlang applications across different levels. We will share techniques for writing highly performant Erlang applications.
Elixir - Easy fun for busy developers @ Devoxx 2016David Schmitz
Did you ever want to create an application that is never down?
Have you ever been jealous of those Erlang guys, that produce applications that practically never fail?
Have you tried Erlang and fled because of its baroque syntax and tooling?
Enter Elixir!
Elixir combines Ruby's love for programmers with the absolute power that is the Erlang platform.
This talk will introduce Java programmers to Elixir and OTP. You will gain an understanding of what Elixir brings to the table and how to build highly scale-able systems with a toolset that is actually fun to use.
Even if you will never use Elixir and OTP for you projects, you will learn some of the concepts, that make Erlang into such a powerful ecosystem.
Erlang supervision trees helps bring fault tolerance, recovery and robustness to erlang applications. The presentation goes through basic and advanced examples along with supervision strategies.
Repeating History...On Purpose...with ElixirBarry Jones
A dive into the highlights of Elixir that make it the ideal platform for the web...and how all these questions were answered figured out 30 years ago. Presented to Upstate Elixir in Greenville, SC on Nov 16.
"RabbitMQ: Message queuing that works" by Gabriele Santomaggio.
In this talk I will show how to use RabbitMQ for two different applications. For the first one, how to setup a simple messaging application using different clients as Java, Java-Script, Python, Erlang, .Net, etc. For the second one, how to distribute long batch processes between more consumers in high-availability configuration. In both examples, we will see how to scale the application in terms of clients and brokers, we will see a few standard configurations for local cluster and geographics one. At the end of this talk, you will have walked through all the basics of RabbitMQ.
Work common problems out via a functional approach. Think in terms of functions, be lazy, use pipelines and handle concurrency with immutable data borrowing concepts from Haskell/Scala/Erlang. All of this in Python.
The Developers Conference - São Paulo 2016
Talk: Erlang sem enrolação!
O que faz o Whatsapp ter 2 milhões de conexões abertas em uma única máquina? Ou o Call of Duty ter 50k/s de requisições? Nesta talk mostrarei uma introdução avançada à linguagem Erlang falando, sem enrolação, sobre os primeiros passos para lidar com a linguagem ressaltando as suas principais características funcionais e também um pouco sobre a OTP.
What does WhatsApp have 2 million open connections on a single machine? Or Call of Duty have 50k/s requests? This talk will show an advanced introduction to Erlang speaking without stalling on the first steps to deal with the language emphasizing its main functional features and also a little about the OTP.
Exemplo / Example:
https://gist.github.com/fmamud/ae4b7ad572abcac2fc3e6d3ca670f6a6
An update of my Perl Myths talk (for http://ossbarcamp.com in Dublin, Ireland, September 2009). It covers jobs, cpan, community, best practices, power tools, and perl 6.
Nerd sniping myself into a rabbit hole... Streaming online audio to a Sonos s...Maarten Balliauw
After buying a set of Sonos-compatible speakers at IKEA, I was disappointed there's no support for playing audio from a popular video streaming service. They stream Internet radio, podcasts and what not. Well, not that service I want it to play!
Determined - and not knowing how deep the rabbit hole would be - I ventured on a trip that included network sniffing on my access point, learning about UPnP and running a web server on my phone (without knowing how to write anything Android), learning how MP4 audio is packaged (and has to be re-packaged). This ultimately resulted in an Android app for personal use, which does what I initially wanted: play audio from that popular video streaming service on Sonos.
Join me for this story about an adventure that has no practical use, probably violates Terms of Service, but was fun to build!
Apache Bigtop has created the de-facto standard in how Hadoop-based stacks are developed, delivered, and managed. We are at it again! The track will present the composition of the next generation of in-memory computing stack that is completely built out of open-source components. The next generation of the Apache data processing stack will focus on in-memory and transactional processing of large amounts of data. We will also be talking about performance benefits that legacy data-processing software based on MapReduce, Hive, and similar, can derive from in-memory computing. This session will discuss and analyze the benefits of practicing Fast Data in the open.
Perl Myths 200802 with notes (OUTDATED, see 200909)Tim Bunce
Perl programming has it's share of myths. This presentation debunks a few popular ones with hard facts. Surprise yourself with the realities.
THIS VERSION IS OUTDATED. PLEASE SEE http://www.slideshare.net/Tim.Bunce/perl-myths-200909
Follow a Firefox crash from its genesis in a collapsing browser process through the dizzying array of collection, storage, and reporting systems that make up Socorro, our open-source crash collector. Enjoy war stories of weird, interlocking failures, and see how we nevertheless continue to fulfill our mandate: “Never lose a crash.” Observe some patterns that emerged from this system which can be useful in yours.
Docker in production, for real!
The Yuzu startup, helped by Vixns, chose to have a docker infrastructure with Mesos/Marathon/Consul. From the development environment to our prod monitoring, we share our mistakes, successes, workflows and tools.
Follow your nose: history frames the futureAlan Dix
Keynote at AVI 2022: Advanced Visual Interfaces, Rome, Italy, 6-10 June 2022
https://alandix.com/academic/talks/
AVI2022-keynote/
As cultures and individuals, we look back to look forward, explicitly using lessons of the past to guide our future decisions. In this 30th anniversary of the AVI conference we can also look back in order to make the next 30 years even better.
In 1996 I gave a keynote at the third AVI, both describing my own work in formal modelling of user interactions, and also pondering variations of ‘advanced visual interfaces’, imagining advanced aural and advanced nasal interfaces. This was part playful but also serious, uncovering the way our different senses give us different cuts through space and time. In particular, smell is deeply associated with memory both personal and spatial – that is history. In the meantime, scent has been used as a metaphor for information seeking, and now long-promised smell-based interfaces are beginning to emerge; we are about to enter an exciting world of multi-sensory experiences.
However, it is on the metaphoric sense of advanced visual and nasal interaction that I want to focus now. The importance of tracing what is past and planning what is to come. This is true for our discipline as a whole, but also in moment-to-moment digital interactions. On the first of April this year, Nielsen Norman Group posted an article entitled “Support Recall Instead of Recognition in UI Design”. It was meant as an April Fool’s Day joke, but in fact the move towards gesture-based touch interactions means this is precisely how many feel today. This has exacerbated the long-term weaknesses of visual interaction heuristics and guidance when it comes to looking back, making it hard to ask, “why did that happen?”, or “how did I manage that?”, especially for older users or those who are less confident with digital technology. Finally, as we move forwards to tackle these and new issues, we need to constantly question what constitutes ‘advanced’: a fast-moving highway for a few or a wider frontier for everyone. The latter often poses the hardest design challenges but is most critical in a world where being digital is central to being a citizen.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
1. “Parsing binaries and protocols
with erlang ?!”
Bhasker V Kode
cofounder & CTO at hover.in
at foss.in
December 4th, 2009
http://developers.hover.in
6. “ha! ofcourse i knew that...
err.... but people scale...
that's what they do .....
that's our way out !!!
scaling out ...
scaling up ...
auto scaling even...!!!
: O ”
foss.in/2009 http://developers.hover.in
7. “scale UP ...!
more RAM seems to stop those
stall those silly CPUunit warnings
my hosting provider gives...
bring on those infinite loops &
polling crons. RealTimeWeb FTW!”
foss.in/2009 http://developers.hover.in
8. “scaling OUT , maybe with a
distributed filesystem
and figure out a way for nodes to
talk, and... Replication... and
location transparency during
weekends... and commodity
hardware which i can't pay for ”
foss.in/2009 http://developers.hover.in
9. More data becoming archival
NOT by choice, but forced to.
Not pushed to handling streams of
data well ( even hadoop!) #bigdata
If you're not compromising, you're
not pushing enough. Disk's loss
must be some else's gain.
fixedlength eg's at fb, twitter, google
foss.in/2009 http://developers.hover.in
10. Erlang for RAM
on the web is the new
Embedded C
foss.in/2009 http://developers.hover.in
11. “THE NEWS TODAY. Once popular
retro format 'binary' continues to
go unnoticed after brief sightings
on wallpapers during the matrix
trilogy ....”
pssst! in files of any mime/content type
in db's that accept binary
in RAM, via caching engines
compact for n/w transfer & storage
the answer to unicode
foss.in/2009 http://developers.hover.in
12. “fine! Binaries are everywhere,
disk's are not keeping up, and i've
got more cores on my nodes every
year.”
foss.in/2009 http://developers.hover.in
13. “But i'm not still not going near a
strict, dynamically typed functional
programming language with
support for concurrency,
communication, and distribution,
automatic memory management &
supports multiple platforms !!!”
foss.in/2009 http://developers.hover.in
16. “ahh... so processes are pseudo
threads in the erlang VM that are
light weight & the base of erlang
programs having their own heap or
message inbox & are meant for
message passing erlang
primitaves. Also the developer can
configure how many cores are
used based on # of schedulers,
which run process's.
foss.in/2009 http://developers.hover.in
18. Let M= msgs to random users
Let N= 100,000 users
Route M msgs to right N users !
typical onenode approach :
for i to M
for j to N
if match, add_update
actor approach:
N concurrent processes listening to all msgs
As new msg arrives, msg pass to all N pids
in each concurrent process: if match, add_update
foss.in/2009 http://developers.hover.in
23. “ahh... so this is what the no
shared memory in erlang, or light
weight process's being garbage
collected easily since they dont
have references to data in each
other's process heap, & messages
copied or shared based on it's
size, likelihood of reuse and also
optimized for binary. tellmemore!!”
foss.in/2009 http://developers.hover.in
27. “Can a spawned process listen as
long as i want it to?”
“Can a spawned process stop
listening when I want it to?”
“Can a spawned process spawn
more processes?”
foss.in/2009 http://developers.hover.in
28. “So though erlang gives a library
called OTP & a db called mnesia for
making life easier you can parse
or create binaries easily, make
clientserver programs, distributed
rpc calls, tailrecursive servers,
message/priority queue's for
flowcontrol, talk to ports and other
lang's, or create any data structure
explicitly (a) inmemory (b)ondisk
of any connected node!
foss.in/2009 http://developers.hover.in
29. “show me the demo's”
● Process related
– Message queue's , Client – server
– RPC , Timeouts
● Binary
– Binary pattern matching, Parse swf/mp3 for metadata
– Networking, comm. with C, Tokyocabinet client eg.
● Process + Binary!
– Building a production ready inmemory CDN
consistently faster than Am4z0n cl0udfr0nt, in stages
open & gzip < concat js's < inmemory < streaming?
foss.in/2009 http://developers.hover.in
30. “Binary pattern matching ?”
<<Value:Size/TypeSignednessEndianism
unit:Unit>>
<<1:32>> = <<0,0,0,1>.
<<1:32/unsigned-little>> = <<1,0,0,0>.
<<_:8,“mnesia”/binary>> = <<”Amnesia”>>.
So <<Bin>> could be unicode characters
( English, hindi, tamil ) or JPG's or http headers
or basically segments of binaries
NewBinary=<<Segment1,Segment2>>.
foss.in/2009 http://developers.hover.in
31. summary of tech at hover.in
● LYME stack since ~dec 07 , 4 (1) nodes (64bit 4GB)
● python crawler + associated NLP parsers, index's now
in tokyo cabinet, inverted index's in erlang 's mnesia db
with binaries of 5 diff indian languages + multiple
contenttypes, cpu timesplicing algo's, priority queue's
for heatseeking algo, flowcontrol, caching engines,
cyclic queues, mapreduces with nonblocking gathers,
headlessfirefox for thumbnails, patches to
tokyocabinet client 'medici'
● Beta in Jan 09, 1 million hovers/month in May'09
●
24 developers + several interns across ~2 years
foss.in/2009 http://developers.hover.in