This document discusses the challenges of developing a scalable real-time chat system and proposes an architecture using frontend servers, backend servers, and Redis for data storage. The proposed architecture separates the connection handling and logic into stateless frontend and backend servers to allow for horizontal scaling. Websockets are used for real-time messaging between servers, with fallback options. Security, load balancing, monitoring, and preventing DDOS attacks are also addressed.
How we deployed ngReact as part of our AngularJS webpage and what speed improvement we saw.
Presented together with János Gyurisán at the 13th AngularJS Meetup in VIenna
Improving Perceived Page Performance with ASP.NET Web API and AJAXChris Bohatka
Javascript is quickly taking over the world. The line between client and server is becoming increasingly blurred and Microsoft is aboard the bandwagon. ASP.NET Web API gives us the ability to remove some of our service layer from the black box and begin interacting directly with the browser. Using Web API and AJAX, we can perform data operations asynchronously, improving our perceived page performance and providing a more seamless end-to-end user experience. In this session, we will walk through creating a Web Api controller, improve our confidence in working with Javascript and connect the dots between client and server.
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRFPaul Mooney
Slides from Paul Mooney's talk at the OWASP Ireland June Chapter meeting offering an overview of the Encrypted Token Pattern, and ARMOR, its .NET implementation.
How we deployed ngReact as part of our AngularJS webpage and what speed improvement we saw.
Presented together with János Gyurisán at the 13th AngularJS Meetup in VIenna
Improving Perceived Page Performance with ASP.NET Web API and AJAXChris Bohatka
Javascript is quickly taking over the world. The line between client and server is becoming increasingly blurred and Microsoft is aboard the bandwagon. ASP.NET Web API gives us the ability to remove some of our service layer from the black box and begin interacting directly with the browser. Using Web API and AJAX, we can perform data operations asynchronously, improving our perceived page performance and providing a more seamless end-to-end user experience. In this session, we will walk through creating a Web Api controller, improve our confidence in working with Javascript and connect the dots between client and server.
OWASP Ireland June Chapter Meeting - Paul Mooney on ARMOR & CSRFPaul Mooney
Slides from Paul Mooney's talk at the OWASP Ireland June Chapter meeting offering an overview of the Encrypted Token Pattern, and ARMOR, its .NET implementation.
Docker Dublin: Just What is a Service Mesh, and if I get one will it make eve...Elton Stoneman
From the Docker Dublin MeetUp, June 2019. Talking about service meshes – Linkerd and Istio in particular. It’s a technical session so there are lots of demos, but it’s grounded in the practical question – do you really need a service mesh, and is it worth the cost? You’ll learn what a service mesh can do, and how it helps to cut a lot of infrastructure concerns from your code.
Existing mobile backend services can be really useful and quick to start with, however if we'd like to have more flexibility over the backend implementation an fairly easy solution would be to build a backend based on MongoDB and Node.JS. We will then drive this backend with EPAM Systems' open source iOS ROAD framework that has a number of good features like attributes and a 'zero-coding web service client' that really rocks !
Docker Dublin: Just What is a Service Mesh, and if I get one will it make eve...Elton Stoneman
From the Docker Dublin MeetUp, June 2019. Talking about service meshes – Linkerd and Istio in particular. It’s a technical session so there are lots of demos, but it’s grounded in the practical question – do you really need a service mesh, and is it worth the cost? You’ll learn what a service mesh can do, and how it helps to cut a lot of infrastructure concerns from your code.
Existing mobile backend services can be really useful and quick to start with, however if we'd like to have more flexibility over the backend implementation an fairly easy solution would be to build a backend based on MongoDB and Node.JS. We will then drive this backend with EPAM Systems' open source iOS ROAD framework that has a number of good features like attributes and a 'zero-coding web service client' that really rocks !
(GAM404) Hunting Monsters in a Low-Latency Multiplayer Game on EC2Amazon Web Services
Hear how Turtle Rock launched Evolve, their fast-paced mercenary-vs-monster first-person shooter (FPS), to millions of players using AWS regions around the globe. Turtle Rock provides an in-depth view into Evolve's architecture on AWS, including both their Amazon EC2 and Elastic Load Balancing web API stack, as well as their Crytek-based UDP game servers. Hear how they used Amazon VPC subnets, along with an RDS MySQL based server registration service, to balance players across Availability Zones and regions. Learn about Turtle Rock's innovative game server scaling logic, which maintains a pool of game server capacity while keeping costs in check. Finally, see Evolve’s Graphite and Grafa monitoring setup, which provides player count and server health status across their worldwide fleet.
A high level overview of the new technologies & architectures in the software market today. A more detailed presentation will be coming next to go in depth into different languages / async models & cloud solution maintenance.
Have you heard the news? SQL Server is coming to Linux! What? That can't be right... or can it?
With the release of SQL Server 2017, Microsoft really is bringing SQL Server to the Linux platform. They've also added support for running SQL Server in Docker containers. How does it work? And how well does it work? In this session, we'll answer these questions (and more!), and we'll talk about how this enables some interesting new possibilities for DevOps.
Mariusz Richtscheid: Architektura typu serverless wraz z terminem "Function as a Service" zyskują coraz większą popularność. To całkiem odmienne podejście do tworzenia aplikacji oraz ich wdrażania ma wiele zalet, ale musimy być też świadomi problemów, jakie się z nim wiążą. W trakcie prezentacji pokażę, w jaki sposób można zmodyfikować istniejącą aplikację Node.js tak, by wykorzystać zalety tej architektury.
Get the EDGE to scale: Using Cloudfront along with edge compute to scale your...Amazon Web Services
You could use Cloud Front to deliver pages faster, however, customized processing still required requests to be forwarded back to compute resources at centralized servers, which may slow down the end user experience. This session shows how a combination of Cloud Front, and edge compute can help you scale out your resources in a much more effective way than you think.
Speaker: Anil Nair
Solution Architect, Amazon India
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptxBrad Spiegel Macon GA
Brad Spiegel Macon GA’s journey exemplifies the profound impact that one individual can have on their community. Through his unwavering dedication to digital inclusion, he’s not only bridging the gap in Macon but also setting an example for others to follow.
Understanding User Behavior with Google Analytics.pdfSEO Article Boost
Unlocking the full potential of Google Analytics is crucial for understanding and optimizing your website’s performance. This guide dives deep into the essential aspects of Google Analytics, from analyzing traffic sources to understanding user demographics and tracking user engagement.
Traffic Sources Analysis:
Discover where your website traffic originates. By examining the Acquisition section, you can identify whether visitors come from organic search, paid campaigns, direct visits, social media, or referral links. This knowledge helps in refining marketing strategies and optimizing resource allocation.
User Demographics Insights:
Gain a comprehensive view of your audience by exploring demographic data in the Audience section. Understand age, gender, and interests to tailor your marketing strategies effectively. Leverage this information to create personalized content and improve user engagement and conversion rates.
Tracking User Engagement:
Learn how to measure user interaction with your site through key metrics like bounce rate, average session duration, and pages per session. Enhance user experience by analyzing engagement metrics and implementing strategies to keep visitors engaged.
Conversion Rate Optimization:
Understand the importance of conversion rates and how to track them using Google Analytics. Set up Goals, analyze conversion funnels, segment your audience, and employ A/B testing to optimize your website for higher conversions. Utilize ecommerce tracking and multi-channel funnels for a detailed view of your sales performance and marketing channel contributions.
Custom Reports and Dashboards:
Create custom reports and dashboards to visualize and interpret data relevant to your business goals. Use advanced filters, segments, and visualization options to gain deeper insights. Incorporate custom dimensions and metrics for tailored data analysis. Integrate external data sources to enrich your analytics and make well-informed decisions.
This guide is designed to help you harness the power of Google Analytics for making data-driven decisions that enhance website performance and achieve your digital marketing objectives. Whether you are looking to improve SEO, refine your social media strategy, or boost conversion rates, understanding and utilizing Google Analytics is essential for your success.
Italy Agriculture Equipment Market Outlook to 2027harveenkaur52
Agriculture and Animal Care
Ken Research has an expertise in Agriculture and Animal Care sector and offer vast collection of information related to all major aspects such as Agriculture equipment, Crop Protection, Seed, Agriculture Chemical, Fertilizers, Protected Cultivators, Palm Oil, Hybrid Seed, Animal Feed additives and many more.
Our continuous study and findings in agriculture sector provide better insights to companies dealing with related product and services, government and agriculture associations, researchers and students to well understand the present and expected scenario.
Our Animal care category provides solutions on Animal Healthcare and related products and services, including, animal feed additives, vaccination
Gen Z and the marketplaces - let's translate their needsLaura Szabó
The product workshop focused on exploring the requirements of Generation Z in relation to marketplace dynamics. We delved into their specific needs, examined the specifics in their shopping preferences, and analyzed their preferred methods for accessing information and making purchases within a marketplace. Through the study of real-life cases , we tried to gain valuable insights into enhancing the marketplace experience for Generation Z.
The workshop was held on the DMA Conference in Vienna June 2024.
2.Cellular Networks_The final stage of connectivity is achieved by segmenting...JeyaPerumal1
A cellular network, frequently referred to as a mobile network, is a type of communication system that enables wireless communication between mobile devices. The final stage of connectivity is achieved by segmenting the comprehensive service area into several compact zones, each called a cell.
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024APNIC
Ellisha Heppner, Grant Management Lead, presented an update on APNIC Foundation to the PNG DNS Forum held from 6 to 10 May, 2024 in Port Moresby, Papua New Guinea.
Instagram has become one of the most popular social media platforms, allowing people to share photos, videos, and stories with their followers. Sometimes, though, you might want to view someone's story without them knowing.
17. WebSocket Load
balancing
Permission and
security model
(Admin, Mods, ...)
Frontend Server Backend Server
UI
Ok, so let’s try this!
Frontend
Server
NodeJs
data storage
Redis
Cluster
hitbox
REST-API
PHP
Nginx
Backend
Server
NodeJs
Auto scalingAuto scaling
Average roundtrip / message: < 300ms
18. • Small, cheap machines
• Handle the connections, no logic
• When it breaks it breaks only for a few user
• Automatic Failover to another chat frontend server
• Socket.io for handling websockets
• Carrier for sending messages between front & back
• Up & Downscale possible as needed
Frontend Server
19. • Small, cheap machines
• Handles all the logic
• Stateless, can be restarted/upgraded any time
• Easy expandable with new features
• Up & Downscale possible as needed
• Load balancing via round robin
Backend Server
20. • Fast
• I mean, REALLY fast!
• You can cluster it
• Easy to back up
Redis
25. WebSocket Load
balancing
Permission and
security model
(Admin, Mods, ...)
Frontend Server Backend Server
UI
Ok, let’s fix Websockets
Frontend
Server
NodeJs
data storage
Redis
Cluster
hitbox
REST-API
PHP
Nginx
Backend
Server
NodeJs
Auto scaling
Auto scaling
Long Polling Fallback
Fallback
Server
NodeJs
34. • Frontend servers report CPU load every 10 Seconds
• Lowest X frontend servers are send to the UI
• UI selects a frontend server randomly from this five
• If UI gets disconnected it removes server from list
• UI tries another frontend server
• IF no servers left UI gets X new frontend servers from API
Load Balancing
Thats me, 1980/81 with my first computer, anyone know the computer? I have studed arts, lived in new york & berlin, have made startups and have crashed startups
What is hitbox? This is the frontpage
This is a streamer, he plays games and streams them. Most of them are also entertainer, making money wiht advertising & subscriptions
6 M uniques/month, number 2 in the world
Sounds easy, or? Exists since 30 years.
So, how hard can it be?
Lot of things to do! And thats just the beginning!
Most important is realtime, you write something and all others should see it as fast as possible.
For example, he dances (and lost 20 kg this way) and people cheer him up in the chat.
So back to the chat, IRC is a protocol that is used since 30 years, we wanted to make something new, something modern, something without netsplits, etc.
We started with this because our backend is already in php, lets see if this works out!
Easy setup:
And mysql as database.
Well, these two sentences tell you already all of the problems....
Imagine „Long running php process to server multiple websocket connections“
It worked for up to 2000 connections, not very scaleable!
So back to the drawing board, we wanted something modern, so lets use modern software!
We went with nodejs & redis, anyone here has experience with nodejs as servers?
We use a two way setup:
Frontend servers and backendservers and redis as a data storage. If we loose the redis data we just loose who is in what chatroom, just press f5 and you are back in.
We use AWS
Single core machines
Same machines as for frontends
I can only recommend it, i never saw a redis instance failing (except for getting slow)
So, looks like a perfect system, lets code it!
We did and...
it worked!
So we could party!
Not so fast!!!!!!!
There we had our first problem, something everyone should support
Its a fucking standard!
But there are firewalls that block it, there are mobile devices that block it or even worse, tell you that a websocket connection is working but it isnt, they just lie to you!
0,5-1% have this problem, but they where mailing us like hell....
0,5-1%
So we had to use fallback servers for long polling. Long polling means a lot of overhead from the http-protocoll, so these servers can handle only 1/10 of the normal frontend server, but it works!
So we thought we can party again.
Well, the hitbox audience is young, so they try a lot... You wont imagine how often we get ddosed or people try to abuse the api....
And last yeatr, someone managed it:
It was during at that time biggest event ever, 60k people on one stream and suddenly all of them saw this.
And we did this!
Well, they did not managed to break our system or steal any userdata, the only think they did was insert in the „nameColor“ some javascript, and we did not validated it. We validated everything else, but not this one, because it is only a number...
so
Really, everything, really, really everything!
Again, we thought we can party!
But.... Then others came and did this
A websocket DDOS! Sending massive amounts of join commands to the chat.
So we had to think about how we can distribute this load better or make it harder for them to reeach all frontend server, remember, they are up & down scaling automatic.
So this is our way how we do load balancing on the frontend servers, works really good.
If they ddos a few servers this servers will not get new connections and from the upscale we get new servers that are not ddosed.
Why the random factor in the ui? F5, more on this later.
So once again we party hard!
Until he came
Rezigiusz a polish Youtuber & streamer with a lot of fans that love to type
Think of it as one direction of poland
When he is streaming he has around 1-15k viewers and they type 2000 messages a second into the chat!
1995 get blocked, but the backend servers have to check this....
So the event loop of nodejs exploded....
But, using async.js, whic is a great tool to queue work we could clean up the event loop, delaying some messages a few milliseconds but letting the main tasks working fine
So for example we made queues for the most important function, login, logout, chatmsg, etc.
So, we can party again!
But, dont forget one of the biggest problems you can run into...
I know, this sound s stupid, but i will give you two examples:
Imagine you have a stream with 100k viewers. Every time a new viewer comes to this stream he/she gets the info about how to get the stream from our server.
Now imagine the streamer has a problem, lets say his computer crashes and the stream drops, mean is getting black or stucked.
What does 100k people do?
This.
And lets hope that your api can handle this!
And they wont stop until trhe have a stream again!
We learned a lot about caching, otherwise you cannot handle this, memcache & redis are your friend here.
The second example is stupid sotware design:
It is quite often that streamer announce when they start to stream and then people are waiting already on the page for them to go online.
Well, we have the chat already connected anyway, why not send a special message over the chat to trigger the start of the stream...
Sounds easy, for our system it is
Because than again you weill self ddos yourself, imagine this with 100k people waiting...
So sometimes realtime is really bad, because it is realtime... And it can destroy you
So we got back to the good old interval because then you distribute the 100k connections over 30 seconds, giving you much more time handle the load.
So, we can party again!
The same guy as at the beginning, he has its own website with animated gifs
Well, at the end something that is very important for me, monitor everything!
Our swiss army knife is statsd from etsy, a great peace of software written in nodes that monitors stuff via udp and works great.
We use it in cobimation with graphite and monitor really everything.
See the down-spike on active chat connections? That is when node is not able to keep the 10 seconds timing for the reporting of the stats, you get used to it
Well, and at the end, is the chat system working? Does it scale?
Well, i dont have a screenshot about our latest record that was close to 200k, but this one shows you a channel with 100k people.
All 154k connections where handled by 16 frontend servers and 8 backend servers, costing us around $20 for the evening.
And dont forget the network traffic!
Around 160-200Mbit per machine, only text outgoing! These cheap machines are limited by around 200mbit.
Thats it, thank you!