This document discusses scaling a social game to support millions of daily users. It summarizes strategies for:
1. Simulating large numbers of users to test the system using Tsung, which can simulate meaningful game sessions.
2. Collecting metrics on system performance from HAProxy and analyzing them using Python and R to understand performance at both the request level and for specific calls.
3. Digging deeper into slow calls by profiling with Erlang to optimize a Redis query, and benchmarking a Redis driver called Eredis using Basho Bench.
4. Measuring internal application performance in production using a homegrown system inspired by Basho Bench to generate latency statistics without overhead.
Erlang factory SF 2011 "Erlang and the big switch in social games"Paolo Negri
talk given at erlang factory 2011 about using erlang to build social games backends
Watch the video of this presentation http://vimeo.com/22144057#at=0
Slides of erlang factory 2011 London talk "Designing for performance with erlang"
Video of this presentation available at http://vimeo.com/26715793#at=0
How Parse Built a Mobile Backend as a Service on AWS (MBL307) | AWS re:Invent...Amazon Web Services
Parse is a BaaS for mobile developers that is built entirely on AWS. With over 150,000 mobile apps hosted on Parse, the stability of the platform is our primary concern, but it coexists with rapid growth and a demanding release schedule. This session is a technical discussion of the current architecture and the design decisions that went in to scaling the platform rapidly and robustly over the past year and a half. We talk about some of the lessons learned managing and scaling MongoDB, Cassandra, Redis, and MySQL in the cloud. We also discuss how Parse went from launching individual instances using chef to managing clusters of hosts with Auto Scaling groups, with instance discovery and registry handled by ZooKeeper, thus enabling us to manage vastly larger sets of services with fewer human resources. This session is useful to anyone who is trying to scale up from startup to established platform without sacrificing agility.
Erlang factory SF 2011 "Erlang and the big switch in social games"Paolo Negri
talk given at erlang factory 2011 about using erlang to build social games backends
Watch the video of this presentation http://vimeo.com/22144057#at=0
Slides of erlang factory 2011 London talk "Designing for performance with erlang"
Video of this presentation available at http://vimeo.com/26715793#at=0
How Parse Built a Mobile Backend as a Service on AWS (MBL307) | AWS re:Invent...Amazon Web Services
Parse is a BaaS for mobile developers that is built entirely on AWS. With over 150,000 mobile apps hosted on Parse, the stability of the platform is our primary concern, but it coexists with rapid growth and a demanding release schedule. This session is a technical discussion of the current architecture and the design decisions that went in to scaling the platform rapidly and robustly over the past year and a half. We talk about some of the lessons learned managing and scaling MongoDB, Cassandra, Redis, and MySQL in the cloud. We also discuss how Parse went from launching individual instances using chef to managing clusters of hosts with Auto Scaling groups, with instance discovery and registry handled by ZooKeeper, thus enabling us to manage vastly larger sets of services with fewer human resources. This session is useful to anyone who is trying to scale up from startup to established platform without sacrificing agility.
A short introduction to the more advanced python and programming in general. Intended for users that has already learned the basic coding skills but want to have a rapid tour of more in-depth capacities offered by Python and some general programming background.
Execrices are available at: https://github.com/chiffa/Intermediate_Python_programming
Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen
Slides of the presentation about Kubernetes practices and learnings at NU.nl.
This presentation was the first of two at the Dutch Kubernetes meetup at the Sanoma Netherlands offices, that took place on Sept. 5th 2019
Cloud Connected Devices on a Global Scale (CPN303) | AWS re:Invent 2013Amazon Web Services
Increasingly, mobile and other connected devices are leveraging the scalability and capabilities of the cloud to deliver services to end users. However, connecting these devices to the cloud presents unique challenges. Resource constraints make it impossible to use many common frameworks and transport restrictions make it difficult to use dynamic cloud resources. In this session, learn how you can develop and deploy highly-scalable global solutions using Amazon Web Services (Amazon Virtual Private Cloud, Elastic IP addresses, Amazon Route 53, Auto Scaling) and tools like Puppet. Hear how Panasonic and Banjo architect their cloud infrastructure from both a start-up and enterprise perspective.
Webinar: Queues with RabbitMQ - Lorna MitchellCodemotion
Queues are a great addition to any application that has some tasks that need processing asynchronously. This could be sending a confirmation email, resizing an avatar, or recalculating a running total of some kind; in all those cases it would be cool to send the response back to the user and then sort out that task later. This session looks at how to use a RabbitMQ job queue in your application. It also looks at how to design elegant and robust long-running workers that will consume the jobs from the queue and process them. This session is ideal for technical leads, developers and architects alike.
CQRS (Command Query Responsibility Segregation) was all the hype in .NET architecture circles a few years back. But has it faded away? Is it old news? I argue that it hasn't, and the concepts of CQRS are alive and well and probably more widely accepted and considered today than a few years ago. From event-driven systems to the Reactive Manifesto, the principles of CQRS are with us and impacting many different tools. In this session, we'll explore those CQRS principles and how they have manifested themselves in the architectures of choice today. You'll come away with a greater appreciation of CQRS and ideas on how to incorporate those principles in your applications today.
Learn how we use NoSQL and SQL databases to build games at Wooga. From prototypes to production systems that deal with the data of millions of players, this talk covers hands on use cases and learnings.
This talk wants to sum up the experience of designing, deploying and maintaining an Erlang application targeting the cloud and precisely AWS as hosting infrastructure.
As the application now serves a significantly large user base with a sustained throughput of thousands of games actions per second we're able to analyse retrospectively our engineering and architectural choices and see how Erlang fits in the cloud environment also comparing it to previous experiences of clouds deployments of other platforms.
We'll discuss properties of Erlang as a language and OTP as a framework and how we used them to design a system that is a good cloud citizen. We'll also discuss topics that are still open for a solution.
At wooga the separate game teams operate their own games. That means that two developers not only develop the backend for a social game but they also the administrator's part.
This presentation gives an insight on how this is done, what tools are used and how the most important challenges were solved.
After more than one year of development, Wooga is heading for the global launch of its game "Kingsbridge"!
This is the first game at Wooga with a backend written in JRuby!
The talk includes an introduction to the problems that were solved by choosing a stateful applicaton server.
I will explain constraints, benefits and obvious differences to traditional database backed application servers.
Safely sharing state in a concurrent environment using JRuby
Using Java concurrency utils in JRuby
Sample problems solved, backed up with code
Practical tips for capacity planning
Games for the Masses: Scaling Rails to the ExtremeWooga
This presentation explains and compares the work of two engineering teams that build Facebook game backends at wooga: Cloud vs. dedicated servers, SQL vs. NoSQL and in-memory database vs. a database hierarchy. He will highlight their respective advantages and disadvantages and discuss some common patterns both teams came up with to solve their problems.
Did you ever work on a feature where you had multiple options on how to display it to the user but you did not know which one would work best? Why speculate about what would work best if you can actually measure it?
Welcome to the world of a/b testing: Implement multiple variations of the same thing, and let different users experience different variations. Measure which variant works best and continue with that one.
At Wooga we extensively use a/b testing to polish features to perfection. This helped us becoming one of the biggest providers of social games on Facebook with about 40 million players every month.
We would like to show you what a/b testing is good for, how you do it and what pitfalls you need to avoid.
A short introduction to the more advanced python and programming in general. Intended for users that has already learned the basic coding skills but want to have a rapid tour of more in-depth capacities offered by Python and some general programming background.
Execrices are available at: https://github.com/chiffa/Intermediate_Python_programming
Kubernetes at NU.nl (Kubernetes meetup 2019-09-05)Tibo Beijen
Slides of the presentation about Kubernetes practices and learnings at NU.nl.
This presentation was the first of two at the Dutch Kubernetes meetup at the Sanoma Netherlands offices, that took place on Sept. 5th 2019
Cloud Connected Devices on a Global Scale (CPN303) | AWS re:Invent 2013Amazon Web Services
Increasingly, mobile and other connected devices are leveraging the scalability and capabilities of the cloud to deliver services to end users. However, connecting these devices to the cloud presents unique challenges. Resource constraints make it impossible to use many common frameworks and transport restrictions make it difficult to use dynamic cloud resources. In this session, learn how you can develop and deploy highly-scalable global solutions using Amazon Web Services (Amazon Virtual Private Cloud, Elastic IP addresses, Amazon Route 53, Auto Scaling) and tools like Puppet. Hear how Panasonic and Banjo architect their cloud infrastructure from both a start-up and enterprise perspective.
Webinar: Queues with RabbitMQ - Lorna MitchellCodemotion
Queues are a great addition to any application that has some tasks that need processing asynchronously. This could be sending a confirmation email, resizing an avatar, or recalculating a running total of some kind; in all those cases it would be cool to send the response back to the user and then sort out that task later. This session looks at how to use a RabbitMQ job queue in your application. It also looks at how to design elegant and robust long-running workers that will consume the jobs from the queue and process them. This session is ideal for technical leads, developers and architects alike.
CQRS (Command Query Responsibility Segregation) was all the hype in .NET architecture circles a few years back. But has it faded away? Is it old news? I argue that it hasn't, and the concepts of CQRS are alive and well and probably more widely accepted and considered today than a few years ago. From event-driven systems to the Reactive Manifesto, the principles of CQRS are with us and impacting many different tools. In this session, we'll explore those CQRS principles and how they have manifested themselves in the architectures of choice today. You'll come away with a greater appreciation of CQRS and ideas on how to incorporate those principles in your applications today.
Learn how we use NoSQL and SQL databases to build games at Wooga. From prototypes to production systems that deal with the data of millions of players, this talk covers hands on use cases and learnings.
This talk wants to sum up the experience of designing, deploying and maintaining an Erlang application targeting the cloud and precisely AWS as hosting infrastructure.
As the application now serves a significantly large user base with a sustained throughput of thousands of games actions per second we're able to analyse retrospectively our engineering and architectural choices and see how Erlang fits in the cloud environment also comparing it to previous experiences of clouds deployments of other platforms.
We'll discuss properties of Erlang as a language and OTP as a framework and how we used them to design a system that is a good cloud citizen. We'll also discuss topics that are still open for a solution.
At wooga the separate game teams operate their own games. That means that two developers not only develop the backend for a social game but they also the administrator's part.
This presentation gives an insight on how this is done, what tools are used and how the most important challenges were solved.
After more than one year of development, Wooga is heading for the global launch of its game "Kingsbridge"!
This is the first game at Wooga with a backend written in JRuby!
The talk includes an introduction to the problems that were solved by choosing a stateful applicaton server.
I will explain constraints, benefits and obvious differences to traditional database backed application servers.
Safely sharing state in a concurrent environment using JRuby
Using Java concurrency utils in JRuby
Sample problems solved, backed up with code
Practical tips for capacity planning
Games for the Masses: Scaling Rails to the ExtremeWooga
This presentation explains and compares the work of two engineering teams that build Facebook game backends at wooga: Cloud vs. dedicated servers, SQL vs. NoSQL and in-memory database vs. a database hierarchy. He will highlight their respective advantages and disadvantages and discuss some common patterns both teams came up with to solve their problems.
Did you ever work on a feature where you had multiple options on how to display it to the user but you did not know which one would work best? Why speculate about what would work best if you can actually measure it?
Welcome to the world of a/b testing: Implement multiple variations of the same thing, and let different users experience different variations. Measure which variant works best and continue with that one.
At Wooga we extensively use a/b testing to polish features to perfection. This helped us becoming one of the biggest providers of social games on Facebook with about 40 million players every month.
We would like to show you what a/b testing is good for, how you do it and what pitfalls you need to avoid.
Wooga is one the leading publishers of social games for web and mobile. Throughout the last three years the numbers of employees have doubled each year. But the company culture, the very essence of what Wooga is all about, remained untouched: It's still all about game teams, that each act as a small and agile startup with Wooga acting as a container (or incubator) for them.
The talk is about a stateful application server built on top of JRuby/JVM. We at Wooga have built and evaluated such an application server, and now I want to share the learnings and obstacles that came up during development.
Scaling up: We aim to fully utilize our available hardware. One box has 32 Cores and more than 32GB RAM available. How to saturate such beasts? Well, since JRuby supports real Threads we can share state (uh oh!) and saturate cores.
Scaling out: What are the options for scaling a stateful application server? How to shard state across many servers safely?
I aim for a talk that gives useful tips and shows code. Basically a ratio between 30% background information, 70% showcasing.
The talk will cover the following topics:
Safely sharing state in a concurrent environment using JRuby
Using Java concurrency utils in JRuby
Practical tips for tuning JRuby/JVM for maximum throughput
Practical tips for evaluating performance tunings
Bonus: Timetraveling
You are not alone - Scaling multiplayer gamesWooga
At Wooga we are creating the next generation of social games. To be truly social, a game needs to offer real interaction between players in different forms.
Taking one of our upcoming games as an example we will present the state synchronization used in client and server, how we do real time communication without websockets and how we are making a backend scalable enough to support millions of users every day. We will also share secrets about our prototypes for first person and real time strategy games.
The Erlang process model with evented IO, asynchronous messaging, links and isolation fits very well to real time multiplayer games. Come to our talk to see how it's used to build a very successful business.
A story of a Ruby programmer having to understand that learning Erlang is more than just syntax. Learn differences in paradigms, pitfalls and applied use cases for this incredibly powerful language
Continuous Integration for iOS (iOS User Group Berlin)Wooga
In this talk at the iOS User Group Berlin, Mattes Groeger shows how Continuous Integration for iOS is realized at Wooga. It covers topics ranging from setup over configuration, testing to distribution.
You can follow him on Twitter: https://twitter.com/MattesGroeger
Painful success - lessons learned while scaling upWooga
Full of hope we started developing our biggest games so far in 2009. In 2010 we released it and gradually scaled up to 1 million users a day. But we were forced to rewrite our whole persistence layer and migrate most of our data from MySQL to Redis to make it work. In 2011 we even got 2 million daily users, but we also had to operate 200 servers which was painful - especially when the whole data center went down. So in 2012 we wanted to make everything better and started large refactoring projects - and made everything worse.
But in 2013 everything will run smoothly and painless - promised! At least we hope so.
This talk explains how we use NoSQL databases at wooga to scale our backend services to deal with the data of millions of players every day. Learn from our experience running Basho's Riak in production and how to use Amazon S3 as a database.
Go beyond server and application stack monitoring, collect application specific metrics and push the information to interested parties. With the help of syslog, EventMachine and a few lines of Ruby code, you can build a language agnostic system to gather and process custom tailored reporting data.
Deploying Apache Flume to enable low-latency analyticsDataWorks Summit
The driving question behind redesigns of countless data collection architectures has often been, ?how can we make the data available to our analytical systems faster?? Increasingly, the go-to solution for this data collection problem is Apache Flume. In this talk, architectures and techniques for designing a low-latency Flume-based data collection and delivery system to enable Hadoop-based analytics are explored. Techniques for getting the data into Flume, getting the data onto HDFS and HBase, and making the data available as quickly as possible are discussed. Best practices for scaling up collection, addressing de-duplication, and utilizing a combination streaming/batch model are described in the context of Flume and Hadoop ecosystem components.
Xin Wang(Apache Storm Committer/PMC member)'s topic covered the relations between streaming and messaging platform, and the challenges and tips in Storm usage.
Debugging Microservices - key challenges and techniques - Microservices Odesa...Lohika_Odessa_TechTalks
Microservice architecture is widespread our days. It comes with a lot of benefits and challenges to solve. Main goal of this talk is to go through troubleshooting and debugging in the distributed micro-service world. Topic would cover:
main aspects of the logging,
monitoring,
distributed tracing,
debugging services on the cluster.
About speaker:
Andrеy Kolodnitskiy is Staff engineer in the Lohika and his primary focus is around distributed systems, microservices and JVM based languages.
Majority of time engineers spend debugging and fixing the issues. This talk will be dedicated to best practicies and tools Andrеys team uses on its project which do help to find issues more efficiently.
Push jobs: an orchestration building block for private ChefChef Software, Inc.
Push jobs is a new feature in Opscode Private Chef that will allow a user to run commands across hundreds of chef managed servers. Push Jobs leverages Erlang/OTP and ZeroMQ to provide scalable and fault tolerant execution.
In this talk I’ll cover the general motivation behind the design and an architectural overview of the system. This will include details of we used Erlang and ZeroMQ to build a robust, scalable system. I’ll also do a demo of the push job feature in action, covering the push jobs server, execution client and knife command line interface.
Story of Warlords: Bringing a turn-based strategy game to mobile Wooga
About three years ago we set out on a mission: We wanted to bring a round-based strategy game, a genre that most gamers so far mainly enjoyed on PCs, to mobile. Today that mission is nearly completed with Warlords having been soft launched in various countries and showing very promising data. This may all sound easy enough, but the truth is that it was a hell of a ride for the team and there were times when we weren´t even sure if the project would continue. Along the way, we´ve learned many valuable things which we believe in the end made the difference and led to the numbers we now see. Wilhelm, Head of Studio and Product Lead for Warlords, will share these learnings in his talk answering the following questions: What set screws did we touch to make the session structure, pacing and content consumption work on mobile? Why different meta game systems are crucial? And finally how sticking to your game vision from start to end can make you survive such a ride.
In it for the long haul - How Wooga boosts long-term retentionWooga
GDC San Francisco 2015 Presentation by Sebastian Nussbaum and Adam Telfer
Today's mobile games market is tough. With thousands of new titles being available to download every week and CPIs rising high, new game concepts need to stand out. But even more importantly, they need to stay relevant for a long time. Long-term retention is the key for having a realistic shot at the top-grossing ranks of the app stores. In the first part of this talk, attendees will learn how mobile game developer Wooga evaluates new IP to ensure a long-lasting gameplay experience. The second half will show one example of how storytelling and episodic content can drive long-term retention. Sebastian will share insights and learnings from Wooga's projects in episodic content production and how to best set up your team for the ride.
Entitas System Architecture with Unity - Maxim Zaks and Simon Schmid Wooga
UNITE Europe 2015 + Unity User Group Presentation
Unity incorporates the component based architecture in a seamless manner. But for some games, a more data driven approach (entity system architecture) fits better. In this talk, Maxim Zaks & Simon Schmid (Wooga) show why entity system architecture fits and how you can use Entitas-CSharp in your own Unity project.
Saying No to the CEO: A Deep Look at Independent Teams - Adam TelferWooga
GDC Europe 2015
Truly independent teams within a game studio are an interesting idea. Wooga, Supercell and others have promoted having this type of structure. Each team having full ownership over decisions in their game. Teams saying no to the rest of the company when they believe in a product. This all sounds great on paper, but what happens in practice? Adam Telfer goes into details of the ups and downs of working in an independent culture. How Wooga has adapted its processes through the years. A culture where you have full control, but also full responsibility.
Innovation dank DevOps (DevOpsCon Berlin 2015)Wooga
“You build it, you run it!” - Wenn Du als Entwickler weisst, dass Du Deine Software selbst betreiben musst, was bist bereit zu tun, um den späteren Betrieb zu vereinfach?
Bei Wooga haben Dutzende von Teams ihre eigene Antwort auf die Frage gesucht und dabei von den Erfahrungen der anderen Teams gelernt. Herausgekommen ist ein großes Experimentierfeld beim Betrieb von Web Services - und eine technologische Innovation, die uns innerhalb weniger Iterationen von einem simplen LAMP-Stack zu lastabhängig skalierenden stateful Servern auf Basis von Erlang oder Akka gebracht hat.
Big Fish, small pond - strategies for surviving in a maturing market - Ed BidenWooga
Quo Vadis Conference Berlin 2015
Mobile games have never been more competitive. Production values and marketing costs are ever increasing and the top grossing charts look strangely similar to the year before. So how can developers maximize their chances of success in such a tough market? Is it better to clone a hit or innovative wildly? Should they focus on one genre, or dabble in many? Ed draws on extensive market analysis and experience developing games to explain why selective innovation is the key to success.
Tom LeClerc's talk at App Promotion Summit Berlin 2014:
REVIEW MINING:
THE APP STORE OPTIMIZER’S SECRET WEAPON
An Overview Of The Value Of Review Mining
What Review Mining Tools You Have At Your Disposal
Using Reviews To Generate Keywords And Understand Your Users
We've learned a lot through doing DevOps: Every commit is automatically integrated, tested, and deployed to a staging environment. And then it only takes one push of a button and the release goes live...
Unfortunately, it's not as simple anymore when operating mobile applications: How can you quickly update your mobile software when the app store provider wants to test your software first for a few days? How can you update your configuration when your app can run offline? And how do you track down errors when the data is distributed to millions of mobile clients? Those were just some of the challenges we encountered during the operation of mobile games with millions of daily users. In this talk we will talk about the solutions we have found to address them.
DevOps goes Mobile - Jax 2014 - Jesper Richter-ReichhelmWooga
DevOps hat uns viel gelehrt: Jedes Commit wird automatisch integriert, getestet und in eine Stagingumgebung installiert. Und einen Knopfdruck später geht das Release dann live.
Aber leider funktioniert das und vieles mehr, was uns in Zeiten von DevOps als Normal erscheint, nicht bei mobilen Applikationen.
Wie kann ich also schnell meine mobile Software anpassen, wenn der Betreiber des App Stores erst tagelang testen will? Wie kann ich Konfiguration anpassen, wenn die App auch offline laufen soll? Und wie funktioniert Fehlersuche, wenn die Daten auf Millionen von mobilen Clients verteilt sind?
Beim Betrieb von mobilen Spielen mit Millionen täglicher Nutzer standen wir genau diesen Fragen gegenüber. Der Vortrag wird darlegen, welche Antworten wir darauf gefunden haben.
Jelly Splash: Puzzling your way to the top of the App Stores - GDC 2014Wooga
The match 3 puzzle genre is almost as old as it gets. Scour the App Store and you'll find hundreds of different varieties out there. Very few of these succeed however, and even less manage to hit the number one spot on the U.S. Apple App Store top download chart. Wooga's Jelly Splash managed to do just that, and in this session Florian Steinhoff, the creator of Jelly Splash, will give a detailed account on how his team managed that and what he learned throughout the development process.
Two Ann(e)s and one Julia_Wooga Lady Power from Berlin_SGA2014Wooga
Three people, three talks: First, Anne explains how Wooga makes it in the hit driven mobile F2P world and frequently creates hit games. Then, Ann and Julia share their insights into the company from an intern’s perspective. Julia is currently working with a team in the prototyping phase using C#. Ann develops Objective-C for one of Wooga's live games. Both of them are students of Chalmers University of Technology, Gothenburg. Anne is in charge of University Relations at Wooga.
Tags: Equality, production, programming, career.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
10. Simulating users
• Must not be too synthetic (like
apachebench)
• Must look like a meaningful game session
• Users must come online at a given rate and
play
11. Tsung
• Multi protocol (HTTP, XMPP) benchmarking tool
• Able to test non trivial call sequences
• Can actually simulate a scripted gaming session
http://tsung.erlang-projects.org/
13. Tsung - configuration
• Not something you fancy writing
• We’re in development, calls change and we
constantly add new calls
• A session might contain hundreds of
requests
• All the calls must refer to a consistent game
state
http://tsung.erlang-projects.org/
14. Tsung - configuration
• From our ruby test code
user.resources(:column => 5, :row => 14)
• Same as
<request subst="true">
<http url="http://server.wooga.com/users/%
%ts_user_server:get_unique_id%%/resources/column/5/
row/14?%%_routing_key%%"
method="POST" contents='{"parameter1":"value1"}'>
</http>
</request>
http://tsung.erlang-projects.org/
15. Tsung - configuration
• Session A session is a
• requests group of requests
• Arrival phase Sessions arrive in
• duration phases with a
specific arrival
• arrival rate rate
http://tsung.erlang-projects.org/
16. Tsung - setup
Application Benchmarking
cluster
app server tsung
HTTP reqs worker
ssh
app server
tsung
master
app server
http://tsung.erlang-projects.org/
17. Tsung
• Generates ~ 2500 reqs/sec on AWS
m1.large
• Flexible but hard to extend
• Code base rather obscure
http://tsung.erlang-projects.org/
19. Tsung-metrics
• Tsung collects measures and provides
reports
• But these measure include tsung network/
cpu congestion itself
• Tsung machines aren’t a good point of view
http://tsung.erlang-projects.org/
20. HAproxy
Application Benchmarking
cluster
app server tsung
HTTP reqs worker
haproxy ssh
app server
tsung
master
app server
21. HAproxy
“The Reliable, High Performance TCP/
HTTP Load Balancer”
• Placed in front of http servers
• Load balancing
• Fail over
22. HAproxy - syslog
• Easy to setup
• Efficient (UDP)
• Provides 5 timings per each request
23. HAproxy
• Time to receive request from client
Application Benchmarking
cluster
app server tsung
haproxy worker
ssh
app server
tsung
master
24. HAproxy
• Time spent in HAproxy queue
Application Benchmarking
cluster
app server tsung
haproxy worker
ssh
app server
tsung
master
25. HAproxy
• Time to connect to the server
Application Benchmarking
cluster
app server tsung
haproxy worker
ssh
app server
tsung
master
26. HAproxy
• Time to receive response headers from server
Application Benchmarking
cluster
app server tsung
haproxy worker
ssh
app server
tsung
master
27. HAproxy
• Total session duration time
Application Benchmarking
cluster
app server tsung
haproxy worker
ssh
app server
tsung
master
28. HAproxy - syslog
• Application urls identify directly server call
• Application urls are easy to parse
• Processing haproxy syslog gives per call
metric
30. Reading/aggregating
metrics
• Python to parse/normalize syslog
• R language to analyze/visualize data
• R language console to interactively explore
benchmarking results
31. R is a free software environment for
statistical computing and graphics.
32. What you get
• Aggregate performance levels (throughput,
latency)
• Detailed performance per call type
• Statistical analysis (outliers, trends,
regression, correlation, frequency, standard
deviation)
35. Digging into the data
• From HAproxy log analisys one call
emerged as exceptionally slow
• Using eprof we were able to determine
that most of the time was spent in a redis
query fetching many keys (MGET)
36. Tracing erldis query
• More than 60% of runtime is spent
manipulating the socket
• gen_tcp:recv/2 is the culprit
• But why is it called so many times?
39. A different approach
• Two ways to use gen_tcp: active or passive
• In passive, use gen_tcp:recv to explicitly ask
for data, blocking
• In active, gen_tcp will send the controlling
process a message when there is data
• Hybrid: active once
40. A different approach
• Is active sockets faster?
• Proof-of-concept proved active socket
faster
• Change erldis or write a new driver?
41. A different approach
• Radical change => new driver
• Keep Erldis queuing approach
• Think about error handling from the start
• Use active sockets
43. Circuit breaker
• eredis has a simple circuit breaker for when
Redis is down/unreachable
• eredis returns immediately to clients if
connection is down
• Reconnecting is done outside request/
response handling
• Robust handling of errors
44. Benchmarking eredis
• Redis driver critical for our application
• Must perform well
• Must be stable
• How do we test this?
45. Basho bench
• Basho produces the Riak KV store
• Basho build a tool to test KV servers
• Basho bench
• We used Basho bench to test eredis
51. Measure internals
HAproxy point of view is valid but how to
measure internals of our application, while
we are live, without the overhead of
tracing?
52. Think Basho bench
• Basho bench can benchmark a redis driver
• Redis is very fast, 100K ops/sec
• Basho bench overhead is acceptable
• The code is very simple
53. Cherry pick ideas from
Basho Bench
• Creates a histogram of timings on the fly,
reducing the number of data points
• Dumps to disk every N seconds
• Allows statistical tools to work on already
aggregated data
• Near real-time, from event to stats in N+5
seconds
54. Homegrown stats
• Measures latency from the edges of our
system (excludes HTTP handling)
• And at interesting points inside the system
• Statistical analysis using R
• Correlate with HAproxy data
• Produces graphs and data specific to our
application
56. Recap
Measure:
• From an external point of view (HAproxy)
• At the edge of the system (excluding
HTTP handling)
• Internals in the single process (eprof)
57. Recap
Analyze:
• Aggregated measures
• Statistical properties of measures
• standard deviation
• distribution
• trends