Redis at the guardian
•Download as KEY, PDF•
0 likes•1,104 views
The document discusses Redis usage at Guardian.co.uk and for a Soulmates architecture. Redis is used to track reviewed articles, count requests and SQL operations in Django, and implement management switches across processes and servers without a centralized database. Switches can be toggled from any server or process by hitting a URL, and all processes can later check the switch value stored in Redis.
Report
Share
Report
Share
Recommended
Sail In The Cloud
Microservices architecture has changed how companies develop and deploy applications. This change has affected testing process as well. New techniques have emerged and others have been enhanced. Does your organization wish to achieve unicorn like speed? With N deploys per day? Or a lead time (time to business value from idea to production) measured in days/weeks instead of months/years?In this session, we will not only describe how to use Istio and Kubernetes but actively demonstrate 3 specific techniques that you can use immediately following the session.
From Chef to Saltstack on Cloud Providers - Incontro DevOps 2015
The document summarizes the author's journey from using on-premise automation with shell scripting to using configuration management tools Chef and SaltStack. It outlines some of the challenges faced with shell scripting and Chef related to idempotence, cross-platform recipes, and lack of orchestration. The author found SaltStack improved on these areas with its use of states, pillars, grains, and modules which allowed for smarter formulas and automation of dynamic infrastructures on cloud providers. Examples are provided of using grains, pillars, peer communication, and Salt Mine to configure applications and services across minions in a declarative way.
J hipster
JHipster is a code generator that creates a full-fledged Java web project with a responsive front-end using AngularJS or Angular 2, a documented REST API, tests, security, and database integration. It leverages technologies like Spring Boot, Swagger, Maven, Npm, Gatling, and more. To get started, install JHipster using Yarn or Npm, generate a new project with 'yo jhipster', and make choices like monolithic or microservices, Maven or Gradle, and AngularJS 1 or Angular 2. The generator handles setting up the project with all necessary configurations and components.
Deploy And Release at Kubernetes Era
Microservices architecture has changed how companies develop, deploy and release applications. Some technologies such as Docker and Kubernetes has emerged to simplify how to put applications into production, increasing the release velocity from months to N times per day.
If you are into DevOps bandwagon, come to this session to learn how Kubernetes and Istio (the new Crown jewel) can help you on reducing the lead time (time to business value from idea too production) and start releasing at unicorn speed.
CloudSkew Architecture
The document discusses best practices for building microservices including:
- Focus on rapid development of features over infrastructure optimization.
- Deploy prototypes and experiments quickly using serverless architectures with auto-scaling and zero server management.
- Design microservices as independent, stateless services that communicate asynchronously through a service bus.
- Instrument services with logging and metrics to monitor performance and observe issues.
What monolith can learn from microservices?
Nowadays everyone is talking about microservices and how cool they are. That’s true, but not all business needs to move to microservices, you can still use monolith architecture and make it more fashion by applying some of the design/elements that you’ve learned from microservices architecture.
Can you use circuit breaker pattern, service isolation, event-driven and non-blocking application, cloud-native approach or zero downtime deployment in monolith architecture? Is it worth? The answer is YES!!!.
Come to this session to learn how to make your Java monolith applications attractive again by borrowing some of the principles of microservices architecture.
Software Quality Visualization
Software development process, in-house or outsourced, might be a challenge. It is important to choose a vendor or hire a team of professionals who understand that quality of source code directly impacts the overall cost of your project. You most probably want to find a vendor who knows how to reveal all the risk hidden behind low quality code. Tricky part in revealing risk and possible quality issues is that you have to know how to do this at the time of development, not when it’s already too late.
Author:
Łukasz Koczwara - Software Development Manager @ STX Next
PyconKR 2019 Lightning Talk - Let The Dogs Out on Kubernetes
This document discusses monitoring Kubernetes clusters using Datadog. It describes how to collect metrics from worker nodes, microservices, and deployments using Datadog. It also discusses challenges with monitoring conntrack states and proposes a solution using an init container to flush conntrack states periodically to avoid issues.
Recommended
Sail In The Cloud
Microservices architecture has changed how companies develop and deploy applications. This change has affected testing process as well. New techniques have emerged and others have been enhanced. Does your organization wish to achieve unicorn like speed? With N deploys per day? Or a lead time (time to business value from idea to production) measured in days/weeks instead of months/years?In this session, we will not only describe how to use Istio and Kubernetes but actively demonstrate 3 specific techniques that you can use immediately following the session.
From Chef to Saltstack on Cloud Providers - Incontro DevOps 2015
The document summarizes the author's journey from using on-premise automation with shell scripting to using configuration management tools Chef and SaltStack. It outlines some of the challenges faced with shell scripting and Chef related to idempotence, cross-platform recipes, and lack of orchestration. The author found SaltStack improved on these areas with its use of states, pillars, grains, and modules which allowed for smarter formulas and automation of dynamic infrastructures on cloud providers. Examples are provided of using grains, pillars, peer communication, and Salt Mine to configure applications and services across minions in a declarative way.
J hipster
JHipster is a code generator that creates a full-fledged Java web project with a responsive front-end using AngularJS or Angular 2, a documented REST API, tests, security, and database integration. It leverages technologies like Spring Boot, Swagger, Maven, Npm, Gatling, and more. To get started, install JHipster using Yarn or Npm, generate a new project with 'yo jhipster', and make choices like monolithic or microservices, Maven or Gradle, and AngularJS 1 or Angular 2. The generator handles setting up the project with all necessary configurations and components.
Deploy And Release at Kubernetes Era
Microservices architecture has changed how companies develop, deploy and release applications. Some technologies such as Docker and Kubernetes has emerged to simplify how to put applications into production, increasing the release velocity from months to N times per day.
If you are into DevOps bandwagon, come to this session to learn how Kubernetes and Istio (the new Crown jewel) can help you on reducing the lead time (time to business value from idea too production) and start releasing at unicorn speed.
CloudSkew Architecture
The document discusses best practices for building microservices including:
- Focus on rapid development of features over infrastructure optimization.
- Deploy prototypes and experiments quickly using serverless architectures with auto-scaling and zero server management.
- Design microservices as independent, stateless services that communicate asynchronously through a service bus.
- Instrument services with logging and metrics to monitor performance and observe issues.
What monolith can learn from microservices?
Nowadays everyone is talking about microservices and how cool they are. That’s true, but not all business needs to move to microservices, you can still use monolith architecture and make it more fashion by applying some of the design/elements that you’ve learned from microservices architecture.
Can you use circuit breaker pattern, service isolation, event-driven and non-blocking application, cloud-native approach or zero downtime deployment in monolith architecture? Is it worth? The answer is YES!!!.
Come to this session to learn how to make your Java monolith applications attractive again by borrowing some of the principles of microservices architecture.
Software Quality Visualization
Software development process, in-house or outsourced, might be a challenge. It is important to choose a vendor or hire a team of professionals who understand that quality of source code directly impacts the overall cost of your project. You most probably want to find a vendor who knows how to reveal all the risk hidden behind low quality code. Tricky part in revealing risk and possible quality issues is that you have to know how to do this at the time of development, not when it’s already too late.
Author:
Łukasz Koczwara - Software Development Manager @ STX Next
PyconKR 2019 Lightning Talk - Let The Dogs Out on Kubernetes
This document discusses monitoring Kubernetes clusters using Datadog. It describes how to collect metrics from worker nodes, microservices, and deployments using Datadog. It also discusses challenges with monitoring conntrack states and proposes a solution using an init container to flush conntrack states periodically to avoid issues.
Migrating to Microservices
In this session Jon will pick up from last year’s popular talk and show strategies for mapping BizTalk Server and Services V1.0 integration solutions to the new BizTalk Microservices architecture. With lots of demos and real-world example you will get a great understanding of how you can move to and embrace this incredible new architecture without starting from scratch.
Are Your Microservices Naked and Afraid?
SpringOne Platform 2016
Speakers: Alan Ho; Head of Developer Programs, Apigee. Sandeep Murusupalli; Solutions Architect, Apigee.
The Cloud is a wild and dangerous place and can create seriously unpredictable traffic to your microservices. Without proper protection & monitoring, your microservices are vulnerable to hackers, self-inflicted denial of service attacks, and extremely high infrastructure costs. Furthermore, as internal data centers begin to mirror the Cloud itself, developers and architects need to rethink services monitoring and protection for both externalized microservice APIs and internal APIs. Attend this session to learn what is the right type of protection (OAuth / Throttling) and monitoring (e.g. bot monitoring) you need to put in place to avoid your microservices becoming a mega-disaster.
Deploy With Confidence
In a monolithic application, different services are developed within same project side by side.
In these kind of applications you don't need to worry about breaking the compatibility between contract interfaces since there is an invisible verifier called compiler that checks that all method calls follows the defined signature.
But in case of microservices, different services are deployed in different runtimes and using different separated networks.
In this scenario, any change on the contract of one service cannot be caught by the compiler since there is no typesafe connection between services..
Now breaking the compatibility between services can be really easy and hard to detect (maybe until your new version of the service is on production).
In this talk we are going to explore why deploying a new version of a service might break everything in microservices architecture, and how to fix with consumer-driven contracts pattern.
DevOps Days Kyiv 2019 -- Immutable everywhere // Sergii Marchenko
The document discusses the principles of immutable infrastructure where changes are made by replacing infrastructure rather than updating in-place. It recommends building new images or artifacts for software/configuration changes rather than updating servers. Docker enables this approach, and it can extend to Kubernetes clusters, AWS resources, and other infrastructure. Keys and subscriptions should also follow an immutable structure defined in code versioned with releases. While this approach requires automation and infrastructure as code, it avoids complex in-place changes and reduces downtime when deploying changes.
Container coneu2016 lab
This document outlines an agenda for a lab comparing Docker Swarm mode, Kubernetes, and Apache Mesos for container orchestration. The lab will be run either on HPE infrastructure accessed via OpenVPN or locally using Vagrant and virtual machines. Setup instructions and detailed lab steps are provided at a GitHub link. The goal is to give participants hands-on experience with each orchestrator in less than two hours without becoming an expert.
Developer Week
The document outlines 7 rules for cloud-native development according to Solomon Hykes:
1. Think in terms of services rather than servers
2. Don't reinvent the wheel and leverage existing services
3. Avoid silos and have integrated, cross-functional teams working on products
4. Don't impose unnecessary restrictions that get in the way of developers
5. Start with standardized approaches and only customize later if needed
6. Enable rapid experimentation through a service-oriented architecture
7. Ship code changes daily to maximize speed and productivity
Win Spinnaker with Winnaker - Open Source North Conf 2017
Spinnaker is an open source tool for deploying software releases to multiple cloud providers. Winnaker is a tool built by Target that helps automate common tasks when using Spinnaker like starting pipelines, getting stage details, integrating with chat tools, and troubleshooting errors. It removes company-specific code so others can contribute. Winnaker is distributed as a Docker container and makes it easy to pressure test Spinnaker and cloud environments by running multiple pipelines.
Continuous Happiness by Continuous Delivery
This document discusses techniques for continuous delivery including continuous integration, infrastructure automation, continuous deployment, monitoring and metrics. The key aspects are performing frequent releases through automated processes to reduce risk, using version control, automated testing, builds, deployment packages and tools for configuration management, deployment and monitoring.
Concurrent version management(tortoise CVS)
The document discusses concurrent version control systems (CVS) and the CVS client TortoiseCVS. It describes how CVS allows multiple developers to work on the same codebase simultaneously, managing different versions and changes. It outlines the features of CVS, including concurrent access, version tagging, and difference tracking. The document then focuses on TortoiseCVS, an open source Windows shell extension that provides a graphical interface for performing CVS functions like check-ins, check-outs, tagging, and branching. It provides instructions and screenshots for using TortoiseCVS for common version control tasks.
Riak: A friendly key/value store for the web.
Talk at DevNation Portland, July 10th, 2010.
Keynote available for download at:
http://dl.dropbox.com/u/458036/presentations/DevNation%20Portland.zip
Sources and additional information in the presenter notes.
Moving from Relational to Document Store
The document summarizes the evolution of the Guardian's content management system from early relational databases and bespoke solutions in the 1990s to modern approaches using NoSQL document databases. It describes periods relying on vendor CMS platforms, monolithic Java applications with Oracle databases, and attempts to scale with partial NoSQL solutions like Solr and memcached. The document advocates a full transition to document stores like MongoDB by migrating systems like user identity management and designing flexible JSON data models instead of relational schemas and ORM complexity.
Neuropsicologia
Este documento discute la etiología diversa de los trastornos neurológicos como accidentes cerebrovasculares, traumatismos craneoencefálicos y tumores cerebrales que provocan déficits cognitivos y de comportamiento que afectan no solo al paciente sino también a sus familiares. Además, señala que la demanda de neuropsicólogos clínicos ha crecido debido a los avances médicos que han mejorado la supervivencia de personas con lesiones graves y el aumento de la esperanza de vida, con entre
The New Guardian
This document outlines the rebuilding of the guardian.co.uk website from 1999-2007 by Stephen Dunn, head of technology strategy, and technical architects Mat Wall and Phil Wills. It notes that guardian.co.uk has been online since 1996, has over 1.5 million pages, receives 181 million pages per month and 19.7 million visitors in January 2008, and has won three Webby awards.
Social Insights on the UK Newspaper Industry
This report compares the different Twitter strategies of newspaper brands, and:
Shows how the strategies affect each audience
Examines and compares the top 10 UK newspaper brands
Voldemort Nosql
Project Voldemort is a distributed key-value store inspired by Amazon Dynamo and Memcached. It was originally developed at LinkedIn to handle high volumes of data and queries in a scalable way across multiple servers. Voldemort uses consistent hashing to partition and replicate data, vector clocks to resolve concurrent write conflicts, and a layered architecture to provide flexibility. It prioritizes performance, availability, and simplicity over more complex consistency guarantees. LinkedIn uses multiple Voldemort clusters to power various real-time services and applications.
NoSql presentation
Presentation given at NoSql EU conference describing architectures past, present & future for guardian.co.uk
Why we chose mongodb for guardian.co.uk
The document discusses the Guardian's transition from using relational databases to NoSQL databases like MongoDB. It describes the early, mid, and modern periods of the Guardian's technology stack, moving from bespoke systems to vendor CMS platforms to monolithic Java applications. Problems with scaling and complexity motivated experimenting with partial NoSQL solutions and APIs. MongoDB was ultimately chosen as it allows flexible schemas, complex queries, and easy scaling similar to relational databases but with simpler implementation. The first project to use MongoDB is replacing the user identity system stored in Oracle.
Are Content Strategists the Next Corporate Rock Stars?
Content Strategy is a lot like managing a Rock Band. You have the rock stars, the promoters, the fans, and the content (songs, image, branding). This presentation will walk you through the perils of bad content, expose you to great content, and answer the question "Are Content Strategists the Next Corporate Rock Stars?"
Functional Scala 2022 - scalajs Alexis.pdf
Scala.js is a way to build robust front-end web applications using Scala that compiles to JavaScript. It provides correctness through type safety, performance comparable to native JavaScript, and easy interoperability. The presenter's company Wiringbits uses Scala.js with libraries like Slinky and ScalablyTyped for React applications. Their Scala.js onboarding process involves assigning trainees simple tasks under a mentor to become productive with Scala and Scala.js quickly, having successfully onboarded about 10 developers through this approach.
RedisConf17 - Reddit - How We Built and Scaled r/place
r/place was a collaborative pixel art project launched by Reddit as an April Fools' joke in 2017. The developers faced challenges of only having a few days to launch before it ended, not impacting the main Reddit site, and a small team with other responsibilities. They used a scalable architecture with websockets, an event collector, Redis for pixel data, and caching at multiple levels to handle over 180,000 writes per second while keeping load times fast. Load testing was critical to identify caching and database optimizations needed to support the massive traffic.
SaltConf 2014: Safety with powertools
SaltConf 2014 keynote - Thomas Jackson, LinkedIn
Safety with Power tools
As infrastructure scales, simple tasks become increasingly difficult. For large infrastructures to be manageable, we use automation. But automation, like any power tool, comes with its own set of risks and challenges. Automation should be handled like production code, and great care should be exercised with power tools. This talk will cover how SaltStack is used at LinkedIn and offer tips and tricks for automating management with SaltStack at massive scale including a look at LinkedIn-inspired Salt features such as blacklist and pre-req states. It will also cover Salt master and minion instrumentation and a compilation of how not to use Salt.
Deccan ruby-conf-talk
1. The document discusses CRUDActions, a Ruby on Rails module created by the speaker to optimize API development by reducing duplication across controllers and allowing APIs to be highly flexible and dynamic.
2. CRUDActions handles common CRUD actions as well as search, sorting, counting, includes/excludes, and joins to optimize API development and make applying global changes easier.
3. While CRUDActions speeds up API development and creates highly optimized APIs, any changes to the module could affect all APIs, and additional testing and documentation is needed to implement it fully.
More Related Content
What's hot
Migrating to Microservices
In this session Jon will pick up from last year’s popular talk and show strategies for mapping BizTalk Server and Services V1.0 integration solutions to the new BizTalk Microservices architecture. With lots of demos and real-world example you will get a great understanding of how you can move to and embrace this incredible new architecture without starting from scratch.
Are Your Microservices Naked and Afraid?
SpringOne Platform 2016
Speakers: Alan Ho; Head of Developer Programs, Apigee. Sandeep Murusupalli; Solutions Architect, Apigee.
The Cloud is a wild and dangerous place and can create seriously unpredictable traffic to your microservices. Without proper protection & monitoring, your microservices are vulnerable to hackers, self-inflicted denial of service attacks, and extremely high infrastructure costs. Furthermore, as internal data centers begin to mirror the Cloud itself, developers and architects need to rethink services monitoring and protection for both externalized microservice APIs and internal APIs. Attend this session to learn what is the right type of protection (OAuth / Throttling) and monitoring (e.g. bot monitoring) you need to put in place to avoid your microservices becoming a mega-disaster.
Deploy With Confidence
In a monolithic application, different services are developed within same project side by side.
In these kind of applications you don't need to worry about breaking the compatibility between contract interfaces since there is an invisible verifier called compiler that checks that all method calls follows the defined signature.
But in case of microservices, different services are deployed in different runtimes and using different separated networks.
In this scenario, any change on the contract of one service cannot be caught by the compiler since there is no typesafe connection between services..
Now breaking the compatibility between services can be really easy and hard to detect (maybe until your new version of the service is on production).
In this talk we are going to explore why deploying a new version of a service might break everything in microservices architecture, and how to fix with consumer-driven contracts pattern.
DevOps Days Kyiv 2019 -- Immutable everywhere // Sergii Marchenko
The document discusses the principles of immutable infrastructure where changes are made by replacing infrastructure rather than updating in-place. It recommends building new images or artifacts for software/configuration changes rather than updating servers. Docker enables this approach, and it can extend to Kubernetes clusters, AWS resources, and other infrastructure. Keys and subscriptions should also follow an immutable structure defined in code versioned with releases. While this approach requires automation and infrastructure as code, it avoids complex in-place changes and reduces downtime when deploying changes.
Container coneu2016 lab
This document outlines an agenda for a lab comparing Docker Swarm mode, Kubernetes, and Apache Mesos for container orchestration. The lab will be run either on HPE infrastructure accessed via OpenVPN or locally using Vagrant and virtual machines. Setup instructions and detailed lab steps are provided at a GitHub link. The goal is to give participants hands-on experience with each orchestrator in less than two hours without becoming an expert.
Developer Week
The document outlines 7 rules for cloud-native development according to Solomon Hykes:
1. Think in terms of services rather than servers
2. Don't reinvent the wheel and leverage existing services
3. Avoid silos and have integrated, cross-functional teams working on products
4. Don't impose unnecessary restrictions that get in the way of developers
5. Start with standardized approaches and only customize later if needed
6. Enable rapid experimentation through a service-oriented architecture
7. Ship code changes daily to maximize speed and productivity
Win Spinnaker with Winnaker - Open Source North Conf 2017
Spinnaker is an open source tool for deploying software releases to multiple cloud providers. Winnaker is a tool built by Target that helps automate common tasks when using Spinnaker like starting pipelines, getting stage details, integrating with chat tools, and troubleshooting errors. It removes company-specific code so others can contribute. Winnaker is distributed as a Docker container and makes it easy to pressure test Spinnaker and cloud environments by running multiple pipelines.
Continuous Happiness by Continuous Delivery
This document discusses techniques for continuous delivery including continuous integration, infrastructure automation, continuous deployment, monitoring and metrics. The key aspects are performing frequent releases through automated processes to reduce risk, using version control, automated testing, builds, deployment packages and tools for configuration management, deployment and monitoring.
Concurrent version management(tortoise CVS)
The document discusses concurrent version control systems (CVS) and the CVS client TortoiseCVS. It describes how CVS allows multiple developers to work on the same codebase simultaneously, managing different versions and changes. It outlines the features of CVS, including concurrent access, version tagging, and difference tracking. The document then focuses on TortoiseCVS, an open source Windows shell extension that provides a graphical interface for performing CVS functions like check-ins, check-outs, tagging, and branching. It provides instructions and screenshots for using TortoiseCVS for common version control tasks.
What's hot (9)
DevOps Days Kyiv 2019 -- Immutable everywhere // Sergii Marchenko
DevOps Days Kyiv 2019 -- Immutable everywhere // Sergii Marchenko
Win Spinnaker with Winnaker - Open Source North Conf 2017
Win Spinnaker with Winnaker - Open Source North Conf 2017
Viewers also liked
Riak: A friendly key/value store for the web.
Talk at DevNation Portland, July 10th, 2010.
Keynote available for download at:
http://dl.dropbox.com/u/458036/presentations/DevNation%20Portland.zip
Sources and additional information in the presenter notes.
Moving from Relational to Document Store
The document summarizes the evolution of the Guardian's content management system from early relational databases and bespoke solutions in the 1990s to modern approaches using NoSQL document databases. It describes periods relying on vendor CMS platforms, monolithic Java applications with Oracle databases, and attempts to scale with partial NoSQL solutions like Solr and memcached. The document advocates a full transition to document stores like MongoDB by migrating systems like user identity management and designing flexible JSON data models instead of relational schemas and ORM complexity.
Neuropsicologia
Este documento discute la etiología diversa de los trastornos neurológicos como accidentes cerebrovasculares, traumatismos craneoencefálicos y tumores cerebrales que provocan déficits cognitivos y de comportamiento que afectan no solo al paciente sino también a sus familiares. Además, señala que la demanda de neuropsicólogos clínicos ha crecido debido a los avances médicos que han mejorado la supervivencia de personas con lesiones graves y el aumento de la esperanza de vida, con entre
The New Guardian
This document outlines the rebuilding of the guardian.co.uk website from 1999-2007 by Stephen Dunn, head of technology strategy, and technical architects Mat Wall and Phil Wills. It notes that guardian.co.uk has been online since 1996, has over 1.5 million pages, receives 181 million pages per month and 19.7 million visitors in January 2008, and has won three Webby awards.
Social Insights on the UK Newspaper Industry
This report compares the different Twitter strategies of newspaper brands, and:
Shows how the strategies affect each audience
Examines and compares the top 10 UK newspaper brands
Voldemort Nosql
Project Voldemort is a distributed key-value store inspired by Amazon Dynamo and Memcached. It was originally developed at LinkedIn to handle high volumes of data and queries in a scalable way across multiple servers. Voldemort uses consistent hashing to partition and replicate data, vector clocks to resolve concurrent write conflicts, and a layered architecture to provide flexibility. It prioritizes performance, availability, and simplicity over more complex consistency guarantees. LinkedIn uses multiple Voldemort clusters to power various real-time services and applications.
NoSql presentation
Presentation given at NoSql EU conference describing architectures past, present & future for guardian.co.uk
Why we chose mongodb for guardian.co.uk
The document discusses the Guardian's transition from using relational databases to NoSQL databases like MongoDB. It describes the early, mid, and modern periods of the Guardian's technology stack, moving from bespoke systems to vendor CMS platforms to monolithic Java applications. Problems with scaling and complexity motivated experimenting with partial NoSQL solutions and APIs. MongoDB was ultimately chosen as it allows flexible schemas, complex queries, and easy scaling similar to relational databases but with simpler implementation. The first project to use MongoDB is replacing the user identity system stored in Oracle.
Are Content Strategists the Next Corporate Rock Stars?
Content Strategy is a lot like managing a Rock Band. You have the rock stars, the promoters, the fans, and the content (songs, image, branding). This presentation will walk you through the perils of bad content, expose you to great content, and answer the question "Are Content Strategists the Next Corporate Rock Stars?"
Viewers also liked (9)
Are Content Strategists the Next Corporate Rock Stars?
Are Content Strategists the Next Corporate Rock Stars?
Similar to Redis at the guardian
Functional Scala 2022 - scalajs Alexis.pdf
Scala.js is a way to build robust front-end web applications using Scala that compiles to JavaScript. It provides correctness through type safety, performance comparable to native JavaScript, and easy interoperability. The presenter's company Wiringbits uses Scala.js with libraries like Slinky and ScalablyTyped for React applications. Their Scala.js onboarding process involves assigning trainees simple tasks under a mentor to become productive with Scala and Scala.js quickly, having successfully onboarded about 10 developers through this approach.
RedisConf17 - Reddit - How We Built and Scaled r/place
r/place was a collaborative pixel art project launched by Reddit as an April Fools' joke in 2017. The developers faced challenges of only having a few days to launch before it ended, not impacting the main Reddit site, and a small team with other responsibilities. They used a scalable architecture with websockets, an event collector, Redis for pixel data, and caching at multiple levels to handle over 180,000 writes per second while keeping load times fast. Load testing was critical to identify caching and database optimizations needed to support the massive traffic.
SaltConf 2014: Safety with powertools
SaltConf 2014 keynote - Thomas Jackson, LinkedIn
Safety with Power tools
As infrastructure scales, simple tasks become increasingly difficult. For large infrastructures to be manageable, we use automation. But automation, like any power tool, comes with its own set of risks and challenges. Automation should be handled like production code, and great care should be exercised with power tools. This talk will cover how SaltStack is used at LinkedIn and offer tips and tricks for automating management with SaltStack at massive scale including a look at LinkedIn-inspired Salt features such as blacklist and pre-req states. It will also cover Salt master and minion instrumentation and a compilation of how not to use Salt.
Deccan ruby-conf-talk
1. The document discusses CRUDActions, a Ruby on Rails module created by the speaker to optimize API development by reducing duplication across controllers and allowing APIs to be highly flexible and dynamic.
2. CRUDActions handles common CRUD actions as well as search, sorting, counting, includes/excludes, and joins to optimize API development and make applying global changes easier.
3. While CRUDActions speeds up API development and creates highly optimized APIs, any changes to the module could affect all APIs, and additional testing and documentation is needed to implement it fully.
SaltConf14 - Thomas Jackson, LinkedIn - Safety with Power Tools
As infrastructure scales, simple tasks become increasingly difficult. For large infrastructures to be manageable, we use automation. But automation, like any power tool, comes with its own set of risks and challenges. Automation should be handled like production code, and great care should be exercised with power tools. This talk will cover how SaltStack is used at LinkedIn and offer tips and tricks for automating management with SaltStack at massive scale including a look at LinkedIn-inspired Salt features such as blacklist and prereq states. It will also cover Salt master and minion instrumentation and a compilation of how not to use Salt.
The dev ops code has no servers
The document discusses the benefits of serverless functions for DevOps workloads. It provides examples of five use cases for serverless functions, including a canonical name redirector, email-to-chat gateway, MySQL slow query logger, CloudTrail elastic gateway, and Markov Twitter bots. It emphasizes that serverless functions allow workloads to avoid idle costs and dependencies while taking advantage of event-driven integration with cloud platforms.
SaltConf 2015: Salt stack at web scale: Better, Stronger, Faster
This talk will discuss best practices for scaling SaltStack from thousands to hundreds of thousands of minions. But the devil is in the details and how do you scale without losing performance and making sure it all works? At LinkedIn we've learned some valuable lessons as we've grown our SaltStack footprint. We'll discuss how to run SaltStack, how to not run SaltStack, and how we've contributed to the Salt project to help make it better, stronger and faster.
Youtube: https://www.youtube.com/watch?v=qjFOY-QrW_k
Intro to Sails.js
Sails.js is a realtime MVC framework for Node.js that provides conventions and structures for building applications. It includes features like a lightweight ORM, policies for authentication and authorization, and integrates Socket.io for realtime functionality. Sails.js aims to simplify building realtime apps by handling websockets and HTTP requests together and providing patterns for application structure. The framework is actively maintained by an open source community.
Ruby codebases in an entropic universe
The document discusses how the entropy of Ruby codebases increases over time if changes are not limited, making future changes more difficult. It advocates for writing specs to establish confidence in code and observing trends in metrics like code coverage, complexity, and churn to catch signs of rising entropy early. Sticking to conventions but knowing when to deviate, and focusing on principles over mechanics can help limit a codebase's entropy.
Viridians on Rails
The document discusses software as a service (SAAS) and why the company Viridian chose to use the Ruby on Rails web application framework. It notes that Rails allows for lower entry costs than other options due to reduced server maintenance needs and flexibility. It also summarizes some key advantages of Rails like its convention over configuration approach and support for modern technologies. The document provides resources for learning Rails including dev environments, tutorials, and open source projects to review.
Turn you Java EE Monoliths into Microservices with WildFly Swarm
This document provides an overview and introduction to WildFly Swarm, which allows developers to decompose a WildFly application server into smaller components called "fractions" to right-size the runtime for microservices. It discusses concepts like fractions, how to get started with a basic WildFly Swarm application, and advanced configuration options. The presentation encourages leveraging existing Java EE expertise while building on mature technologies to develop microservices.
Evolving toward devops through transaction centric monitoring
I gave this presentation at a London FITE club meeting on March 5th 2013
http://eventful.com/events/fite-club-peter-holditch-appdynamics-evolving-toward-devop-/E0-001-054955463-8?utm_source=apis&utm_medium=apim&utm_campaign=apic
Hadoop Performance at LinkedIn
This is part of a presentation I did at Intel a month or so ago. Some of the content has been removed due to NDA, etc.
Connect js nodejs_devops_shubhra
DevOps for Node.js - Debugging, Triage, CPU profiling, Heap Analysis, Monitoring, Scaling and fix it tech for Node.js applications using open-source tooling. Presented at Connect.js conference in Atlanta
Scaling Instagram
This document summarizes Mike Krieger's talk on scaling Instagram from its early days with 2 engineers to supporting over 30 million users. Some key points include: starting simply with Django and PostgreSQL; adopting Redis for caching and queuing; implementing database sharding in PostgreSQL as user growth increased database size; focusing on simplicity, monitoring, and nimble iteration; and scaling components individually while maintaining a minimal overall architecture. Krieger emphasizes optimizing for operational simplicity and solving problems with existing tools before building custom solutions.
89025069 mike-krieger-instagram-at-the-airbnb-tech-talk-on-scaling-instagram
This document summarizes Mike Krieger's talk on scaling Instagram from its early days with 2 engineers to supporting over 30 million users. Some key points include: starting simply with Django and PostgreSQL; adopting Redis for caching and queuing; implementing database sharding in PostgreSQL as user growth increased database size; using a variety of tools like Nginx, HAProxy, Memcached and monitoring with Munin and StatsD; and focusing on simplicity, instrumentation, and nimble iteration to adapt as needs changed.
Introduction to Apache Spark
Introduction to Apache Spark Workshop at Lambda World 2015 on October 23th and 24th, 2015, celebrated in Cádiz. Speakers: @fperezp and @juanpedromoreno
Github Repo: https://github.com/47deg/spark-workshop
Conference slides: MySQL Cluster Performance Tuning
This presentation goes through performance tuning basics in MySQL Cluster.
It also covers the new parameters and status variables of MySQL Cluster 7.2 to determine issues with e.g disk data performance and query (join) performance.
Build a Deep Learning App with Tensorflow & Redis by Jayesh Ahire and Sherin ...
Build a deep learning app with tensor flow and Redis by Jayesh Ahire and Sherin Thomas of Tensorwerk - Redis Day Bangalore 2020
Rails App performance at the limit - Bogdan Gusiev
The document discusses optimizing Rails application performance. It begins by examining where Rails may be slow, such as ActiveRecord and Action Controller. The author advocates for effective database schema as the best general optimization technique. Specific optimizations should target problematic parts of the application and may include avoiding ActiveRecord, conditional GET requests, rewriting controllers in Rack, and introducing caching. The goals of optimization are to improve throughput, reduce infrastructure costs, and make targeted improvements where needed.
Similar to Redis at the guardian (20)
RedisConf17 - Reddit - How We Built and Scaled r/place
RedisConf17 - Reddit - How We Built and Scaled r/place
SaltConf14 - Thomas Jackson, LinkedIn - Safety with Power Tools
SaltConf14 - Thomas Jackson, LinkedIn - Safety with Power Tools
SaltConf 2015: Salt stack at web scale: Better, Stronger, Faster
SaltConf 2015: Salt stack at web scale: Better, Stronger, Faster
Turn you Java EE Monoliths into Microservices with WildFly Swarm
Turn you Java EE Monoliths into Microservices with WildFly Swarm
Evolving toward devops through transaction centric monitoring
Evolving toward devops through transaction centric monitoring
89025069 mike-krieger-instagram-at-the-airbnb-tech-talk-on-scaling-instagram
89025069 mike-krieger-instagram-at-the-airbnb-tech-talk-on-scaling-instagram
Conference slides: MySQL Cluster Performance Tuning
Conference slides: MySQL Cluster Performance Tuning
Build a Deep Learning App with Tensorflow & Redis by Jayesh Ahire and Sherin ...
Build a Deep Learning App with Tensorflow & Redis by Jayesh Ahire and Sherin ...
Rails App performance at the limit - Bogdan Gusiev
Rails App performance at the limit - Bogdan Gusiev
Recently uploaded
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI, built on the robust SAP Business Technology Platform (SAP BTP), is the latest and most advanced version of our AI development, reaffirming our commitment to delivering top-tier AI solutions. Skybuffer AI harnesses all the innovative capabilities of the SAP BTP in the AI domain, from Conversational AI to cutting-edge Generative AI and Retrieval-Augmented Generation (RAG). It also helps SAP customers safeguard their investments into SAP Conversational AI and ensure a seamless, one-click transition to SAP Business AI.
With Skybuffer AI, various AI models can be integrated into a single communication channel such as Microsoft Teams. This integration empowers business users with insights drawn from SAP backend systems, enterprise documents, and the expansive knowledge of Generative AI. And the best part of it is that it is all managed through our intuitive no-code Action Server interface, requiring no extensive coding knowledge and making the advanced AI accessible to more users.
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Best 20 SEO Techniques To Improve Website Visibility In SERP
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Generating privacy-protected synthetic data using Secludy and Milvus
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
TrustArc Webinar - 2024 Global Privacy Survey
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on integration of Salesforce with Bonterra Impact Management.
Interested in deploying an integration with Salesforce for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Inconsistent user experience and siloed data, high costs, and changing customer expectations – Citizens Bank was experiencing these challenges while it was attempting to deliver a superior digital banking experience for its clients. Its core banking applications run on the mainframe and Citizens was using legacy utilities to get the critical mainframe data to feed customer-facing channels, like call centers, web, and mobile. Ultimately, this led to higher operating costs (MIPS), delayed response times, and longer time to market.
Ever-changing customer expectations demand more modern digital experiences, and the bank needed to find a solution that could provide real-time data to its customer channels with low latency and operating costs. Join this session to learn how Citizens is leveraging Precisely to replicate mainframe data to its customer channels and deliver on their “modern digital bank” experiences.
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Free A4 downloadable and printable Cyber Security, Social Engineering Safety and security Training Posters . Promote security awareness in the home or workplace. Lock them Out From training providers datahops.com
Trusted Execution Environment for Decentralized Process Mining
Presentation of the paper "Trusted Execution Environment for Decentralized Process Mining" given during the CAiSE 2024 Conference in Cyprus on June 7, 2024.
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
This case study explores designing a scalable e-commerce platform, covering key requirements, system components, and best practices.
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
June Patch Tuesday
Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.
Azure API Management to expose backend services securely
How to use Azure API Management to expose backend service securely
Fueling AI with Great Data with Airbyte Webinar
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
Recently uploaded (20)
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Skybuffer AI: Advanced Conversational and Generative AI Solution on SAP Busin...
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
System Design Case Study: Building a Scalable E-Commerce Platform - Hiike
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Azure API Management to expose backend services securely
Azure API Management to expose backend services securely
Redis at the guardian
- 1. REDIS AT GUARDIAN.CO.UK Michael Brunton-Spall - @bruntonspall
- 4. Before Redis Originally selected a random article to start reviewing SQL = ORDER BY RAND() LIMIT 1 Inefficient and horrible
- 5. If in doubt, ask
- 6. With Redis Maintain redis set of all available pages per assignment Maintain redis set of reviewed pages per assignment SDIFFSTORE the two sets Use SRANDMEMBER on the stored set
- 7. SDIFFSTORE
- 8. SRANDMEMBER
- 9. SOULMATES
- 10. Soulmates
- 11. What do we use it for? github.com/simonw/django-redis-monitor Written internally by Simon Willison and Daniel Vydra Counts requests per second / SQLops per second in django Management switches
- 12. Management Switches Whats the problem? We’re basically a java shop In Java, management switches are easy In python not so much
- 13. How Java Works
- 14. How Python Works
- 15. Soulmates Architecture Django Django Frontend Frontend Redis Redis Scala API Scala API Solr Slave Solr Slave WebApp WebApp Solr Master MongoDB MongoDB
- 16. Switches must be externalised Normally we’d use the database but Soulmates django app doesn’t have a database So we used redis
- 18. Implementation
- 19. Redis Switches Hit a URL on any server, any process to toggle switch All processes are able to check that switch later Redis processes are local to the machine
Editor's Notes
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n
- \n