The document describes LinkedIn's use of Couchbase for caching and the automation of Couchbase clusters using SaltStack. Key points:
- LinkedIn uses Couchbase to store cached data for read scaling across hundreds of clusters totaling thousands of servers.
- Automation is achieved using SaltStack's states, pillars and grains to configure Couchbase installation, cluster expansion/reduction, and uninstall remotely.
- A Couchbase execution module and Salt runners implement cluster operations like setup, expansion, reduction through the REST API and CLI while providing output to the user.
This webcast covers the theoretical introduction to Web Farms and how to build Drupal Web Farms with IIS. Don't miss the second part of the webcast (also part of this series) where a full demo on creating Drupal Web Farms with 4 virtual machines will be presented. If you are already familiar with Web Farms, Application Request Router, Web Farm Framework you can skip to part 2. Otherwise, this webcast is highly recommended and propaedeutic to grasp all the basic knowledge that you might need later.
Streamline Hadoop DevOps with Apache AmbariJayush Luniya
Ambari talk at Hadoop Summit, Tokyo 2016
Abstract
Apache Ambari has become an indispensable tool for operating Hadoop clusters from as small as 10s of nodes to 1000s of nodes. Ambari’s deep knowledge of the Hadoop stack allows it to deploy a cluster within minutes and manage the entire lifecycle: scaling, security, upgrades, and more. This talk will cover the central features important to cluster operators and the latest innovations from the community. We will discuss automatically deploying clusters with Blueprints, adding custom services, scaling the number of hosts as the data needs grow, adding High Availability for critical services, securing with MIT kerberos, and upgrading the Hadoop stack with features like Rolling & Express Upgrade. More advanced users will also be interested in using Ambari’s powerful REST API to automate workflows. For users and data scientists, Ambari provides LDAP sync, Role-Based Access Control to handle user permissions, and a framework to host Ambari Views such as as the newly added Views for Hive, Oozie, Capacity Scheduler, Tez, Storm, and Zeppelin. Lastly, we will cover how to monitor the health of the cluster via Alerts and troubleshoot problems by using new features like LogSearch and Ambari Metrics Systems integrated with Grafana UI.
Accumulo Summit 2014: Benchmarking Accumulo: How Fast Is Fast?Accumulo Summit
Speaker: Mike Drob
Apache Accumulo has long held a reputation for enabling high-throughput operations in write-heavy workloads. In this talk, we use the Yahoo! Cloud Serving Benchmark (YCSB) to put real numbers on Accumulo performance. We then compare these numbers to previous versions, to other databases, and wrap up with a discussion of parameters that can be tweaked to improve them.
This webcast covers the theoretical introduction to Web Farms and how to build Drupal Web Farms with IIS. Don't miss the second part of the webcast (also part of this series) where a full demo on creating Drupal Web Farms with 4 virtual machines will be presented. If you are already familiar with Web Farms, Application Request Router, Web Farm Framework you can skip to part 2. Otherwise, this webcast is highly recommended and propaedeutic to grasp all the basic knowledge that you might need later.
Streamline Hadoop DevOps with Apache AmbariJayush Luniya
Ambari talk at Hadoop Summit, Tokyo 2016
Abstract
Apache Ambari has become an indispensable tool for operating Hadoop clusters from as small as 10s of nodes to 1000s of nodes. Ambari’s deep knowledge of the Hadoop stack allows it to deploy a cluster within minutes and manage the entire lifecycle: scaling, security, upgrades, and more. This talk will cover the central features important to cluster operators and the latest innovations from the community. We will discuss automatically deploying clusters with Blueprints, adding custom services, scaling the number of hosts as the data needs grow, adding High Availability for critical services, securing with MIT kerberos, and upgrading the Hadoop stack with features like Rolling & Express Upgrade. More advanced users will also be interested in using Ambari’s powerful REST API to automate workflows. For users and data scientists, Ambari provides LDAP sync, Role-Based Access Control to handle user permissions, and a framework to host Ambari Views such as as the newly added Views for Hive, Oozie, Capacity Scheduler, Tez, Storm, and Zeppelin. Lastly, we will cover how to monitor the health of the cluster via Alerts and troubleshoot problems by using new features like LogSearch and Ambari Metrics Systems integrated with Grafana UI.
Accumulo Summit 2014: Benchmarking Accumulo: How Fast Is Fast?Accumulo Summit
Speaker: Mike Drob
Apache Accumulo has long held a reputation for enabling high-throughput operations in write-heavy workloads. In this talk, we use the Yahoo! Cloud Serving Benchmark (YCSB) to put real numbers on Accumulo performance. We then compare these numbers to previous versions, to other databases, and wrap up with a discussion of parameters that can be tweaked to improve them.
The following depicts the automatic automatic migration of administration and managed server in case of failure
This concept is useful very useful in site failover as well as managed server failover
We have used the Virtual IP and Virtual Hostname concept
Speaker: Jean-Daniel Cryans (Cloudera)
HBase Replication has come a long way since its inception in HBase 0.89 almost four years ago. Today, master-master and cyclic replication setups are supported; many bug fixes and new features like log compression, per-family peers configuration, and throttling have been added; and a major refactoring has been done. This presentation will recap the work done during the past four years, present a few use cases that are currently in production, and take a look at the roadmap.
SQL Server Alwayson for SharePoint HA/DR Step by Step GuideLars Platzdasch
SQL Server Alwayson for Sharepoint HA/DR SQL Konferenz 2017
-What is SQL Server AlwaysOn?
-AlwaysOn Failover Clustering
-AlwaysOn Availability Groups
-Why AlwaysOn Availability Groups for SharePoint?
-Requirements and Prerequisites
-Step by Step guide to implementing AlwaysOn Availability Groups
Demonstration
lessons learned
May 2013 HUG: Apache Sqoop 2 - A next generation of data transfer toolsYahoo Developer Network
Apache Sqoop 2 is the next generation of the massively successful open source tool designed to transfer data between traditional SQL databases and warehouses into Apache Hadoop. Sqoop 2 is designed as a client-server system with a repository which stores connection and job information. Sqoop 2 is designed to support secure job submission and multiple different roles for users. In this talk, we will discuss the issues users faced in Sqoop 1, and the design of Sqoop 2 and how the issues faced in Sqoop 1 are being handled in Sqoop 2.
Presenter(s): Hari Shreedharan, Software Engineer, Cloudera
Hadoop World 2011: Hadoop and Performance - Todd Lipcon & Yanpei Chen, ClouderaCloudera, Inc.
Performance is a thing that you can never have too much of. But performance is a nebulous concept in Hadoop. Unlike databases, there is no equivalent in Hadoop to TPC, and different use cases experience performance differently. This talk will discuss advances on how Hadoop performance is measured and will also talk about recent and future advances in performance in different areas of the Hadoop stack.
Trainingicon provides Oracle Rac Dba Online Training along with Oracle Apps, Dba, Technical, Scm, and Siebel Trainings LiveProjectswith Real time 13+ Experts
AUDWC 2016 - Using SQL Server 20146 AlwaysOn Availability Groups for SharePoi...Michael Noel
SQL Server 2016 provides for unprecedented high availability and disaster recovery options for SharePoint farms in the form of AlwaysOn Availability Groups. Using this new technology, SharePoint architects can provide for near-instant failover at the data tier, without the risk of any data loss. In addition, the latest version of this technology, available with SQL Server 2016, allows for replicas of SharePoint databases to be stored in the cloud in Microsoft’s Azure cloud offering. This technology, which will be demonstrated live, completely changes the data tier design options for SharePoint and revolutionises high availability options for a farm. This session covers in step-by-step detail the exact configuration required to enable this functionality for a SharePoint 2013 farm, based on the best practices, tips and tricks, and real-world experience of the presenter in deploying this technology in production.
Understand the differences between SQL AlwaysOn options, and determine the requirements to deploy the technologies
Examine how SQL Server 2016 AlwaysOn Availability Groups can provide aggressive Service Level Agreements (SLAs) with a Recovery Point Objective (RPO) of zero and a Recovery Time Objective (RTO) of a few seconds.
See the exact steps required to enable SQL Server 2016 AlwaysOn Availability Groups for a SharePoint 2013 On-Premises environment, including options for storing replicas in Microsoft’s Azure cloud service.
Unbreakable SharePoint 2013 with SQL Server Always On Availability Groups (HA...serge luca
SharePoint 2013 High Availability and Disaster Recovery with SQL Server Always On Availability Groups (HA and DR) - SharePoint Saturday Helsinki-Serge Luca (SharePoint MVP) et Isabelle Van Campenhoudt(SQ Server MVP); ShareQL, Belgium
April 2016 HUG: The latest of Apache Hadoop YARN and running your docker apps...Yahoo Developer Network
Apache Hadoop YARN is a modern resource-management platform that handles resource scheduling, isolation and multi-tenancy for a variety of data processing engines that can co-exist and share a single data-center in a cost-effective manner.
In the first half of the talk, we are going to give a brief look into some of the big efforts cooking in the Apache Hadoop YARN community.
We will then dig deeper into one of the efforts - supporting Docker runtime in YARN. Docker is an application container engine that enables developers and sysadmins to build, deploy and run containerized applications. In this half, we'll discuss container runtimes in YARN, with a focus on using the DockerContainerRuntime to run various docker applications under YARN. Support for container runtimes (including the docker container runtime) was recently added to the Linux Container Executor (YARN-3611 and its sub-tasks). We’ll walk through various aspects of running docker containers under YARN - resource isolation, some security aspects (for example container capabilities, privileged containers, user namespaces) and other work in progress features like image localization and support for different networking modes.
Speakers:
Vinod Kumar Vavilapalli is the Hadoop YARN and MapReduce guy at Hortonworks. He is a long term Hadoop contributor at Apache, Hadoop committer and a member of the Apache Hadoop PMC. He has a Bachelors degree from Indian Institute of Technology Roorkee in Computer Science and Engineering. He has been working on Hadoop for nearly 9 years and he still has fun doing it. Straight out of college, he joined the Hadoop team at Yahoo! Bangalore, before Hortonworks happened. He is passionate about using computers to change the world for better, bit by bit.
Sidharta Seethana is a software engineer at Hortonworks. He works on the YARN team, focussing on bringing new kinds of workloads to YARN. Prior to joining Hortonworks, Sidharta spent 10 years at Yahoo! Inc., working on a variety of large scale distributed systems for core platforms/web services, search and marketplace properties, developer network and personalization.
Couchbase Connect 2016: Monitoring Production Deployments The Tools – LinkedInMichael Kehoe
Good monitoring can be the difference between a great night's sleep or hearing your phone go off at 2:37 a.m. because of a production outage. Couchbase Server provides a large number of metrics which can be overwhelming if you do not know the critical things to focus on or how to expose that information to your monitoring system. In this talk we will look at example production incidents, going in depth around specific things to monitor, and how this information can be used to find issues, work out root cause, and discover trends.
LinkedIn serves traffic for its 467 million members from four data centers and multiple PoPs spread geographically around the world. Serving live traffic from from many places at the same time has taken us from a disaster recovery model to a disaster avoidance model where we can take an unhealthy data center or PoP out of rotation and redistribute its traffic to a healthy one within minutes, with virtually no visible impact to users. The geographical distribution of our infrastructure also allows us to optimize the end-user's experience by geo routing users to the best possible PoP and datacenter.
This talk provide details on how LinkedIn shifts traffic between its PoPs and data centers to provide the best possible performance and availability for its members. We will also touch on the complexities of performance in APAC, how IPv6 is helping our members and how LinkedIn stress tests data centers verify its disaster recovery capabilities.
Using SaltStack to Auto Triage and Remediate Production SystemsMichael Kehoe
LinkedIn created an auto-remediation system named Nurse which leverages SaltStack and the CherryPy API to auto-triage and remediate issues with production systems. See how LinkedIn uses SaltStack with Nurse in its production environment and learn how to architect your own auto-triage and remediation system.
The following depicts the automatic automatic migration of administration and managed server in case of failure
This concept is useful very useful in site failover as well as managed server failover
We have used the Virtual IP and Virtual Hostname concept
Speaker: Jean-Daniel Cryans (Cloudera)
HBase Replication has come a long way since its inception in HBase 0.89 almost four years ago. Today, master-master and cyclic replication setups are supported; many bug fixes and new features like log compression, per-family peers configuration, and throttling have been added; and a major refactoring has been done. This presentation will recap the work done during the past four years, present a few use cases that are currently in production, and take a look at the roadmap.
SQL Server Alwayson for SharePoint HA/DR Step by Step GuideLars Platzdasch
SQL Server Alwayson for Sharepoint HA/DR SQL Konferenz 2017
-What is SQL Server AlwaysOn?
-AlwaysOn Failover Clustering
-AlwaysOn Availability Groups
-Why AlwaysOn Availability Groups for SharePoint?
-Requirements and Prerequisites
-Step by Step guide to implementing AlwaysOn Availability Groups
Demonstration
lessons learned
May 2013 HUG: Apache Sqoop 2 - A next generation of data transfer toolsYahoo Developer Network
Apache Sqoop 2 is the next generation of the massively successful open source tool designed to transfer data between traditional SQL databases and warehouses into Apache Hadoop. Sqoop 2 is designed as a client-server system with a repository which stores connection and job information. Sqoop 2 is designed to support secure job submission and multiple different roles for users. In this talk, we will discuss the issues users faced in Sqoop 1, and the design of Sqoop 2 and how the issues faced in Sqoop 1 are being handled in Sqoop 2.
Presenter(s): Hari Shreedharan, Software Engineer, Cloudera
Hadoop World 2011: Hadoop and Performance - Todd Lipcon & Yanpei Chen, ClouderaCloudera, Inc.
Performance is a thing that you can never have too much of. But performance is a nebulous concept in Hadoop. Unlike databases, there is no equivalent in Hadoop to TPC, and different use cases experience performance differently. This talk will discuss advances on how Hadoop performance is measured and will also talk about recent and future advances in performance in different areas of the Hadoop stack.
Trainingicon provides Oracle Rac Dba Online Training along with Oracle Apps, Dba, Technical, Scm, and Siebel Trainings LiveProjectswith Real time 13+ Experts
AUDWC 2016 - Using SQL Server 20146 AlwaysOn Availability Groups for SharePoi...Michael Noel
SQL Server 2016 provides for unprecedented high availability and disaster recovery options for SharePoint farms in the form of AlwaysOn Availability Groups. Using this new technology, SharePoint architects can provide for near-instant failover at the data tier, without the risk of any data loss. In addition, the latest version of this technology, available with SQL Server 2016, allows for replicas of SharePoint databases to be stored in the cloud in Microsoft’s Azure cloud offering. This technology, which will be demonstrated live, completely changes the data tier design options for SharePoint and revolutionises high availability options for a farm. This session covers in step-by-step detail the exact configuration required to enable this functionality for a SharePoint 2013 farm, based on the best practices, tips and tricks, and real-world experience of the presenter in deploying this technology in production.
Understand the differences between SQL AlwaysOn options, and determine the requirements to deploy the technologies
Examine how SQL Server 2016 AlwaysOn Availability Groups can provide aggressive Service Level Agreements (SLAs) with a Recovery Point Objective (RPO) of zero and a Recovery Time Objective (RTO) of a few seconds.
See the exact steps required to enable SQL Server 2016 AlwaysOn Availability Groups for a SharePoint 2013 On-Premises environment, including options for storing replicas in Microsoft’s Azure cloud service.
Unbreakable SharePoint 2013 with SQL Server Always On Availability Groups (HA...serge luca
SharePoint 2013 High Availability and Disaster Recovery with SQL Server Always On Availability Groups (HA and DR) - SharePoint Saturday Helsinki-Serge Luca (SharePoint MVP) et Isabelle Van Campenhoudt(SQ Server MVP); ShareQL, Belgium
April 2016 HUG: The latest of Apache Hadoop YARN and running your docker apps...Yahoo Developer Network
Apache Hadoop YARN is a modern resource-management platform that handles resource scheduling, isolation and multi-tenancy for a variety of data processing engines that can co-exist and share a single data-center in a cost-effective manner.
In the first half of the talk, we are going to give a brief look into some of the big efforts cooking in the Apache Hadoop YARN community.
We will then dig deeper into one of the efforts - supporting Docker runtime in YARN. Docker is an application container engine that enables developers and sysadmins to build, deploy and run containerized applications. In this half, we'll discuss container runtimes in YARN, with a focus on using the DockerContainerRuntime to run various docker applications under YARN. Support for container runtimes (including the docker container runtime) was recently added to the Linux Container Executor (YARN-3611 and its sub-tasks). We’ll walk through various aspects of running docker containers under YARN - resource isolation, some security aspects (for example container capabilities, privileged containers, user namespaces) and other work in progress features like image localization and support for different networking modes.
Speakers:
Vinod Kumar Vavilapalli is the Hadoop YARN and MapReduce guy at Hortonworks. He is a long term Hadoop contributor at Apache, Hadoop committer and a member of the Apache Hadoop PMC. He has a Bachelors degree from Indian Institute of Technology Roorkee in Computer Science and Engineering. He has been working on Hadoop for nearly 9 years and he still has fun doing it. Straight out of college, he joined the Hadoop team at Yahoo! Bangalore, before Hortonworks happened. He is passionate about using computers to change the world for better, bit by bit.
Sidharta Seethana is a software engineer at Hortonworks. He works on the YARN team, focussing on bringing new kinds of workloads to YARN. Prior to joining Hortonworks, Sidharta spent 10 years at Yahoo! Inc., working on a variety of large scale distributed systems for core platforms/web services, search and marketplace properties, developer network and personalization.
Couchbase Connect 2016: Monitoring Production Deployments The Tools – LinkedInMichael Kehoe
Good monitoring can be the difference between a great night's sleep or hearing your phone go off at 2:37 a.m. because of a production outage. Couchbase Server provides a large number of metrics which can be overwhelming if you do not know the critical things to focus on or how to expose that information to your monitoring system. In this talk we will look at example production incidents, going in depth around specific things to monitor, and how this information can be used to find issues, work out root cause, and discover trends.
LinkedIn serves traffic for its 467 million members from four data centers and multiple PoPs spread geographically around the world. Serving live traffic from from many places at the same time has taken us from a disaster recovery model to a disaster avoidance model where we can take an unhealthy data center or PoP out of rotation and redistribute its traffic to a healthy one within minutes, with virtually no visible impact to users. The geographical distribution of our infrastructure also allows us to optimize the end-user's experience by geo routing users to the best possible PoP and datacenter.
This talk provide details on how LinkedIn shifts traffic between its PoPs and data centers to provide the best possible performance and availability for its members. We will also touch on the complexities of performance in APAC, how IPv6 is helping our members and how LinkedIn stress tests data centers verify its disaster recovery capabilities.
Using SaltStack to Auto Triage and Remediate Production SystemsMichael Kehoe
LinkedIn created an auto-remediation system named Nurse which leverages SaltStack and the CherryPy API to auto-triage and remediate issues with production systems. See how LinkedIn uses SaltStack with Nurse in its production environment and learn how to architect your own auto-triage and remediation system.
Reducing MTTR and False Escalations: Event Correlation at LinkedInMichael Kehoe
LinkedIn’s production stack is made up of over 900 applications and over 2200 internal API’s. With any given application having many interconnected pieces, it is difficult to escalate to the right person in a timely manner.
In order to combat this, LinkedIn built an Event Correlation Engine that monitors service health and maps dependencies between services to correctly escalate to the SRE’s who own the unhealthy service.
We’ll discuss the approach we used in building a correlation engine and how it has been used at LinkedIn to reduce incident impact and provide better quality of life to LinkedIn’s oncall engineers.
Configuration management and orchestration with SaltAnirban Saha
An overview and details about using Salt, the configuration management and orchestrating tool from SaltStack. Presented at LinuxCon and CloudOpen Europe 2014, Dusseldorf.
Software-defined Datacenter Maintenance - No More Sleepless Nights and Long W...SUSE
how SUSE Manager can help you to make maintenance easier. Meaning not sitting behind your monitor in the weekend of in the middle of the night. What would be easier if you get a mail telling you when something failed. The complete maintenance should be automated. Video on https://youtu.be/_-cGTAlvN3Q
Staying Ahead of the Curve with Spring and Cassandra 4VMware Tanzu
SpringOne 2020
Staying Ahead of the Curve with Spring and Cassandra 4
Mark Paluch, Spring Data Project Lead at VMware
Alexandre Dutra, Technical Manager at DataStax
Staying Ahead of the Curve with Spring and Cassandra 4 (SpringOne 2020)Alexandre Dutra
Spring and Cassandra are two of the leading technologies for building cloud native applications. In this talk by the project leads for Spring Data and the Cassandra Java Driver, we’ll cover the recent improvements in the latest and greatest versions of Spring Boot, Spring Data Cassandra, Cassandra 4.0 and the Cassandra Java driver. Whether you’re a novice, intermediate, or expert developer, this content will help you get started or migrate your existing application to the latest innovations. We’ll illustrate these new concepts with code samples and snippets that you can find on GitHub to help you get things done faster with these tools.
SaltConf14 - Ben Cane - Using SaltStack in High Availability EnvironmentsSaltStack
An overview on the benefits and best practices of using SaltStack for consistency and automation in highly available enterprise environments such as financial services.
Movile Internet Movel SA: A Change of Seasons: A big move to Apache CassandraDataStax Academy
A few years ago, processing large volumes of data was an exclusive problem of big companies. Nowadays, technological advancement allows people to be connected with each other all the time, generating and consuming large amounts of data.
In the challenge to follow Movile's exponential growth and increasing volume of information, we soon realized that traditional relational database and data analysis solutions were no longer a good fit to solve new order issues. Therefore, we present Movile's 'Change Of Seasons', a use case on adopting Apache Cassandra as a solution for critical high-performance distributed systems.
Cassandra Summit 2015 - A Change of SeasonsEiti Kimura
A CHANGE OF SEASONS: A big move to Apache Cassandra!
This is an extended version of the material presented at Cassandra Summit 2015 - Santa Clara - California - USA.
In this presentation I will show you 3 moves, use cases, that constitute our Big Move to Apache Cassandra @Movile.
Walking through relational model to NoSQL solution, hybrid platforms and a staggering cost reduction and throughput increase.
Choosing to migrate to Kubernetes can be a tough decision, and even tougher to execute. We at Kash Corp took the plunge just over a year ago with Kubernetes 1.2, and haven't looked back. This talk will detail some of our solutions to dealing with Configuration Management, Continuous Delivery, Monitoring, Maintenance, as well as talk about mistakes, frustrations and lessons learned along the way, and where we're going next.
WinOps Conf 2016 - Ed Wilson - Configuration Management with Azure DSCWinOps Conf
Configuration management at scale, even with PowerShell and PowerShell DSC, can quickly become complicated, error-prone, and unruly. The new Desired State Configuration (DSC) feature of Azure Automation, in the Microsoft’s Operations Management Suite, provides a solution - a central, secure location for all your PowerShell DSC items and reports, that is scalable, reliable, and highly-available. Come learn how it can transform configuration management across your organization, using the PowerShell tools and knowledge you already have.
Cloud providers like Amazon or Goggle have great user experience to create and manage PaaS and IaaS services. But is it possible to reproduce same experience and flexibility locally, in on premise datacenter? This talk describes success story of creation private cloud based on DC/OS cluster. It is used to host and share different services like hadoop or kafka for development teams, dynamically manage services and resource pools with GKE integration.
(ARC402) Deployment Automation: From Developers' Keyboards to End Users' Scre...Amazon Web Services
Some of the best businesses today are deploying their code dozens of times a day. How? By making heavy use of automation, smart tools, and repeatable patterns to get process out of the way and keep the workflow moving. Come to this session to learn how you can do this too, using services such as AWS OpsWorks, AWS CloudFormation, Amazon Simple Workflow Service, and other tools. We'll discuss a number of different deployment patterns, and what aspects you need to focus on when working toward deployment automation yourself.
Openstack Rally - Benchmark as a Service. Openstack Meetup India. Ananth/Rahul.Rahul Krishna Upadhyaya
Slide deck used at the presentation at Openstack India Meetup on 01/March 2014 at Netapp, Bangalore. Slide talks about installation and use of Rally and its scope to benchmark and measure performance. There is little on how to install Cisco Openstack as a All in One setup.
Deploying to and Configuring WebSphere Application Server with UrbanCode DeployIBM DevOps
Integrating middleware configuration into your application delivery lifecycle can be difficult and usually requires painful manual processes and constant surveillance.
But, there is hope! IBM UrbanCode Deploy has a new and improved middleware configuration plugin for WebSphere Application Server that provides automated updates to WebSphere as part of the application deployment process. Instead of wrestling with manual changes, join us in this session to learn how this plugin can help you update, manage and configure multiple WebSphere instances automatically and automate application deployments on top every time.
Similar to Couchbase Orchestration and Scaling a Caching Infrastructure At LinkedIn. (20)
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Connector Corner: Automate dynamic content and events by pushing a button
Couchbase Orchestration and Scaling a Caching Infrastructure At LinkedIn.
1. Couchbase Orchestration and Scaling a Caching
Infrastructure at LinkedIn
Issa Fattah
Senior SRE for New York Engineering
Member of Couchbase Virtual Team
April 20th, 2016
2. Who We Are
Couchbase supported by Virtual Team
– 10 SREs
– 2 Software Engineers
– Sponsoring by a Director
– 5-90% of their time to supporting Couchbase
– In addition to their day to day responsibilities
Everyone can and is encouraged to contribute
3. Overview
LinkedIn Story
Why Couchbase?
Development and Operations
Clusters and Numbers
Automation Use Cases
Salt Stack
– States, Pillars, Grains, Runner, Execution Module
Automation Output
– Caveats/Improvements
Conclusion
4. The LinkedIn Story
Founded in 2002
Grown into the worlds largest professional social
media network
Offices in 24 countries, 30 cities around the world
Available in 24 languages
Revenue of $862M in Q4 2015
5. The LinkedIn Story
Growth in Site Features
– Member Profiles,
Connections, and Sharing
– Post and apply for Jobs
– LinkedIn Groups and
Company Pages
– Premium tools for hiring,
marketing, and sales
Growth in Internet traffic
– Billions of page hits per
day
– Global, round the clock
traffic
Growth in Audience
– 400+M members
– 3M+ company pages
– 2.1M+ groups
6. The LinkedIn Story
Difficult for storage systems to keep up
“Read-scaling”
– Store data in cache memory
– Replicate entire databases
– Temporary data such as for de-duping
– Memcached, EHCache, Custom
Infrastructure around storage systems
– Cache invalidations
– Reliable data replication
7. Why Couchbase?
Evaluated systems to replace Memcached: Mongo,
Redis, and others
Couchbase had advantages
– Drop-in replacement for Memcached
– Built in replication and cluster expansion
– Memory latency for operations
– Persistence i.e. asynchronous writes to disk
– Utilize some of the development infrastructure we’ve built
8. Why Couchbase?
Partitioning
– Partitioning done automatically
– Expansion and rebalancing of cluster
Warm Caches
– Replication to protect against server failures
– On-disk data for server reboot
– Backup/restore and live data pumps for data transfer
across data centers
9. Development and Operations
Increasing number of servers, focused on remote
caches
Important for our caches to remain warm
Built up operational tools and standards
– Deployment configuration scripts
– Common pattern for development
– Support libraries for our developers
– Monitoring of the servers
10. Development and Operations: Tools
SALT modules to build and configure a new cluster
Set a master, setup/expand/reduce a cluster
Custom RPM with some backported bug fixes
Integrate with monitoring dashboards
– Grab over 300 metrics
– Per-host: Key to value ratio
– Aggregated metrics: QPS, data vs memory size
Monitor high watermark, latency, data ejection, etc.
11. Clusters and Numbers
Approximately 300 couchbase clusters in all colos
Current versions supported: 2.2.0 and 3.0.1
Average cluster size of 10 hosts
Largest cluster is 72 hosts.
Single and Multi-tenant clusters
Highest throughput cluster is 1.8M QPS.
12. Clusters and Numbers:
Credential Cache
Stores metadata
for users creds of
2 providers.
~8000 QPS
Avg. GET call
time ~1 ms.
18 hosts
13. Clusters and Numbers:
Credential Cache
~98% hit-ratio
All in Memory.
No ejections to disk
No performance
penalty (when
compared to RAM
access)
14. Couchbase Summary
In-Memory cache that fits into our existing
infrastructure
Provides eventual-consistency persistence
Read-scaling with acceptable latencies
Management and monitoring of the clusters
Rich set of tooling extended by members of CBVT
using Salt
15. Automation Use Cases
Build and deploy Couchbase in a reliable and consistent
way
Support multiple versions of Couchbase
Metadata to describe the cluster/buckets being
provisioned
Provide a layer of abstraction that is easy to use
Scale cluster sizes as needed.
Decommission clusters entirely.
17. Terminology
Salt Master
– central server that manages hosts which run agents, called Salt Minions.
Salt Runner
– application of convenience executed by the salt-run command on the Salt Master.
Execution Module
– similar to a Salt Runner except that is executed on the Minion host.
The method of configuration management provided by Salt consists of:
– Pillars: centrally managed data, rendered on the master
– Grains: data specific to the minion host that is being targeted
Range
– distributed metadata store that contains information about clusters of hosts.
18. Salt State
Expresses
the state of
a host in a
small easy-
to-read and
understand
file:
# If couchbase user and group do not
#exist, create it.
couchbase:
group.present:
- gid: 500
user.present:
- shell: /bin/bash
- home: /opt/couchbase
- createhome: True
- uid: 500
- gid: 500
19. Salt State
Specify path for
couchbase
bucket data that
is eventually
persisted to disk.
Uniform
configuration
across all hosts in
a cluster.
#Create path where couchbase will
#store bucket data:
create_data_dir:
file.directory:
- name: ‘/path/to/data’
- user: couchbase
- group: couchbase
- dir_mode: 700
20. Salt State
Install couchbase.
Version and
package are hard-
coded.
#Install couchbase-server version.
couchbase-server:
pkg.installed:
- name: ‘couchbase-server’
- require:
- user: couchbase
- group: couchbase
- file: password_file
- fromrepo: ifattah-
repo,RPMS.os
- refresh: True
- version: ‘3.0.1-1444.10’
21. Salt State
State successful if:
– All previous blocks
were successful.
– couchbase-server
process is running
Vanilla state
Robust templates
# Make sure that service is running
#after installation
service:
- running
- require:
- pkg: couchbase-server
22. Salt State Templating
Parameterize the version we want to install
Jinja Template + Pillar metadata
{% if salt['pillar.get']('couchbase_pillar:version') == '2.2.0' %}
{% set pkg_name = ’prod-couchbase-server' %}
{% set version = '2.2.0-85' %}
{% elif salt['pillar.get']('couchbase_pillar:version') == '3.0.1' %}
{% set pkg_name = ’prod-couchbase-server' %}
{% set version = '3.0.1-14' %}
{% endif %}
23. Salt State Templating
Install couchbase.
Version and package
are taken from pillar
data.
State (SLS file) can be
applied via CLI or via
Python Salt client
#Install couchbase-server version.
couchbase-server:
pkg.installed:
- name: {{ pkg_name }}
- require:
- user: couchbase
- group: couchbase
- fromrepo: some-repo
- refresh: True
- version: {{ version }}
24. Salt State
Can be applied via CLI or Salt Runner:
sudo salt hostname state.sls couchbase.setup
hostname1:
----------
ID: required_packages
Function: pkg.installed
Result: True
Comment: All specified packages are already installed.
Started: 20:37:00.729487
Duration: 6502.956 ms
Changes:
25. Salt Pillars
YAML-defined.
Every cluster’s composition.
– Buckets
– Replicas
– RAM Size
– Type
SASL or no SASL?
Host lists are obtained by Range.
Grain stores the name of this file so
that we can retrieve the admin
password for this cluster only.
#!yaml|gpg
couchbase_pillar:
admin_password: ‘<encrypted string>’
host_range: '%ifattah.couchbase.99'
data_path: /mnt/foo
version: 3.0.1
buckets:
bucketA:
type: couchbase
auth: none
port: 13337
replicas: 1
ram: 256
bucketB:
type: couchbase
auth: none
port: 13338
replicas: 1
ram: 256
26. Salt Grains
Host-specific data:
– OS version
– Kernel version
– Total Physical RAM
Set the pillar file which stores the encrypted admin password.
When pillar data is made available to a targeted minion:
– Include pillar file matching the name of the grain:
• couchbase.cluster.{{ grains['couchbase_cluster'] }}
Ensures a cluster is only accessing it’s specific pillar metadata
Set the cluster_ramsize (RAM to allocate to couchbase):
Ensures all hosts’ RAM utilized in the same way.
27. Salt Execution Module
Available functions executed by Salt Minion on targeted
hosts
Post-installation steps required by couchbase
Constructs cli commands and API requests
28. Salt Execution Module
Constructs Couchbase commands:
– Set cluster’s admin password, data_path (taken from pillar data)
– Add/remove host(s) from a cluster
– Issue a rebalance of data
– Get status/stop rebalance
– Create new buckets
– Flush existing buckets
Issue requests via HTTP REST API:
– Get Couchbase version from running node
– Enable auto-failover (when a node fails to respond).
– Rename a node
– Get membership status/health of all hosts in a cluster.
29. Salt Runner
Provides 4 functions to the user:
– setup_cluster:
• Build a cluster from scratch with provided metadata from pillar and host list
from range.
• Applies ‘setup’ state for installation.
– expand_cluster (reduce_cluster)
• Compares range host list with couchbase membership.
• Add hosts to couchbase cluster to match range (source of truth)
– Uninstall
• Ensures couchbase is removed by applying another state ‘uninstall.sls’
All above runner functions verify the results of all functions
executed by the minion (execution module)
30. Salt Runner
Ties everything together:
– sudo salt-run couchbase.setup_cluster %ifattah.couchbase.99
Apply ‘couchbase.setup’ state to given hosts
– Ensure all pre-installation and installation pre-requisites are met.
Sets grain value for a host to obtain the decrypted admin
password (name of pillar file)
Salt’s cmd_iter() invokes a function in the execution module
on the targeted minion host.
Runner output informs the user throughout the process
31. Salt Runner Output (setup_cluster)
Couchbase will be installed on:
salt-minion1.linkedin.local
salt-minion2.linkedin.local
salt-minion3.linkedin.local
salt-minion4.linkedin.local
Master node is: salt-minion1.linkedin.local
couchbase-server version: 3.0.1
Couchbase data path: /mnt/foo
Decrypted Couchbase Administrator password: adminadmin
Decrypted Couchbase readonly password: readonly
MB of RAM allocated for couchbase-server (cluster-init-ramsize): 1419
The following buckets will be created:
- bucketA
- bucketB
32. Salt Runner Output (setup_cluster)
[INFO] Non-interactive: Answered yes to ‘ Is the above information correct?
Proceed?’
[INFO] Beginning installation ...
[INFO] SUCCESS: Successfully ran couchbase.setup on salt-minion1…4.linkedin.local
[INFO] SUCCESS: couchbase.setup state applied to all hosts.
[INFO] Syncing modules from base env…
[INFO] Modules on salt-minion1.linkedin.local were synced.
[INFO] Modules on salt-minion2.linkedin.local were synced.
[INFO] Modules on salt-minion3.linkedin.local were synced.
[INFO] Modules on salt-minion4.linkedin.local were synced.
[INFO] Initializing couchbase on all cluster nodes…
[INFO] SUCCESS: Successfully set data_path on salt-minion4.linkedin.local
[INFO] SUCCESS: Successfully set data_path on salt-minion2.linkedin.local
[INFO] SUCCESS: Successfully set data_path on salt-minion3.linkedin.local
[INFO] SUCCESS: Successfully set data_path on salt-minion1.linkedin.local
[INFO] SUCCESS: Successfully set cluster_ramsize to 1419M
33. Salt Runner Output (setup_cluster)
[INFO] SUCCESS: Successfully renamed salt-minion1.linkedin.local with FQDN.
[INFO] SUCCESS: Successfully created readonly admin account
[INFO] Adding salt-minion2.linkedin.local to cluster...
[INFO] SUCCESS: Successfully added salt-minion2.linkedin.local to the cluster
[INFO] Adding salt-minion3.linkedin.local to cluster...
[INFO] SUCCESS: Successfully added salt-minion3.linkedin.local to the cluster
[INFO] Adding salt-minion4.linkedin.local to cluster...
[INFO] SUCCESS: Successfully added salt-minion4.linkedin.local to the cluster
[INFO] Starting Rebalance...
[INFO] SUCCESS: Successfully rebalanced the cluster
[INFO] SUCCESS: Successfully created bucketA bucket
[INFO] SUCCESS: Successfully created bucketB bucket
[INFO] Starting Rebalance...
[INFO] SUCCESS: Successfully rebalanced the cluster
[INFO] SUCCESS: Successfully enabled autofailover for the cluster
[INFO] INSTALLATION COMPLETE!
35. Salt Runner Output (reduce_cluster)
[INFO] ** Starting preliminary procedures for cluster reduction... **
[INFO] Setting grain...
[INFO] Summary of operations to be performed:
[INFO] The following hosts will be removed from the couchbase cluster:
[INFO] - salt-minion4.linkedin.local
[INFO] - salt-minion3.linkedin.local
[INFO] All commands will be executed on the chosen master node: salt-
minion1.linkedin.local
[INFO] Non-interactive: Answered yes to 'Is the above information correct?
Proceed?'
[INFO] WARNING: Cluster reduction can be a dangerous operation.
[INFO] Non-interactive: Answered yes to 'Are you sure this cluster can operate
with 2 fewer nodes? Proceed?’
[INFO] Marking hosts for removal and beginning rebalance...
[INFO] - salt-minion4.linkedin.local
[INFO] - salt-minion3.linkedin.local
[INFO] Rebalancing may take a while…
36. Salt Runner Output (reduce_cluster)
[INFO] Waiting 30 seconds after rebalance, before uninstall.
[INFO] Uninstalling couchbase from removed hosts...
[INFO] Couchbase will be uninstalled from:
[INFO] - salt-minion4.linkedin.local
[INFO] - salt-minion3.linkedin.local
[INFO] Non-interactive: Answered yes to 'Is the above information correct?
Proceed?'
[INFO] Syncing modules from base env...
[INFO] Modules on salt-minion3.linkedin.local were synced.
[INFO] Modules on salt-minion4.linkedin.local were synced.
[INFO] Beginning removal ...
[INFO] SUCCESS: removed couchbase_cluster grain from salt-minion4.linkedin.local
[INFO] SUCCESS: removed couchbase_cluster grain from salt-minion3.linkedin.local
[INFO] SUCCESS: Successfully ran couchbase.uninstall on salt-minion3…
[INFO] SUCCESS: Successfully ran couchbase.uninstall on salt-minion4…
[INFO] UNINSTALLED!
[INFO] Cluster reduction complete!
True
38. Salt Runner Output (expand_cluster)
[INFO] ** Starting preliminary procedures for cluster reduction... **
[INFO] Setting grain...
[INFO] Summary of operations to be performed:
[INFO] The following hosts will be removed from the couchbase cluster:
[INFO] - salt-minion4.linkedin.local
[INFO] - salt-minion3.linkedin.local
[INFO] All commands will be executed on the chosen master node: salt-
minion1.linkedin.local
[INFO] Non-interactive: Answered yes to 'Is the above information correct?
Proceed?'
[INFO] WARNING: Cluster reduction can be a dangerous operation.
[INFO] Non-interactive: Answered yes to 'Are you sure this cluster can operate
with 2 fewer nodes? Proceed?’
[INFO] Marking hosts for removal and beginning rebalance...
[INFO] - salt-minion4.linkedin.local
[INFO] - salt-minion3.linkedin.local
[INFO] Rebalancing may take a while…
39. Salt Runner Output (expand_cluster)
[INFO] ** Starting preliminary procedures for cluster expansion... **
[INFO] Setting grain...
[INFO] The following hosts belong to the range cluster %ifattah.couchbase.9999 :
[INFO] - salt-minion1.linkedin.local, clusterMember=active, status=healthy
[INFO] - salt-minion2.linkedin.local, clusterMember=active, status=healthy
[INFO] The following hosts are being added for cluster expansion:
- salt-minion3.linkedin.local
- salt-minion4.linkedin.local
[INFO] Non-interactive: Answered yes to 'Is the above information correct?'
[INFO] Beginning installation ...
[INFO] SUCCESS: Successfully ran couchbase.setup on salt-minion4.linkedin.local
[INFO] SUCCESS: Successfully ran couchbase.setup on salt-minion3.linkedin.local
[INFO] SUCCESS: couchbase.setup state applied to new hosts.
40. Salt Runner Output (expand_cluster)
[INFO] Syncing modules from base env...
[INFO] Modules on salt-minion4.linkedin.local were synced.
[INFO] Modules on salt-minion3.linkedin.local were synced.
[INFO] Initializing couchbase on new cluster nodes...
[INFO] SUCCESS: Successfully set data_path on salt-minion4.linkedin.local
[INFO] SUCCESS: Successfully set data_path on salt-minion3.linkedin.local
[INFO] Adding salt-minion3.linkedin.local to cluster...
[INFO] SUCCESS: Successfully added salt-minion3.linkedin.local to the cluster
[INFO] Adding salt-minion4.linkedin.local to cluster...
[INFO] SUCCESS: Successfully added salt-minion4.linkedin.local to the cluster
[INFO] Starting Rebalance...
[INFO] SUCCESS: Successfully rebalanced the cluster
[INFO] EXPANSION COMPLETE!
True
42. Caveats/Improvements
If grain isn’t set, pillar render error occurs on the
minion.
– To avoid leaking password to minion logs, set
safe_renderrer_error to false.
Frequent updates to execution module as functions are
tested for Couchbase 4.x
Logic to infer how many hosts can be safely removed
Provide a frontend interface
– Instead of running from salt master host.
43. Conclusions
Couchbase
– Provides a robust caching layer
– Powers critical parts of linkedin.com
Saltstack
– Quickly and dynamically provision clusters
– Reliably scale clusters as needed.
Automation is your friend
Open-source version can be made available if there is
enough interest.