This talk is from ApacheCon North America 2017 - Cassandra serving netflix @ scale - https://apachecon2017.sched.com/event/9zvG/cassandra-serving-netflix-scale-vinay-chella-netflix
https://www.youtube.com/watch?v=2l0_onmQsPI&index=3&t=284s&list=PL7uQt4PWyRW0XoVhEnNcSdCw5ufLEn9HA
This session will re-evaluate Cassandra’s relationship with runtime and build systems, pointing out ways that the existing systems fall down, and identifying avenues for improvement. Over the past few years, a number of platforms have emerged for running user code. Container runtimes like Docker, container orchestrators such as Kubernetes, and metrics collections agents like Prometheus and Spectator have all gained popularity and mind-share. Cassandra functionality such as metrics, bootstrapping, and monitoring integrates with the newer paradigms, but in an ad-hoc and improvised fashion. By taking a purposeful approach to integrating with these new methods of deployment, the Cassandra community can more fully benefit from their advertised strengths. The Cassandra build system based on Ant+Ivy dates to the early 2000’s, and reflects legacy complexity that could be avoided with modern build systems. Cassandra’s system package builds are not much better and often fail to integrate with industry standards such as systemd. Iterating on the existing systems is difficult, but this technical debt slows innovation in our build systems. In this talk, we propose solutions to make building, deploying and monitoring Cassandra easy and low overhead, while taking advantage of cloud advancements wherever possible.
How netflix manages petabyte scale apache cassandra in the cloudVinay Kumar Chella
At Netflix, we manage petabytes of data in Apache Cassandra which must be reliably accessible to users in mere milliseconds. To achieve this, we have built sophisticated control planes that turn our persistence layer based on Apache Cassandra into a truly self-driving system. We will start with the user interface that Netflix developers use to interact with their Cassandra databases and dive deep into the automation that powers it all. From cluster creation, through scaling up, to cluster death, complex automation drives large fleets of virtual machines hosted on the AWS cloud. First, we will cover the basics of how Netflix deploys Apache Cassandra. In particular, this begins with how we mold Apache Cassandra to the Netflix philosophy of immutable infrastructure, including managing software and hardware upgrades in the face of ever-failing hardware. Then we will explore the concrete techniques needed for such a massive deployment, specifically pull-based control planes and auto-healing strategies. Next, we will cover how Netflix has automated complex but critical Apache Cassandra maintenance tasks such as continuous snapshot backups and always-on anti-entropy repair for keeping our datasets safe and consistent. Both of these systems have gone through multiple architectural evolutions, and we have learned many lessons along the way. Lastly, we will share some of the ways this has gone wrong, and what you can do to avoid them. We will cover a few case studies of major Cassandra outages at Netflix, their root cause, and what we learned from those incidents. At the end of this talk, we hope that participants leave with concrete understanding of the challenges in running massive scale Apache Cassandra as well as solid advice and techniques for building their own self-driving data persistence layer.
Beginning Operations: 7 Deadly Sins for Apache Cassandra OpsDataStax Academy
The internal battle has been fought, and Cassandra is your group's NoSQL platform of choice! Hooray! But now what? This talk will introduce you to all the basic operations concepts you need to know to start your foray into the wonderful world of Cassandra off right. Or even if you have already started but are looking for a solid holistic overview... this is the talk for you!
Capturing live traffic in a typical micro-services setup is one of the industry standard strategies for various needs including testing, canary, and dark launches, but when it comes to databases, capturing and replaying live traffic is quite challenging but equally critical. Cassandra is one of the few databases which supports a live traffic capture and replay feature out of the box starting with 4.0. As part of this talk, Vinay Chella will dive deeper into the query logging framework that is introduced in Cassandra 4.0, and the features built on top of this framework, including full query logging, audit logging, and traffic replay. At end of this talk, you will learn how to use audit logging, live traffic capture and replay features in the upcoming release of Cassandra.
This talk is from Distributed Data Summit SF 2018 - http://distributeddatasummit.com/2018-sf/sessions#netflix2
Operating C* can involve a lot of required manpower, complex automation, or both. Some of this complexity comes from operational/configuration activity of the underlying kernel and hardware but much of it is operation complexity stemming from C* itself. Some examples of this complexity are restarting the database in a safe way, reliability backing up and restoring snapshots, monitoring the health of the datastore, and even ensuring eventual consistency through repair. As a result of these complexities, C* operators end up with complicated operational setups, which are expensive to build, manage and monitor. As part of this talk, we will share lessons learned in managing such complexity via our Priam sidecar including recent innovations in how our sidecar ensures the highest possible uptime and correctness of Cassandra. We then use this to motivate building in the management sidecar directly as part of C* itself (CASSANDRA-14395).
This sessions covers diagnosing and solving common problems encountered in production, using performance profiling tools. We’ll also give a crash course to basic JVM garbage collection tuning. Attendees will leave with a better understanding of what they should look for when they encounter problems with their in-production Cassandra cluster. This talk is intended for people with a general understanding of Cassandra, but it not required to have experience running it in production.
This talk is from ApacheCon North America 2017 - Cassandra serving netflix @ scale - https://apachecon2017.sched.com/event/9zvG/cassandra-serving-netflix-scale-vinay-chella-netflix
https://www.youtube.com/watch?v=2l0_onmQsPI&index=3&t=284s&list=PL7uQt4PWyRW0XoVhEnNcSdCw5ufLEn9HA
This session will re-evaluate Cassandra’s relationship with runtime and build systems, pointing out ways that the existing systems fall down, and identifying avenues for improvement. Over the past few years, a number of platforms have emerged for running user code. Container runtimes like Docker, container orchestrators such as Kubernetes, and metrics collections agents like Prometheus and Spectator have all gained popularity and mind-share. Cassandra functionality such as metrics, bootstrapping, and monitoring integrates with the newer paradigms, but in an ad-hoc and improvised fashion. By taking a purposeful approach to integrating with these new methods of deployment, the Cassandra community can more fully benefit from their advertised strengths. The Cassandra build system based on Ant+Ivy dates to the early 2000’s, and reflects legacy complexity that could be avoided with modern build systems. Cassandra’s system package builds are not much better and often fail to integrate with industry standards such as systemd. Iterating on the existing systems is difficult, but this technical debt slows innovation in our build systems. In this talk, we propose solutions to make building, deploying and monitoring Cassandra easy and low overhead, while taking advantage of cloud advancements wherever possible.
How netflix manages petabyte scale apache cassandra in the cloudVinay Kumar Chella
At Netflix, we manage petabytes of data in Apache Cassandra which must be reliably accessible to users in mere milliseconds. To achieve this, we have built sophisticated control planes that turn our persistence layer based on Apache Cassandra into a truly self-driving system. We will start with the user interface that Netflix developers use to interact with their Cassandra databases and dive deep into the automation that powers it all. From cluster creation, through scaling up, to cluster death, complex automation drives large fleets of virtual machines hosted on the AWS cloud. First, we will cover the basics of how Netflix deploys Apache Cassandra. In particular, this begins with how we mold Apache Cassandra to the Netflix philosophy of immutable infrastructure, including managing software and hardware upgrades in the face of ever-failing hardware. Then we will explore the concrete techniques needed for such a massive deployment, specifically pull-based control planes and auto-healing strategies. Next, we will cover how Netflix has automated complex but critical Apache Cassandra maintenance tasks such as continuous snapshot backups and always-on anti-entropy repair for keeping our datasets safe and consistent. Both of these systems have gone through multiple architectural evolutions, and we have learned many lessons along the way. Lastly, we will share some of the ways this has gone wrong, and what you can do to avoid them. We will cover a few case studies of major Cassandra outages at Netflix, their root cause, and what we learned from those incidents. At the end of this talk, we hope that participants leave with concrete understanding of the challenges in running massive scale Apache Cassandra as well as solid advice and techniques for building their own self-driving data persistence layer.
Beginning Operations: 7 Deadly Sins for Apache Cassandra OpsDataStax Academy
The internal battle has been fought, and Cassandra is your group's NoSQL platform of choice! Hooray! But now what? This talk will introduce you to all the basic operations concepts you need to know to start your foray into the wonderful world of Cassandra off right. Or even if you have already started but are looking for a solid holistic overview... this is the talk for you!
Capturing live traffic in a typical micro-services setup is one of the industry standard strategies for various needs including testing, canary, and dark launches, but when it comes to databases, capturing and replaying live traffic is quite challenging but equally critical. Cassandra is one of the few databases which supports a live traffic capture and replay feature out of the box starting with 4.0. As part of this talk, Vinay Chella will dive deeper into the query logging framework that is introduced in Cassandra 4.0, and the features built on top of this framework, including full query logging, audit logging, and traffic replay. At end of this talk, you will learn how to use audit logging, live traffic capture and replay features in the upcoming release of Cassandra.
This talk is from Distributed Data Summit SF 2018 - http://distributeddatasummit.com/2018-sf/sessions#netflix2
Operating C* can involve a lot of required manpower, complex automation, or both. Some of this complexity comes from operational/configuration activity of the underlying kernel and hardware but much of it is operation complexity stemming from C* itself. Some examples of this complexity are restarting the database in a safe way, reliability backing up and restoring snapshots, monitoring the health of the datastore, and even ensuring eventual consistency through repair. As a result of these complexities, C* operators end up with complicated operational setups, which are expensive to build, manage and monitor. As part of this talk, we will share lessons learned in managing such complexity via our Priam sidecar including recent innovations in how our sidecar ensures the highest possible uptime and correctness of Cassandra. We then use this to motivate building in the management sidecar directly as part of C* itself (CASSANDRA-14395).
This sessions covers diagnosing and solving common problems encountered in production, using performance profiling tools. We’ll also give a crash course to basic JVM garbage collection tuning. Attendees will leave with a better understanding of what they should look for when they encounter problems with their in-production Cassandra cluster. This talk is intended for people with a general understanding of Cassandra, but it not required to have experience running it in production.
Adam Zegelin is Instaclustr's founding software engineer. This presentation will investigate how using micro-batching for submitting writes to Cassandra can improve throughput and reduce client application CPU load. Micro-batching combines writes for the same partition key into a single network request and ensures they hit the “fast path” for writes on a Cassandra node.
AddThis: Scaling Cassandra up and down into containers with ZFSDataStax Academy
ZFS is an advanced file, raid, and volume management system originally developed by Sun Microsystems, 'The Last Word in File Systems' has been unavailable on Linux until recently. AddThis uses ZFS to more effectively scale up dedicated hardware, getting twice the performance at half the cost. ZFS is also fundamental to containerization, allowing nodes from multiple clusters to be co-located with safe persistent storage.
Cassandra Summit 2014: Active-Active Cassandra Behind the ScenesDataStax Academy
Presenter: Roopa Tangirala, Senior Cloud Data Architect at Netflix
High availability is an important requirement for any online business and trying to architect around failures and expecting infrastructure to fail, and even then be highly available, is the key to success. One such effort here at Netflix was the Active-Active implementation where we provided region resiliency. This presentation will discuss the brief overview of the active-active implementation and how it leveraged Cassandra’s architecture in the backend to achieve its goal. It will cover our journey through A-A from Cassandra’s perspective, the data validation we did to prove the backend would work without impacting customer experience. The various problems we faced, like long repair times and gc_grace settings, plus lessons learned and what would we do differently next time around, will also be discussed.
These slides are from the recent meetup @ Uber - Apache Cassandra at Uber and Netflix on new features in 4.0.
Abstract:
A glimpse of Cassandra 4.0 features:
There are a lot of exciting features coming in 4.0, but this talk covers some of the features that we at Netflix are particularly excited about and looking forward to. In this talk, we present an overview of just some of the many improvements shipping soon in 4.0.
This presentation will investigate how using micro-batching for submitting writes to Cassandra can improve throughput and reduce client application CPU load.
Micro-batching combines writes for the same partition key into a single network request and ensures they hit the "fast path" for writes on a Cassandra node.
About the Speaker
Adam Zegelin Technical Co-founder, Instaclustr
As Instaclustrs founding software engineer, Adam provides the foundation knowledge of our capability and engineering environment. He delivers business-focused value to our code-base and overall capability architecture. Adam is also focused on providing Instaclustr's contribution to the broader open source community on which our products and services rely, including Apache Cassandra, Apache Spark and other technologies such as CoreOS and Docker.
Summary of past Cassandra benchmarks performed by Netflix and description of how Netflix uses Cassandra interspersed with a live demo automated using Jenkins and Jmeter that created two 12 node Cassandra clusters from scratch on AWS, one with regular disks and one with SSDs. Both clusters were scaled up to 24 nodes each during the demo.
Mesosphere and Contentteam: A New Way to Run CassandraDataStax Academy
We, Ben Whitehead and Robert Stupp, will show you how to run Cassandra on Mesos. We will go through all the technical steps how to plan, setup and operate even large scale Cassandra clusters on Mesos. Further we illustrate how the Cassandra-on-Mesos framework helps you to setup Cassandra on Mesos, schedule regular maintenance tasks and manage hardware failures in the heart of your data center.
Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...DataStax
We use Apache Cassandra at BlackRock to help power our Aladdin investment management platform. Like most users, we love Cassandra’s scalability and fault tolerance. One challenge we’ve faced is keeping data consistent between data centers. Cassandra is great at replicating data to multiple data centers, and many users take advantage of this feature to achieve eventual consistency in multi-region clusters. At BlackRock, we have several use cases where eventual consistency is not good enough; sometimes we need to guarantee that the most recent data is available from all locations. Cassandra’s tunable consistency makes it possible to achieve this extreme level of resiliency. In this talk we’ll discuss our experience from the past several years using Cassandra for cross-WAN consistency, some of the novel ways we’ve dealt with the performance implications, and our ideas for improving support for this usage model in future versions of Cassandra.
About the Speaker
Randy Fradin Vice President, BlackRock
Randy Fradin is part of BlackRock’s Aladdin Product Group. His team is responsible for developing the core software infrastructure in BlackRock’s Aladdin platform, including scalable storage, compute, and messaging services. Previously he spent time developing the market data, risk reporting, and core trading functions in Aladdin. He has been an enthusiastic Cassandra user since 2011.
Performance tuning - A key to successful cassandra migrationRamkumar Nottath
In last few years, technology has seen a major drift in the dominance of traditional / RDMBS databases across different domains. Expeditious adoption of NoSQL databases especially Cassandra in the industry opens up a lot more discussions on what are the major challenges that are faced during implementation of Cassandra and how to mitigate it. Many a times we conclude that migration or POC (proof of concept) is not successful; however the real flaw might be in the data modeling, identifying the right hardware configurations, database parameters, right consistency level and so on. There's no one good model or configuration which fits all use cases and all applications. Performance tuning an application is truly an art and requires perseverance. This paper delve into different performance tuning considerations and anti-patterns that need to be considered during Cassandra migration / implementation to make sure we are able to reap the benefits of Cassandra, what makes it a ‘Visionary’ in 2014 Gartner’s Magic Quadrant for Operational Database Management Systems.
Arc305 how netflix leverages multiple regions to increase availability an i...Ruslan Meshenberg
Learn how to make your services more resilient and available by embracing principles of isolation and redundancy. See details of 2 projects - Isthmus and Active/Active to learn how Netflix architects for availability in multi-regional environment.
An experience sharing of the OpenStack deployment at Suning.com, a large online retailer in China. The talk presents the challenges and opportunities on orchestrating the enterprise workloads using Heat.
Cassandra is the dominant data store used at Netflix and it's health is critical to many of its services. In this talk we will share details of the recent redesign of our health monitoring system and how we leveraged a reactive stream processing system to give us a real-time view our entire fleet while dramatically improving accuracy and reducing false alarms in our alerting.
About the Speaker
Jason Cacciatore Senior Software Engineer, Netflix
Jason Cacciatore is a Senior Software Engineer at Netflix, where he's been working for the past several years. He's interested in stateful distributed systems and has a diverse background in technology. In his spare time he enjoys spending time with his wife and two sons, reading non-fiction, and watching Netflix documentaries.
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...DataStax
With the addition of vnodes (Virtual Nodes), Cassandra users were able to gain a few benefits as a result of streaming when it came to bootstrapping and decommissioning nodes. On the flip side, having to route requests on larger clusters became a lot more intensive of a workload for all nodes that were then forced to act coordinator nodes. By setting up a tier of proxy nodes, we were able to have our cluster of 50 nodes perform with a 300% improvement on average in a mixed workload environment. This is an explanation of what we did, how we did it, and why it works.
About the Speaker
Eric Lubow CTO, SimpleReach
Eric Lubow is CTO of SimpleReach, where he builds highly-scalable distributed systems for processing analytics data. Eric is also a DataStax MVP for Cassandra, and co-author of Practical Cassandra. In his spare time, Eric is a skydiver, motorcycle rider, mixed martial artist, and dog dad.
Програми Microsoft для розробників та стартапів Kyrylo Bezpalyi
Можливості та програми Microsoft для початківців, процессіоналів та стартапів по розробці програмного забезпечення та програмних сервісів будь-якого роду та масштабу.
Adam Zegelin is Instaclustr's founding software engineer. This presentation will investigate how using micro-batching for submitting writes to Cassandra can improve throughput and reduce client application CPU load. Micro-batching combines writes for the same partition key into a single network request and ensures they hit the “fast path” for writes on a Cassandra node.
AddThis: Scaling Cassandra up and down into containers with ZFSDataStax Academy
ZFS is an advanced file, raid, and volume management system originally developed by Sun Microsystems, 'The Last Word in File Systems' has been unavailable on Linux until recently. AddThis uses ZFS to more effectively scale up dedicated hardware, getting twice the performance at half the cost. ZFS is also fundamental to containerization, allowing nodes from multiple clusters to be co-located with safe persistent storage.
Cassandra Summit 2014: Active-Active Cassandra Behind the ScenesDataStax Academy
Presenter: Roopa Tangirala, Senior Cloud Data Architect at Netflix
High availability is an important requirement for any online business and trying to architect around failures and expecting infrastructure to fail, and even then be highly available, is the key to success. One such effort here at Netflix was the Active-Active implementation where we provided region resiliency. This presentation will discuss the brief overview of the active-active implementation and how it leveraged Cassandra’s architecture in the backend to achieve its goal. It will cover our journey through A-A from Cassandra’s perspective, the data validation we did to prove the backend would work without impacting customer experience. The various problems we faced, like long repair times and gc_grace settings, plus lessons learned and what would we do differently next time around, will also be discussed.
These slides are from the recent meetup @ Uber - Apache Cassandra at Uber and Netflix on new features in 4.0.
Abstract:
A glimpse of Cassandra 4.0 features:
There are a lot of exciting features coming in 4.0, but this talk covers some of the features that we at Netflix are particularly excited about and looking forward to. In this talk, we present an overview of just some of the many improvements shipping soon in 4.0.
This presentation will investigate how using micro-batching for submitting writes to Cassandra can improve throughput and reduce client application CPU load.
Micro-batching combines writes for the same partition key into a single network request and ensures they hit the "fast path" for writes on a Cassandra node.
About the Speaker
Adam Zegelin Technical Co-founder, Instaclustr
As Instaclustrs founding software engineer, Adam provides the foundation knowledge of our capability and engineering environment. He delivers business-focused value to our code-base and overall capability architecture. Adam is also focused on providing Instaclustr's contribution to the broader open source community on which our products and services rely, including Apache Cassandra, Apache Spark and other technologies such as CoreOS and Docker.
Summary of past Cassandra benchmarks performed by Netflix and description of how Netflix uses Cassandra interspersed with a live demo automated using Jenkins and Jmeter that created two 12 node Cassandra clusters from scratch on AWS, one with regular disks and one with SSDs. Both clusters were scaled up to 24 nodes each during the demo.
Mesosphere and Contentteam: A New Way to Run CassandraDataStax Academy
We, Ben Whitehead and Robert Stupp, will show you how to run Cassandra on Mesos. We will go through all the technical steps how to plan, setup and operate even large scale Cassandra clusters on Mesos. Further we illustrate how the Cassandra-on-Mesos framework helps you to setup Cassandra on Mesos, schedule regular maintenance tasks and manage hardware failures in the heart of your data center.
Maintaining Consistency Across Data Centers (Randy Fradin, BlackRock) | Cassa...DataStax
We use Apache Cassandra at BlackRock to help power our Aladdin investment management platform. Like most users, we love Cassandra’s scalability and fault tolerance. One challenge we’ve faced is keeping data consistent between data centers. Cassandra is great at replicating data to multiple data centers, and many users take advantage of this feature to achieve eventual consistency in multi-region clusters. At BlackRock, we have several use cases where eventual consistency is not good enough; sometimes we need to guarantee that the most recent data is available from all locations. Cassandra’s tunable consistency makes it possible to achieve this extreme level of resiliency. In this talk we’ll discuss our experience from the past several years using Cassandra for cross-WAN consistency, some of the novel ways we’ve dealt with the performance implications, and our ideas for improving support for this usage model in future versions of Cassandra.
About the Speaker
Randy Fradin Vice President, BlackRock
Randy Fradin is part of BlackRock’s Aladdin Product Group. His team is responsible for developing the core software infrastructure in BlackRock’s Aladdin platform, including scalable storage, compute, and messaging services. Previously he spent time developing the market data, risk reporting, and core trading functions in Aladdin. He has been an enthusiastic Cassandra user since 2011.
Performance tuning - A key to successful cassandra migrationRamkumar Nottath
In last few years, technology has seen a major drift in the dominance of traditional / RDMBS databases across different domains. Expeditious adoption of NoSQL databases especially Cassandra in the industry opens up a lot more discussions on what are the major challenges that are faced during implementation of Cassandra and how to mitigate it. Many a times we conclude that migration or POC (proof of concept) is not successful; however the real flaw might be in the data modeling, identifying the right hardware configurations, database parameters, right consistency level and so on. There's no one good model or configuration which fits all use cases and all applications. Performance tuning an application is truly an art and requires perseverance. This paper delve into different performance tuning considerations and anti-patterns that need to be considered during Cassandra migration / implementation to make sure we are able to reap the benefits of Cassandra, what makes it a ‘Visionary’ in 2014 Gartner’s Magic Quadrant for Operational Database Management Systems.
Arc305 how netflix leverages multiple regions to increase availability an i...Ruslan Meshenberg
Learn how to make your services more resilient and available by embracing principles of isolation and redundancy. See details of 2 projects - Isthmus and Active/Active to learn how Netflix architects for availability in multi-regional environment.
An experience sharing of the OpenStack deployment at Suning.com, a large online retailer in China. The talk presents the challenges and opportunities on orchestrating the enterprise workloads using Heat.
Cassandra is the dominant data store used at Netflix and it's health is critical to many of its services. In this talk we will share details of the recent redesign of our health monitoring system and how we leveraged a reactive stream processing system to give us a real-time view our entire fleet while dramatically improving accuracy and reducing false alarms in our alerting.
About the Speaker
Jason Cacciatore Senior Software Engineer, Netflix
Jason Cacciatore is a Senior Software Engineer at Netflix, where he's been working for the past several years. He's interested in stateful distributed systems and has a diverse background in technology. In his spare time he enjoys spending time with his wife and two sons, reading non-fiction, and watching Netflix documentaries.
Optimizing Your Cluster with Coordinator Nodes (Eric Lubow, SimpleReach) | Ca...DataStax
With the addition of vnodes (Virtual Nodes), Cassandra users were able to gain a few benefits as a result of streaming when it came to bootstrapping and decommissioning nodes. On the flip side, having to route requests on larger clusters became a lot more intensive of a workload for all nodes that were then forced to act coordinator nodes. By setting up a tier of proxy nodes, we were able to have our cluster of 50 nodes perform with a 300% improvement on average in a mixed workload environment. This is an explanation of what we did, how we did it, and why it works.
About the Speaker
Eric Lubow CTO, SimpleReach
Eric Lubow is CTO of SimpleReach, where he builds highly-scalable distributed systems for processing analytics data. Eric is also a DataStax MVP for Cassandra, and co-author of Practical Cassandra. In his spare time, Eric is a skydiver, motorcycle rider, mixed martial artist, and dog dad.
Програми Microsoft для розробників та стартапів Kyrylo Bezpalyi
Можливості та програми Microsoft для початківців, процессіоналів та стартапів по розробці програмного забезпечення та програмних сервісів будь-якого роду та масштабу.
Чим займаються програмісти або як почати писати кодKyrylo Bezpalyi
Основные обязанности программистов, чем они занимаются, что означает «Быть программистом». И что же надо знать, чтобы устроится на работу в ИТ-компанию на позицию младший разработчик.
Зачем SQL тестировщику и что ему нужно про него знатьKyrylo Bezpalyi
Доклад для начинающий тестровщиков программного обеспечения в котором пойдет речь о языке структурированых запросов SQL и примерах базовых видов запросов
Windows Server 2012 Active Directory Domain and Trust (Forest Trust)Serhad MAKBULOĞLU, MBA
Serhad Makbuloğlu tarafından verilecek bu web seminerinde, Windows Server 2012 Active Directory Domain and Trust (Forest Trust) konusu anlatılacaktır. Bu web semineri teknik içeriğe sahiptir. Ayrıca Web seminer sonundaki soru&cevap bölümünde de bu ürünle ilgili merak ettiğiniz soruların cevaplarını bulabilirsiniz.
Neutron Done the SDN Way
Dragonflow is an open source distributed control plane implementation of Neutron which is an integral part of OpenStack. Dragonflow introduces innovative solutions and features to implement networking and distributed network services in a manner that is both lightweight and simple to extend, yet targeted towards performance-intensive and latency-sensitive applications. Dragonflow aims at solving the performance
Challenge: Recent success of Docker containers reveals arrival of a new era: the number of CPUs is exploding 10-100 folds up, and cloud networking is already in a new movement of scalability upgrade
Question: To scale UP or OUT? I.e., UPgrade or OUTgrade?
Answer from DaoliCloud’s practice: Better scale OUT, ,,.and Openflow can help
PPTV is using CloudStack 3.0.2 in its production environment. Currently there are more than 150 hosts, and migrate their apps to cloud everyday (10 host per day). At the end of 2013, there will be more than 1000 hosts in a CloudStack environment.
[NetPonto] Arquitectura dos Serviços da plataforma Windows AzureVitor Tomaz
O Windows Azure é uma plataforma que fornece serviços de alta disponibilidade e escalabilidade. Nesta sessão iremos abordar a arquitectura dos serviços base desta plataforma (Compute, Storage e SQLAzure) de modo a entendermos de que forma é que a escalabilidade e alta disponibilidade são conseguidas. Iremos ver as diferenças para as plataformas "tradicionais" e algumas consequências no desenvolvimento de soluções para este ambiente.
Apresentação do Vitor Tomaz sobre a Arquitectura dos Serviços da plataforma Windows Azure na 4a Reunião Presencial da Comunidade NetPonto em Coimbra (http://netponto.org).
Galera cluster for MySQL - Introduction SlidesSeveralnines
This set of slides gives you an overview of Galera, configuration basics and deployment best practices.
The following topics are covered:
- Concepts
- Node provisioning
- Network partitioning
- Configuration example
- Benchmarks
- Deployment best practices
- Galera monitoring and management
Always On - Wydajność i bezpieczeństwo naszych danych - High Availability SQL...SQLExpert.pl
SQL Server 2012 udostępnia zupełnie nowe spojrzenie na zagadnienia związanie z Wysoką dostępnością (High Avability). Najbardziej oczekiwaną nową funkcjonalnością jest AlwaysOn.
W trakcie sesji chcemy pokazać praktyczne zastosowanie AlwaysOn, odpowiadając na pytania:
• Co daje AlwaysOn czego nie było do tej pory
• W jakim stopniu AlwaysOn może zastąpić mirroring oraz logshiping
• Jak zbudować High Avability na potrzeby naszej organizacji
• Jak efektywnie skorzystać z wielu replik danych.
• Czym jest AlwaysOn Failover Cluster
W ramach sesji przedstawiona będzie koncepcja budowy rozwiązania spełniające oczekiwania w zakresie High Avability, dotyczące wydajności jak i bezpieczeństwa naszych danych, a także zgodne z potrzebami Distaster Recovery.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Enhancing Performance with Globus and the Science DMZGlobus
ESnet has led the way in helping national facilities—and many other institutions in the research community—configure Science DMZs and troubleshoot network issues to maximize data transfer performance. In this talk we will present a summary of approaches and tips for getting the most out of your network infrastructure using Globus Connect Server.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Welcome to the first live UiPath Community Day Dubai! Join us for this unique occasion to meet our local and global UiPath Community and leaders. You will get a full view of the MEA region's automation landscape and the AI Powered automation technology capabilities of UiPath. Also, hosted by our local partners Marc Ellis, you will enjoy a half-day packed with industry insights and automation peers networking.
📕 Curious on our agenda? Wait no more!
10:00 Welcome note - UiPath Community in Dubai
Lovely Sinha, UiPath Community Chapter Leader, UiPath MVPx3, Hyper-automation Consultant, First Abu Dhabi Bank
10:20 A UiPath cross-region MEA overview
Ashraf El Zarka, VP and Managing Director MEA, UiPath
10:35: Customer Success Journey
Deepthi Deepak, Head of Intelligent Automation CoE, First Abu Dhabi Bank
11:15 The UiPath approach to GenAI with our three principles: improve accuracy, supercharge productivity, and automate more
Boris Krumrey, Global VP, Automation Innovation, UiPath
12:15 To discover how Marc Ellis leverages tech-driven solutions in recruitment and managed services.
Brendan Lingam, Director of Sales and Business Development, Marc Ellis
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
3. SQL Azure model
Each account has zero or more servers
Account Azure wide, provisioned in a common portal
Billing instrument
Each server has one or more databases
Zone for authentication: userId+password
Zone for administration and billing
Server Metadata about the databases and usage
Network access control based on client IP
Has unique DNS name and unit of geo-location
Each database has standard SQL objects
Unit of consistency and high availability
(autonomous replication)
Database Contains Users, Tables, Views, Indices, etc…
Most granular unit of usage reports
Three SKUs available (1GB, 10GB and 50GB)
3 12/25/2011
4. SQL Azure Network Topology use standard SQL
Applications
Application client libraries: ODBC,
ADO.Net, PHP, …
Internet
Azure
Cloud
TDS (tcp)
Security Boundary Load balancer forwards „sticky‟
LB sessions to TDS protocol tier
TDS (tcp)
Gateway Gateway Gateway Gateway Gateway Gateway
Gateway: TDS protocol gateway, enforces AUTHN/AUTHZ policy; proxy to CloudDB
TDS (tcp)
L SQL SQL SQL SQL
4 12/25/2011
Scalability and Availability: Fabric, Failover, Replication, and Load balancing
5. Replication
All reads are
completed at
the primary Ack
Read
Write
Writes Value
Ack
replicated to Ack
write quorum of
replicas Ack P Ack
Commit on
secondaries S Write Write S
first then
primary
S Write Write S
Each
transaction has
a commit
5
sequence 12/25/2011
number (epoch,
6. Reconfiguration
Types of reconfiguration
Primary failover
Removing a failed
secondary Failed
Adding recovered replica
Building a new secondary
S
B P
Assumes S
Failed
Failure detector
Leader election Safe in the presence
Both services provided by of cascading failures
Fabric layer
6 12/25/2011
7. Partition Management
Partition Manager (PM) is a highly available service
running in the Master cluster
Ensures all partitions are operational
Places replicas across failure domains
(rack/switch/server)
Ensures all partitions have target replica count
Balances the load across all the nodes
Each node manages multiple partitions
Global state maintained by the PM can be recreated
from the local node state in event of disaster (GPM
rebuild)
7 12/25/2011
8. System in Operation
Primary master node
SQL Server Partition Manager
Partition
Global Load Balancer
Managemen
Partition t
Partition Placement
map
Advisor
Leader
Fabric
Elector
Data Node Data Node Data Node Data Node Data Node Data Node
100 101 102 103 104 105
P S P P S P
S S S S S S
S P S S S S
S S S S
S
Fabric
Secondary Primary Secondary
8 12/25/2011
9. SQL node Architecture
Single physical DB for entire Machine SQL
node Instance
DB files and log shared master
across every logical
database/partition tempdb
Allows better logging msdb
throughput with sequential
IO/group commits CloudNode
No auto-growth on demand
stalls DB1 DB2 master
Uniform manageability and DB5 DB1 DB7
backup
Each partition is a “silo” with DB3 DB4 DB7
its own independent schema
Local SQL backup guards
against software bugs
9 12/25/2011
10. Recap
Two kinds of nodes:
Data nodes store application data
Master nodes store cluster metadata
Node failures are reliably detected
On every node, SQL and Fabric processes monitor each
other
Fabric processes monitor each other across nodes
Local failures cause nodes to fail-fast
Failures cause reconfiguration and placement
changes
10 12/25/2011
11. Hardware Architecture
Each rack hosts 2 pods of
L2 Switch 20 machines each
Each pod has a TOR mini-
switch
10GB uplink to L2 switch
Each SQL Azure machine
runs on commodity box
Example:
8 cores
32 GB RAM
1TB+ SATA drives
Programmable power
1Gb NIC
Machine spec changes as
hardware (pricing) evolves
11 12/25/2011
12. Hardware Challenges
SATA drives
On-disk cache and lack of true "write through" results
in Write Ahead Logging violations
DB requires in-order writes to be honored
Can force flush cache, but causes performance
degradation
Disk failures happen daily (at scale), fail-fast on
those
Bit-flips (enabled page checksums)
Drives just disappear
IOs are misdirected
Faulty NIC
Encountered message corruption
12 Enabled message signing and checksums 12/25/2011
13. Software Deployment
OS is automatically imaged via deployment
All the services are setup using file copy
Guarantees on which version is running
Provides fast switch to new version
Minimal global state allows running side by side
Yes, that includes the SQL Server DB engine
Rollout is monitored to ensure high availability
Knowledge of replica state health ensure SLA is met
Two phase rollouts for data or protocol changes
Leverages internal Autopilot technologies with SQL
Azure extensions
13 12/25/2011
14. Software Challenges
Lack of real-time OS features
CPU priority
High priority for Fabric lease traffic
Page Faults/GC
Locked pages for SQL and Fabric (in managed
code)
Fail fast or not?
Yes, for corruption/AV
No, for other issues unless centrally controlled
What is really considered failed?
Some failures are non-deterministic or hangs
Multiple protocols / channels means partial failures too
14 12/25/2011
15. Monitoring
Health model w/repair actions
Reboot -> Re-deploy -> Re-image (OS) -> RMA cycle
Additional monitoring for SQL tier
Connect / network probes
Memory leaks / hung worker processes
Database corruption detection
Trace and performance stats capture
Sourced from regular SQL trace and support mechanisms
Stored locally and pushed to a global cluster wide store
Global cluster used for service insight and problem tracking
15 12/25/2011
16. SQL Azure Login Process
7
1
TDS Gateway
Front-end Node
TDS Session Protocol Parser
6 2
Gateway Logic
Global Partition Map
Master Node
8 3
Master Node
Components
4 5
Backend Node 1 Backend Node 2 Backend Node 3
SQL Instance SQL Instance SQL Instance
SQL DB SQL DB SQL DB
16 Scalability and and Availability: Fabric,Failover,Replication, and Load balancing
Scalability Availability: Fabric, Failover, Replication, and Load balancing
12/25/2011
17. Security/Attack Considerations
Service
Secure channel required (SSL)
Denial Of Service trend tracking
Packet Inspection
Server
IP allow list (Firewall)
Idle connection culling
Generated server names
Database
Disallow the most commonly attacked user id‟s (SA,
Admin, root, guest, etc)
Standard SQL Authn/Authz mode
17 12/25/2011
22. Summary
Cloud is different
Not a different place to host code
SQL Azure IS SQL Server…a TDS endpoint
Create DB‟s and manage using what we already
know
Considerations and futures paint exciting picture of
what to expect looking forward
Opportunities are great
Customers want a utility approach to storage
New businesses and abilities in scale, availability, etc
But the price must be paid
Which is a good thing, otherwise everyone would be
22 12/25/2011
doing it!
