SUE 2018 - Migrating a 130TB Cluster from Elasticsearch 2 to 5 in 20 Hours Wi...Fred de Villamil
The talk I gave at the Snow Unix Event in Nederland about upgrading a massive production Elasticsearch cluster from a major version to another without downtime and a complete rollback plan.
Cassandra Day 2014: Interactive Analytics with Cassandra and SparkEvan Chan
Take your analytics to the next level by using Apache Spark to accelerate complex interactive analytics using your Apache Cassandra data. Includes an introduction to Spark as well as how to read Cassandra tables in Spark.
SUE 2018 - Migrating a 130TB Cluster from Elasticsearch 2 to 5 in 20 Hours Wi...Fred de Villamil
The talk I gave at the Snow Unix Event in Nederland about upgrading a massive production Elasticsearch cluster from a major version to another without downtime and a complete rollback plan.
Cassandra Day 2014: Interactive Analytics with Cassandra and SparkEvan Chan
Take your analytics to the next level by using Apache Spark to accelerate complex interactive analytics using your Apache Cassandra data. Includes an introduction to Spark as well as how to read Cassandra tables in Spark.
Johnny Miller – Cassandra + Spark = Awesome- NoSQL matters Barcelona 2014NoSQLmatters
Johnny Miller – Cassandra + Spark = Awesome
This talk will discuss how Cassandra and Spark can work together to deliver real-time analytics. This is a technical discussion that will introduce the attendees to the basic principals on Cassandra and Spark, why they work well together and examples usecases.
APACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van NiekerkSpark Summit
Many data scientists are already making heavy usage of the Jupyter ecosystem for analyzing data using interactive notebooks.
Apache Toree (incubating) is a Jupyter kernel designed to act as a gateway to Spark by enabling users Spark from standard Jupyter notebooks. This allows users to easily integrate Spark into their existing Jupyter deployments, This allows users to easily move between languages and contexts without needing to switch to a different set of tools.
Apache Toree is designed expressly for interactive work. It supports interpreters in Scala, Python, and R.
In this talk, I will cover the design of Toree, how it interacts with the Jupyter ecosystem and various ways in which users can extend the functionality of Apache Toree via a powerful plugin system.
Spark and Object Stores —What You Need to Know: Spark Summit East talk by Ste...Spark Summit
If you are running Apache Spark in cloud environments, Object Stores —such as Amazon S3 or Azure WASB— are a core part of your system. What you can’t do is treat them like “just another filesystem” —do that and things will, eventually, go horribly wrong.
This talk looks at the object stores in the cloud infrastructures, including underlying architectures., compares them to what a “real filesystem” is expected to do and shows how to use object stores efficiently and safely as sources of and destinations of data.
It goes into depth on recent “S3a” work, showing how including improvements in performance, security, functionality and measurement —and demonstrating how to use make best use of it from a spark application.
If you are planning to deploy Spark in cloud, or doing so today: this is information you need to understand. The performance of you code and integrity of your data depends on it.
Scalability is one key feature that drives OpenStack adoption around the globe. With the dispatch of OpenStack's 14th release dubbed Newton, OpenStack makers affirm that it now incorporates enhanced scaling abilities. The Operations team at the OpenStack Innovation Center decided to demonstrate that OpenStack is effortlessly scalable to 500 nodes in a single region. To complete this task, we decided to work on a tool that allows them to target a set number of servers and "partition" them; if you have 10 physical servers, split those into 100 virtual servers, and use them as compute nodes within an OpenStack deployment. In this session we will examine the simulator tool, deep dive into its internals, and run through a demo. You will learn how you might utilize it to test the scaling of various OpenStack services and decrease the cost of creating labs and staging environments within your company.
Breaking Spark: Top 5 mistakes to avoid when using Apache Spark in productionNeelesh Srinivas Salian
Spark has been growing in deployments for the past year. The increasing amount of data being analyzed and processed through the framework is massive and continues to push the boundaries of the engine. Drawing on experiences across 150+ production deployments, Neelesh Srinivas Salian explores common issues observed in a cluster environment setup with Apache Spark and offers guidelines to help setup a real-world environment when planning an Apache Spark deployment in a cluster. Attendees can use these observations to improve the usability and supportability of Apache Spark and avoid such issues in their projects.
Topics include:
Scaling the architecture
Memory configurations
End-user code
Incompatible dependencies
Administration- and operation-related issues
Kerberizing Spark: Spark Summit East talk by Abel Rincon and Jorge Lopez-MallaSpark Summit
Spark had been elected, deservedly, as the main massive parallel processing framework, and HDFS is the one of the most popular Big Data storage technologies. Therefore its combination is one of the most usual Big Data’s use cases. But, what happens with the security? Can these two technologies coexist in a secure environment? Furthermore, with the proliferation of BI technologies adapted to Big Data environments, that demands that several users interacts with the same cluster concurrently, can we continue to ensure that our Big Data environments are still secure? In this lecture, Abel and Jorge will explain which adaptations of Spark´s core they had to perform in order to guarantee the security of multiple concurrent users using a single Spark cluster, which can use any of its cluster managers, without degrading the outstanding Spark’s performance.
Off-the-shelf, Perforce is a remarkably fast and stable system. Perforce admins can change that, however, by adding triggers that are slow, crash, or have external dependencies that cause them to fail. Find out how the Pixar team has addressed this challenge with a checkin-driven work queue system that allows for independent, parallel and asynchronous execution of checkin logic. They'll also briefly describe how they notify 10,000 computers when a checkin is completed.
AWS re:Invent 2019 - DAT328 Deep Dive on Amazon Aurora PostgreSQLGrant McAlister
Amazon Aurora with PostgreSQL compatibility is a relational database service that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases. In this session, we review the functionality in order to understand the architectural differences that contribute to improved scalability, availability, and durability. You'll also get a deep dive into the capabilities of the service and a review of the latest available features. Finally, we walk you through the techniques that you can use to migrate to Amazon Aurora.
Accelerating Shuffle: A Tailor-Made RDMA Solution for Apache Spark with Yuval...Spark Summit
The opportunity in accelerating Spark by improving its network data transfer facilities has been under much debate in the last few years. RDMA (remote direct memory access) is a network acceleration technology that is very prominent in the HPC (high-performance computing) world, but has not yet made its way to mainstream Apache Spark. Proper implementation of RDMA in network-oriented applications can improve scalability, throughput, latency and CPU utilization. In this talk we are going to present a new RDMA solution for Apache Spark that shows amazing improvements in multiple Spark use cases. The solution is under development in our labs, and is going to be released to the public as an open-source plug-in.
Teradata Partners Conference Oct 2014 Big Data Anti-PatternsDouglas Moore
Big Data Anti-Patterns: Lessons from the Front Lines
Drawn from over 50 client engagements, big data anti-patterns are common practices that make for bad solutions.
Running Emerging AI Applications on Big Data Platforms with Ray On Apache SparkDatabricks
With the rapid evolution of AI in recent years, we need to embrace advanced and emerging AI technologies to gain insights and make decisions based on massive amounts of data. Ray (https://github.com/ray-project/ray) is a fast and simple framework open-sourced by UC Berkeley RISELab particularly designed for easily building advanced AI applications in a distributed fashion.
Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...StampedeCon
At the StampedeCon 2015 Big Data Conference: Picking your distribution and platform is just the first decision of many you need to make in order to create a successful data ecosystem. In addition to things like replication factor and node configuration, the choice of file format can have a profound impact on cluster performance. Each of the data formats have different strengths and weaknesses, depending on how you want to store and retrieve your data. For instance, we have observed performance differences on the order of 25x between Parquet and Plain Text files for certain workloads. However, it isn’t the case that one is always better than the others.
An Update on the A400M Program: June 2014ICSA, LLC
In this presentation to the press at the Airbus Trade Media Briefing, Rafael Tentor-Aunon, head of the A400M program, provided an update on June 9,2014. The first two A400Ms are in service in France and Second Line of Defense has visited the squadron in France and will report soon on the introduction of the aircraft into service.
Johnny Miller – Cassandra + Spark = Awesome- NoSQL matters Barcelona 2014NoSQLmatters
Johnny Miller – Cassandra + Spark = Awesome
This talk will discuss how Cassandra and Spark can work together to deliver real-time analytics. This is a technical discussion that will introduce the attendees to the basic principals on Cassandra and Spark, why they work well together and examples usecases.
APACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van NiekerkSpark Summit
Many data scientists are already making heavy usage of the Jupyter ecosystem for analyzing data using interactive notebooks.
Apache Toree (incubating) is a Jupyter kernel designed to act as a gateway to Spark by enabling users Spark from standard Jupyter notebooks. This allows users to easily integrate Spark into their existing Jupyter deployments, This allows users to easily move between languages and contexts without needing to switch to a different set of tools.
Apache Toree is designed expressly for interactive work. It supports interpreters in Scala, Python, and R.
In this talk, I will cover the design of Toree, how it interacts with the Jupyter ecosystem and various ways in which users can extend the functionality of Apache Toree via a powerful plugin system.
Spark and Object Stores —What You Need to Know: Spark Summit East talk by Ste...Spark Summit
If you are running Apache Spark in cloud environments, Object Stores —such as Amazon S3 or Azure WASB— are a core part of your system. What you can’t do is treat them like “just another filesystem” —do that and things will, eventually, go horribly wrong.
This talk looks at the object stores in the cloud infrastructures, including underlying architectures., compares them to what a “real filesystem” is expected to do and shows how to use object stores efficiently and safely as sources of and destinations of data.
It goes into depth on recent “S3a” work, showing how including improvements in performance, security, functionality and measurement —and demonstrating how to use make best use of it from a spark application.
If you are planning to deploy Spark in cloud, or doing so today: this is information you need to understand. The performance of you code and integrity of your data depends on it.
Scalability is one key feature that drives OpenStack adoption around the globe. With the dispatch of OpenStack's 14th release dubbed Newton, OpenStack makers affirm that it now incorporates enhanced scaling abilities. The Operations team at the OpenStack Innovation Center decided to demonstrate that OpenStack is effortlessly scalable to 500 nodes in a single region. To complete this task, we decided to work on a tool that allows them to target a set number of servers and "partition" them; if you have 10 physical servers, split those into 100 virtual servers, and use them as compute nodes within an OpenStack deployment. In this session we will examine the simulator tool, deep dive into its internals, and run through a demo. You will learn how you might utilize it to test the scaling of various OpenStack services and decrease the cost of creating labs and staging environments within your company.
Breaking Spark: Top 5 mistakes to avoid when using Apache Spark in productionNeelesh Srinivas Salian
Spark has been growing in deployments for the past year. The increasing amount of data being analyzed and processed through the framework is massive and continues to push the boundaries of the engine. Drawing on experiences across 150+ production deployments, Neelesh Srinivas Salian explores common issues observed in a cluster environment setup with Apache Spark and offers guidelines to help setup a real-world environment when planning an Apache Spark deployment in a cluster. Attendees can use these observations to improve the usability and supportability of Apache Spark and avoid such issues in their projects.
Topics include:
Scaling the architecture
Memory configurations
End-user code
Incompatible dependencies
Administration- and operation-related issues
Kerberizing Spark: Spark Summit East talk by Abel Rincon and Jorge Lopez-MallaSpark Summit
Spark had been elected, deservedly, as the main massive parallel processing framework, and HDFS is the one of the most popular Big Data storage technologies. Therefore its combination is one of the most usual Big Data’s use cases. But, what happens with the security? Can these two technologies coexist in a secure environment? Furthermore, with the proliferation of BI technologies adapted to Big Data environments, that demands that several users interacts with the same cluster concurrently, can we continue to ensure that our Big Data environments are still secure? In this lecture, Abel and Jorge will explain which adaptations of Spark´s core they had to perform in order to guarantee the security of multiple concurrent users using a single Spark cluster, which can use any of its cluster managers, without degrading the outstanding Spark’s performance.
Off-the-shelf, Perforce is a remarkably fast and stable system. Perforce admins can change that, however, by adding triggers that are slow, crash, or have external dependencies that cause them to fail. Find out how the Pixar team has addressed this challenge with a checkin-driven work queue system that allows for independent, parallel and asynchronous execution of checkin logic. They'll also briefly describe how they notify 10,000 computers when a checkin is completed.
AWS re:Invent 2019 - DAT328 Deep Dive on Amazon Aurora PostgreSQLGrant McAlister
Amazon Aurora with PostgreSQL compatibility is a relational database service that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases. In this session, we review the functionality in order to understand the architectural differences that contribute to improved scalability, availability, and durability. You'll also get a deep dive into the capabilities of the service and a review of the latest available features. Finally, we walk you through the techniques that you can use to migrate to Amazon Aurora.
Accelerating Shuffle: A Tailor-Made RDMA Solution for Apache Spark with Yuval...Spark Summit
The opportunity in accelerating Spark by improving its network data transfer facilities has been under much debate in the last few years. RDMA (remote direct memory access) is a network acceleration technology that is very prominent in the HPC (high-performance computing) world, but has not yet made its way to mainstream Apache Spark. Proper implementation of RDMA in network-oriented applications can improve scalability, throughput, latency and CPU utilization. In this talk we are going to present a new RDMA solution for Apache Spark that shows amazing improvements in multiple Spark use cases. The solution is under development in our labs, and is going to be released to the public as an open-source plug-in.
Teradata Partners Conference Oct 2014 Big Data Anti-PatternsDouglas Moore
Big Data Anti-Patterns: Lessons from the Front Lines
Drawn from over 50 client engagements, big data anti-patterns are common practices that make for bad solutions.
Running Emerging AI Applications on Big Data Platforms with Ray On Apache SparkDatabricks
With the rapid evolution of AI in recent years, we need to embrace advanced and emerging AI technologies to gain insights and make decisions based on massive amounts of data. Ray (https://github.com/ray-project/ray) is a fast and simple framework open-sourced by UC Berkeley RISELab particularly designed for easily building advanced AI applications in a distributed fashion.
Choosing an HDFS data storage format- Avro vs. Parquet and more - StampedeCon...StampedeCon
At the StampedeCon 2015 Big Data Conference: Picking your distribution and platform is just the first decision of many you need to make in order to create a successful data ecosystem. In addition to things like replication factor and node configuration, the choice of file format can have a profound impact on cluster performance. Each of the data formats have different strengths and weaknesses, depending on how you want to store and retrieve your data. For instance, we have observed performance differences on the order of 25x between Parquet and Plain Text files for certain workloads. However, it isn’t the case that one is always better than the others.
An Update on the A400M Program: June 2014ICSA, LLC
In this presentation to the press at the Airbus Trade Media Briefing, Rafael Tentor-Aunon, head of the A400M program, provided an update on June 9,2014. The first two A400Ms are in service in France and Second Line of Defense has visited the squadron in France and will report soon on the introduction of the aircraft into service.
My session @ UXcamp Switzerland, Zürich, May 23, 2014.
I'm looking at the UX job landscape in Switzerland and am asking, where those jobs are that allow you to strategically influence the product or service you're defining the UX for.
Primer Taller Gold Standard en Colombia: Introducción GS y mercados de carbon...Fundación Natura Colombia
Fundación Natura y The Gold Standard Foundation (GSF) llevaron a cabo el Primer Taller Gold Standard en Colombia sobre estándares, metodologías y experiencias nacionales en el desarrollo de estrategias y proyectos de Estufas Eficientes de Leña. Este evento contó con la presencia de expertos nacionales e internacionales. Compartimos una de las presentaciones.
Lesen Sie unsere aktuelle Broschüre zum Thema "Forschung und Entwicklung in Österreich" und erfahren Sie alles zum Forschungsstandort Österreich. Welche Forschungsprämie gibt es, wie hoch ist unsere Forschungsquote, welche Forschungsinstitute gibt es, und vieles mehr...
Securing Critical Raw Materials for Europe - Presentation at IMMC Istanbul 2016 Martin Tauber
18th International Metallurgy & Materials Congress was held in the TUYAP Fair, Convention and Congress Center-Beylikdüzü / ISTANBUL between 29 September - 01 October 2016 together with the 18th International Metallurgical and Materials Conference organized by UCTEA the Chamber of Metallurgical and Materials Engineers, is the most comprehensive international event held in Turkey for 45 years which embraces all sectors in this area.Great honour to speak about Critical Raw Materials for Europe and Turkeys role in that. My thanks to the organising committee for the invitation.
Hadoop Operations: Keeping the Elephant Running SmoothlyMichael Arnold
Pune Hadoop Admins Meetup
From its beginnings years ago at large Internet sites, Hadoop is spreading everywhere. There are multitudes of cool and interesting things that Hadoop allows your organization to do, but running the actual infrastructure may not be as sexy as the application(s) running on top. Operations can be pure grunt-work, exacerbated by the fact that there is usually one machine out of dozens (or more) that is throwing a wrench in the works. In this talk, I will cover my experiences of running Hadoop, provide some recommended practices to simplify your days and nights in the trenches, and highlight some of the lessons learned along the way.
Pilot Hadoop Towards 2500 Nodes and Cluster RedundancyStuart Pook
Hadoop has become a critical part of Criteo's operations. What started out as a proof of concept has turned into two in-house bare-metal clusters of over 2200 nodes. Hadoop contains the data required for billing and, perhaps even more importantly, the data used to create the machine learning models, computed every 6 hours by Hadoop, that participate in real time bidding for online advertising.
Two clusters do not necessarily mean a redundant system, so Criteo must plan for any of the disasters that can destroy a cluster.
This talk describes how Criteo built its second cluster in a new datacenter and how to do it better next time. How a small team is able to run and expand these clusters is explained. More importantly the talk describes how a redundant data and compute solution at this scale must function, what Criteo has already done to create this solution and what remains undone.
Sanger OpenStack presentation March 2017Dave Holland
A description of the Sanger Institute's journey with OpenStack to date, covering RHOSP, Ceph, S3, user applications, and future plans. Given at the Sanger Institute's OpenStack Day.
UKOUG version of a presentation trying to establish the sensible limits of parallelism on a couple of hardware configurations. Detailed white paper is at http://oracledoug.com/px_slaves.pdf
It’s 2017, and big data challenges are as real as they get. Our customers have petabytes of data living in elastic and scalable commodity storage systems such as Azure Data Lake Store and Azure Blob storage.
One of the central questions today is finding insights from data in these storage systems in an interactive manner, at a fraction of the cost.
Interactive Query leverages [Hive on LLAP] in Apache Hive 2.1, brings the interactivity to your complex data warehouse style queries on large datasets stored on commodity cloud storage.
In this session, you will learn how technologies such as Low Latency Analytical Processing [LLAP] and Hive 2.x are making it possible to analyze petabytes of data with sub second latency with common file formats such as csv, json etc. without converting to columnar file formats like ORC/Parquet. We will go deep into LLAP’s performance and architecture benefits and how it compares with Spark and Presto in Azure HDInsight. We also look at how business analysts can use familiar tools such as Microsoft Excel and Power BI, and do interactive query over their data lake without moving data outside the data lake.
Speaker
Ashish Thapliyal, Principal Program Manager, Microsoft Corp
Big Data and OpenStack, a Love Story: Michael Still, RackspaceOpenStack
Big Data and OpenStack, a Love Story
Audience: Intermediate
Topic: Storage
Abstract: Increasingly we’re being asked to build out clusters of machines to solve big data problems. These clusters can become quite large, reaching up to thousands of machines. Of course, our operational budgets don’t scale linearly like our machine counts do, and we’re asked to do more and more with less. This talk will explore how organisations around the world are using OpenStack to automate the management of their big data implementations, harnessing interesting characteristics of big data workloads along the way.
Speaker Bio: Michael Still, Rackspace
OpenStack core developer and former Nova PTL, as well as experienced software and reliability engineer. Part of the team that grew Google Mobile to being a billion dollar business. Director of linux.conf.au 2013. Author of The Definitive Guide to ImageMagick (www.imagemagickbook.com) and Practical MythTV (www.mythtvbook.com) from Apress, as well as a bunch of articles.
OpenStack Australia Day Government - Canberra 2016
https://events.aptira.com/openstack-australia-day-canberra-2016/
Good software quality matters. How about Big Data and Hadoop? Every day we process large amount of medical information, population scale. Any question why we pay attention to quality? Let's disclosure some points based on our practical cases.
- Quality assurance in Hadoop: fears and reality.
- Practical options for testing infrastructure.
- Test strategy specials.
- Lowering verification efforts.
- Growth driven development: verification at scale.
- I want better elephant: current gaps.
Oracle hardware includes a full-suite of scalable engineered systems, servers, and storage that enable enterprises to optimize application and database performance, protect crucial data, and lower costs.
With Oracle, customers have freedom from the complexity of having multiple databases, analytics tools, and machine learning environments. Oracle's data management platform makes it easier and faster for application developers to create microservices-based applications with multiple data types.
Similar to Welcome to a Computing Revolution - Alex Lesser (20)
What it Means to be a Next-Generation Managed Service ProviderDatadog
Webinar that took place on July 12 2017.
The emergence of cloud-based infrastructure has dramatically reshaped
the IT landscape for managed service providers and their customers. Infrastructure is now dynamic, elastic, and instantly available to any individual or organization.
Customers are becoming increasingly aware of the value of cloud services, and with this heightened awareness comes the desire to partner with providers who can guide them toward innovative business solutions and high-performance environments. But in this new landscape, gaining insight into the status and performance of dynamic infrastructure and applications is more challenging than ever.
Join us as we host Thomas Robinson, Solutions Architect at Amazon Web Services, and Patrick Hannah, VP of Engineering at CloudHesive, to discuss what it means to be a next-generation managed service provider and how Datadog provides visibility into modern cloud infrastructure and helps you adopt new approaches to remain competitive in this ever-changing environment.
Lifting the Blinds: Monitoring Windows Server 2012Datadog
Operating systems monitor resources continuously in order to effectively schedule processes.
In this webinar, Evan Mouzakitis (Datadog) discusses how to get operational data from Windows Server 2012 using a variety of native tools.
A granular look into The Do's and Don't of Post Incident Analysis, featuring Jason Hand - DevOps Evangelist - from VictorOps and Jason Yee - Technical Writer/Evangelist - from Datadog.
Topics include a breakdown of the process in the following order:
- Service disruptions
- Detection
- Diagnosis
- Post-incident analysis
- Framework
Go through the result of our latest large-scale study about Docker usage in real environment. Analyze and see the impact for operations and monitoring.
PyData NYC 2015 - Automatically Detecting Outliers with Datadog Datadog
Monitoring even a modestly-sized systems infrastructure quickly becomes untenable without automated alerting. For many metrics it is nontrivial to define ahead of time what constitutes “normal” versus “abnormal” values. This is especially true for metrics whose baseline value fluctuates over time. To make this problem more tractable, Datadog provides outlier detection functionality to automatically identify any host (or group of hosts) that is behaving abnormally compared to its peers.
These slides cover the algorithms we use for outlier detection, and show how easy they are to implement using Python. This presentation also covers the lessons we've learned from using outlier detection on our own systems, along with some real-life examples on how to avoid false positives and negatives.
Learn more at www.datadoghq.com.
Monitoring Docker at Scale - Docker San Francisco Meetup - August 11, 2015Datadog
In this session I showed building a multi-container app from beginning to end, using Docker, Docker-Machine, Docker-Compose and everything in between. You can even try it out yourself using the link in the deck to a repo on GitHub.
Monitoring Docker containers - Docker NYC Feb 2015Datadog
Alexis goals this presentation are three-fold:
1) Dive into key Docker metrics
2) Explain operational complexity. In other words I want to take what we have seen on the field and show you where the pain points will be.
3) Rethink monitoring of Docker containers. The old tricks won’t work.
Containerization (à la Docker) is increasing the elastic nature of cloud infrastructure by an order of magnitude. If you have adopted Docker, or are considering it, you are probably facing questions like:
- How many containers can you run on a given Amazon EC2 instance type?
- Which metric should you look at to measure contention?
- How do you manage fleets of containers at scale?
Datadog’s CTO, Alexis Lê-Quôc, presents the challenges and benefits of running Docker containers at scale. Alexis explains how to use quantitative performance patterns to monitor your infrastructure at the new level of magnitude and increased complexity introduced by containerization.
In this presentation, Mike walks through the philosophical shift of treating the servers that you have in-house as if they were part of a “cloud” and disposable, and then jumps into a technical demonstration of how to actually tear down and reconstruct your infrastructure at a moment’s notice.
What I’m going to talk about
‣Briefly we do and for whom
‣Where we started
‣The kind of data we deal with
‣How it fits altogether
‣A few things we learned along the way
‣Q+A
Examination of the old way of computing and the new way - the Dev & Ops way
Aggregate - the more tools the merrier
Correlate - because issues spread
Collaborate - you can't solve problems on your own
Analyze - not just alert whack-a-mole
Datadog is monitoring that does not suck. It's metrics friendly, people friendly and developer friendly monitoring.
Learn more at https://www.datadoghq.com/
Dig into an alert using Datadog graphs to correlate data from all of your system and determine and resolve the cause of your performance issue.
Learn more about Datadog's infrastructure monitoring at https://www.datadoghq.com
Best practices for monitoring your IT infrastructure using StatsD. Find dashboard examples here: https://p.datadoghq.com/sb/9b246c4ade
Monitor StatsD easily with Datadog. Learn more at https://www.datadoghq.com
Alerting: more signal, less noise, less painDatadog
Is this talk for me?
✓I am or will be on-call
✓I don’t like being alerted
✓I want the pain to go away
The next 40 minutes
1. Alerts == pain?
2. Measure alerts
3. Concrete (& fun) steps
Learn more about Datadog's infrastructure monitoring as a service at https://www.datadoghq.com.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
3. ž NY TEAMS I LIKE – Yankees & Giants
ž NY TEAMS I DON’T LIKE – Jets & Mets
ž Proud Father of 6 Week Old Baby Girl
¡ Marley Blaze Lesser
4. ž Founded in 1986
ž Hardware Geeks
¡ Family Run Biz That Actually Did Start in My
Parents Garage
ž 99% of Systems are Shipped
Linux Based.
5. ž We try the things other companies won’t.
¡ Example – Former Google Operations Manager
Wanted A Server to Powers On / Off and Removable
from Both Front / Rear of Rack.
¡ We designed and manufactured it. Now it is a
Patent Pending Chassis Design.
6. ž We try the things other companies won’t.
¡ Example – Former Google Operations Manager
Wanted A Server to Powers On / Off and Removable
from Both Front / Rear of Rack.
¡ We designed and manufactured it. Now it is a
Patent Pending Chassis Design.
7. ž True Tier 1 Manufacturer will work with you to
design servers from bare metal.
ž Well known in the HPC market with over 1500
“Supercomputers” delivered to 36+ countries.
ž Offer a full suite of data center services to help
small DevOps team go global.
9. ž Offer a full suite of data center services to
help small DevOps team go global.
¡ International Logistics
¡ Remote Hands Support
¡ Monitoring
¡ Inventory Management
¡ Configuration Change Management
10. ž Offer a full suite of data center services to
help small DevOps team go global.
¡ International Logistics
¡ Remote Hands Support
¡ Monitoring
¡ Inventory Management
¡ Configuration Change Management
11. OpenX US Navy US Air Force
Integral Ads Lawrence Berkeley
National Lab
Los Alamos National Lab
Sailthru Idaho National
Laboratory
Naval Research Center
Harvard University M.I.T Northwestern University
Stanford University California Institute of
Technology
University of California
Northrop Grumman MITRE Corporation Raytheon
Roche Diagnostics Motorola Life Technologies
Shell Energy ATK NASA
13. ž CloudOOP 12000
¡ Only Enterprise platform offering up to 50TB of
storage in 1U Rack Space.
¡ Supports 12 or 16 Xeon E5 Cores
¡ Supports up to 128GB ECC Memory
¡ Supports 12 x 3.5” SATA / SAS Drives
¡ Optional 2 x 2.5” Drive for OS
¡ Optional Redundant Power Supply
14. ž CloudOOP 12000
¡ Lowest possible power draw (appx. 5Watts / TB)
¡ Proven compatible with Cloudera, MapR, Horton
Works, Apace
¡ We do have Benchmark Test Results Available. I’d
be happy to share them.
15. ž CloudRax
¡ Fully Integrated Rack Solution Offering Up to 2
Petabytes in Standard 42Rack.
¢ All Necessary Hardware Installed & Cabled
16. ž CloudRax
¡ Includes pre-insallation of Hadoop Distribution
¡ Includes pre-installation of Hawkeye Management
System.
¢ Monitoring Utility
¢ Node Replication / Imaging Utility
¢ Network Setting Preconfigured
¢ Security Settings
¢ Power Management
17. ž CloudSeek 1000xS
¡ Offers up to 10 x SSD or SFF drives in 1U Rack
Space.
ž MongoDB Partner
¡ Free & Discounted Passes for Upcoming MongoDB
show in New York (June 21)