ARCHITECTING INFLUXENTERPRISE FOR SUCCESSInfluxData
In this session, everyone will learn how to architect their own InfluxEnterprise clusters to be performant and resilient whether in a single data center or spread across multiple datacenters.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1OKo5FN.
Danny Yuan discusses how stream processing is used in Uber's real-time system to solve a wide range of problems, including but not limited to real-time aggregation and prediction on geospatial time series, data migration, monitoring and alerting, and extracting patterns from data streams. Yuan also presents the architecture of the stream processing pipeline. Filmed at qconsf.com.
Danny Yuan is a software engineer in Uber. He's currently working on streaming systems for Uber's logistics platform. Prior to joining Uber, he worked on building Netflix's cloud platform. His work includes predictive autoscaling, distributed tracing service, real-time data pipeline that scaled to process hundreds of billions of events every day, and Netflix's low-latency crypto services.
Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...Spark Summit
Data scientists write SQL queries everyday. Very often they know how to write correct queries but don’t know why their queries are slow. This is more obvious in Spark than in Redshift as Spark requires additional tuning such as caching while Redshift does heavy lifting behind the scene.
In this talk I will cover a few lessons we learned from migrating one of the biggest table here (900M+ rows/day) from AWS Redshift to Spark.
Specifically:
– Why and how do we migrate?
– How do we tune the query for Spark to gain 10x speed vs direct translated from Redshift
– How do we scale the team on Spark (with 80+ people in our data science team)
Real World Analytics with Solr Cloud and SparkQAware GmbH
Apache Big Data Conference 2016, Vancouver BC: Talk by Johannes Weigend (@JohannesWeigend, CTO at QAware).
Abstract: Apache Solr is a distributed NoSQL database with impressive search capabilities. Apache Spark makes M/R faster and richer. In this code-intense session shows how to combine both to solve real-time search and processing problems. The demos feature a portable Solr Cloud / Spark Cluster based on Intel NUC Hardware.
Very Large Data Files, Object Stores, and Deep Learning—Lessons Learned While...Databricks
In this session, IBM will present details on advanced Apache Spark analytics currently being performed through a collaborative project with the SETI Institute, NASA, Swinburne University, Stanford University and IBM. The Allen Telescope Array in northern California has been continuously scanning the skies for over two decades, generating data archives with over 200 million signal events.
Come and learn how astronomers and researchers are using Apache Spark, in conjunction with assets such as IBM’s Cognitive Compute Cluster with over 700 GPUs, to train neural net models for signal classification, and to perform computationally intensive Spark workloads on multi-terabyte binary signal files. The speakers will also share details on one of the key components of this implementation: Stocator, an open source (Apache License 2.0) object store connector for Hadoop and Apache Spark, specifically designed to optimize their performance with object stores. Learn how Stocator works, and see how it was able to greatly improve performance and reduce the quantity of resources used, both for ground-to-cloud uploads of very large signal files, and for subsequent access of radio data for analysis using Spark.
ARCHITECTING INFLUXENTERPRISE FOR SUCCESSInfluxData
In this session, everyone will learn how to architect their own InfluxEnterprise clusters to be performant and resilient whether in a single data center or spread across multiple datacenters.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1OKo5FN.
Danny Yuan discusses how stream processing is used in Uber's real-time system to solve a wide range of problems, including but not limited to real-time aggregation and prediction on geospatial time series, data migration, monitoring and alerting, and extracting patterns from data streams. Yuan also presents the architecture of the stream processing pipeline. Filmed at qconsf.com.
Danny Yuan is a software engineer in Uber. He's currently working on streaming systems for Uber's logistics platform. Prior to joining Uber, he worked on building Netflix's cloud platform. His work includes predictive autoscaling, distributed tracing service, real-time data pipeline that scaled to process hundreds of billions of events every day, and Netflix's low-latency crypto services.
Migrating from Redshift to Spark at Stitch Fix: Spark Summit East talk by Sky...Spark Summit
Data scientists write SQL queries everyday. Very often they know how to write correct queries but don’t know why their queries are slow. This is more obvious in Spark than in Redshift as Spark requires additional tuning such as caching while Redshift does heavy lifting behind the scene.
In this talk I will cover a few lessons we learned from migrating one of the biggest table here (900M+ rows/day) from AWS Redshift to Spark.
Specifically:
– Why and how do we migrate?
– How do we tune the query for Spark to gain 10x speed vs direct translated from Redshift
– How do we scale the team on Spark (with 80+ people in our data science team)
Real World Analytics with Solr Cloud and SparkQAware GmbH
Apache Big Data Conference 2016, Vancouver BC: Talk by Johannes Weigend (@JohannesWeigend, CTO at QAware).
Abstract: Apache Solr is a distributed NoSQL database with impressive search capabilities. Apache Spark makes M/R faster and richer. In this code-intense session shows how to combine both to solve real-time search and processing problems. The demos feature a portable Solr Cloud / Spark Cluster based on Intel NUC Hardware.
Very Large Data Files, Object Stores, and Deep Learning—Lessons Learned While...Databricks
In this session, IBM will present details on advanced Apache Spark analytics currently being performed through a collaborative project with the SETI Institute, NASA, Swinburne University, Stanford University and IBM. The Allen Telescope Array in northern California has been continuously scanning the skies for over two decades, generating data archives with over 200 million signal events.
Come and learn how astronomers and researchers are using Apache Spark, in conjunction with assets such as IBM’s Cognitive Compute Cluster with over 700 GPUs, to train neural net models for signal classification, and to perform computationally intensive Spark workloads on multi-terabyte binary signal files. The speakers will also share details on one of the key components of this implementation: Stocator, an open source (Apache License 2.0) object store connector for Hadoop and Apache Spark, specifically designed to optimize their performance with object stores. Learn how Stocator works, and see how it was able to greatly improve performance and reduce the quantity of resources used, both for ground-to-cloud uploads of very large signal files, and for subsequent access of radio data for analysis using Spark.
Netflix viewing data architecture evolution - EBJUG Nov 2014Philip Fisher-Ogden
Netflix's architecture for viewing data has evolved as streaming usage has grown. Each generation was designed for the next order of magnitude, and was informed by learnings from the previous. From SQL to NoSQL, from data center to cloud, from proprietary to open source, look inside to learn how this system has evolved. (slides from a talk given at the East Bay Java Users Group MeetUp in Nov 2014)
Improved Applications with IPv6: an overviewCisco DevNet
A session in the DevNet Zone at Cisco Live, Berlin. Deployment of IPv6 is rapidly expanding. Support for IPv6 in applications is lagging behind. Vendors such as Apple are taking steps to rectify this situation and raise awareness to the problem. The session explains why this evolution happens now, including looking at Cisco collected adoption statistics and how IPv4 address exhaustion issues affect application performance. The majority of the session focuses on how applications should adopt and support IPv6. We start with simple dual-stack IPv4/IPv6 support in applications for different operating systems to achieve functional parity with IPv4. We then discuss some of the more interesting ways of delivering applications which are only feasible with IPv6 and we explain how they can help to improve application functionality. The session includes aspects such as utilizing multiple IPv6 addresses, "happy eyeball" techniques, using different type of IPv6 addresses and more. The session will discuss existing and evolving technologies.
Real Time Data Processing With Spark Streaming, Node.js and Redis with Visual...Brandon O'Brien
Contact:
https://www.linkedin.com/in/brandonjobrien
@hakczar
Code examples available at https://github.com/br4nd0n/spark-streaming and https://github.com/br4nd0n/spark-viz
A demo and explanation of building a streaming application using Spark Streaming, Node.js and Redis with a real time visualization. Includes discussion of internals of Spark and Spark streaming including RDD partitioning and code and data distribution and cluster resource allocation.
Introduction to Streaming Distributed Processing with StormBrandon O'Brien
Contact:
https://www.linkedin.com/in/brandonjobrien
@hakczar
Introducing streaming data concepts, Storm cluster architecture, Storm topology architecture, and demonstrate working example of a WordCount topology for SIGKDD Seattle chapter meetup.
Presented by Brandon O'Brien
Code example: https://github.com/OpenDataMining/brandonobrien
Meetup: http://www.meetup.com/seattlesigkdd/events/222955114/
Ready to build something with Cassandra but don't want to use something heavy like Java or C#? Let's take a quick tour though the DataStax Ruby and NodeJS drivers. You'll see how they are easy to use and just as powerful as any other driver. Connect, insert, query like a pro, and if there is a server failure? No problem. I'll show you how these drivers keep your app online despite server failures. This is a quick talk, but we'll have plenty of time to make you productive. It's that easy!
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per DayAnkur Bansal
Building data pipelines is pretty hard! Building a multi-datacenter active-active real time data pipeline for multiple classes of data with different durability, latency and availability guarantees is much harder.
Real time infrastructure powers critical pieces of Uber (think Surge) and in this talk we will discuss our architecture, technical challenges, learnings and how a blend of open source infrastructure (Apache Kafka and Samza) and in-house technologies have helped Uber scale.
stackconf 2020 | Ignite talk: Opensource in Advanced Research Computing, How ...NETWAYS
Opensource software is becoming a pillar in our everyday life, leveraged by our cell phones, our transportation systems and on the websites we visit. In this quick talk, we will have a look on how Canada’s Advanced Research Computing (“ARC”) organizations use opensource software to deploy and operate some of the largest Supercomputers and Cloud deployments on Earth. We will briefly introduce the systems and dig deeper into the opensource technologies that together make the magic happen !
Engineering Leader opportunity @ Netflix - Playback Data SystemsPhilip Fisher-Ogden
Across the globe, 75M Netflix members love watching 125M hours per day of TV shows and movies. They love the ease of starting on one device and resuming on another, and the Playback Data Systems team makes that happen. We’re looking for a senior engineering manager to lead this high-impact team at Netflix.
Attributions for images:
https://www.flickr.com/photos/theholyllama/5738164504/ and https://www.flickr.com/photos/brewbooks/7780990192/, no changes made, https://creativecommons.org/licenses/by-sa/2.0/
https://www.flickr.com/photos/crschmidt/2956721498/, no changes made, https://creativecommons.org/licenses/by/2.0/
InfluxEnterprise Architecture Patterns by Tim Hall & Sam DillardInfluxData
In this InfluxDays NYC 2019 presentation, InfluxData VP of Products Tim Hall and Sales Engineer Sam Dillard discuss architecture patterns with InfluxEnterprise time series platform. They cover an overview of InfluxEnterprise, features, ingestion and query rates, deployment examples, replication patterns, and general advice. Presentation highlights include InfluxEnterprise cluster architecture and how to determine if you're ready for adopting InfluxEnterprise.
This is the talk I gave at the Seattle Spark Meetup in March, 2015. I discussed some Spark Streaming fundamentals, integration points with Kafka, Flume etc.
Teaching Apache Spark Clusters to Manage Their Workers Elastically: Spark Sum...Spark Summit
Devops engineers have applied a great deal of creativity and energy to invent tools that automate infrastructure management, in the service of deploying capable and functional applications. For data-driven applications running on Apache Spark, the details of instantiating and managing the backing Spark cluster can be a distraction from focusing on the application logic. In the spirit of devops, automating Spark cluster management tasks allows engineers to focus their attention on application code that provides value to end-users.
Using Openshift Origin as a laboratory, we implemented a platform where Apache Spark applications create their own clusters and then dynamically manage their own scale via host-platform APIs. This makes it possible to launch a fully elastic Spark application with little more than the click of a button.
We will present a live demo of turn-key deployment for elastic Apache Spark applications, and share what we’ve learned about developing Spark applications that manage their own resources dynamically with platform APIs.
The audience for this talk will be anyone looking for ways to streamline their Apache Spark cluster management, reduce the workload for Spark application deployment, or create self-scaling elastic applications. Attendees can expect to learn about leveraging APIs in the Kubernetes ecosystem that enable application deployments to manipulate their own scale elastically.
Lambda Architecture: The Best Way to Build Scalable and Reliable Applications!Tugdual Grall
Lambda Architecture is a useful framework to think about designing big data applications. This framework has been built initially at Twitter. In this presentation you will learn, based on concrete examples how to build deploy scalable and fault tolerant applications, with a focus on Big Data and Hadoop.
This presentation was delivered at the OOP conference, Munich, Feb 2016
Rental Cars and Industrialized Learning to Rank with Sean DownesDatabricks
Data can be viewed as the exhaust of online activity. With the rise of cloud-based data platforms, barriers to data storage and transfer have crumbled. The demand for creative applications and learning from those datasets has accelerated. Rapid acceleration can quickly accrue disorder, and disorderly data design can turn the deepest data lake into an impenetrable swamp.
In this talk, I will discuss the evolution of the data science workflow at Expedia with a special emphasis on Learning to Rank problems. From the heroic early days of ad-hoc Spark exploration to our first production sort model on the cloud, we will explore the process of industrializing the workflow. Layered over our story, I will share some best practices and suggestions on how to keep your data productive, or even pull your organization out of the data swamp.
Big Data Day LA 2015 - Big Data Day LA 2015 - Applying GeoSpatial Analytics u...Data Con LA
This session will explore how to apply GeoSpatial analytics using Apache Spark on high-velocity streaming (data-in-motion) and high-volume batch (data-at-rest). Demonstrations will be performed throughout the session to cement these concepts.
(BDT403) Netflix's Next Generation Big Data Platform | AWS re:Invent 2014Amazon Web Services
As Netflix expands their services to more countries, devices, and content, they continue to evolve their big data analytics platform to accommodate the increasing needs of product and consumer insights. This year, Netflix re-innovated their big data platform: they upgraded to Hadoop 2, transitioned to the Parquet file format, experimented with Pig on Tez for the ETL workload, and adopted Presto as their interactive querying engine. In this session, Netflix discusses their latest architecture, how they built it on the Amazon EMR infrastructure, the contributions put into the open source community, as well as some performance numbers for running a big data warehouse with Amazon S3.
Detecting and Catching the Bad Guys Using Deception
Traditional controls are well known for their short comings in the face of modern cyber-attacks. Cyber security technologies will make use of signature based, behavioral, Next Generation capabilities or attempt to augment capabilities by leveraging cloud based or on premise cyber analytics warehouse and threat intelligence feeds via indicator of compromise (IOC) or other mechanisms. Although the later efforts have increased organizational cyber capabilities, they only do so with proper investments in people, process and technology. Additionally, as attackers adapt to defenses, these controls begin to experience decreasing marginal rates of defensive capability.
Deception programs, architectures and technologies endeavor to augment existing cyber security capabilities through the use of honeypots or honey net (decoys) or breadcrumbs or broken glass (deceptions).
Advanced deception technologies are differentiated by the use of distributed deception technology which features agentless, simple deployment capabilities with lightweight deceptions that leverage operating system objects deceive attackers into triggering alerts. Normal users would never trigger the deceptions as an attacker would, resulting in high fidelity alerting with near-zero false positives. Such technology consequently serves to not only augment cyber security capabilities post-breach but provides a new, highly effective post-breach cyber security capability along with precise real-time forensics.
James Muren is a strategist and delivers workshops in cyber security strategy, GRC and security architecture that are used to develop long-term strategies and tactical roadmaps for customers that addresses security for legacy and cloud architectures. As a strategic management consultant and having built fully capable cyber programs in the past, he helps mentor and lead teams for programs & projects in information technology & cyber security. James is primarily focused on the business benefits of cyber security, and the demonstration of those benefits through metrics that can be quickly communicated to executive leadership. By properly integrating security controls within a regulatory and policy context, security programs such as breach and incident response, data governance, forensics, etc. can properly demonstrate value, receive proper investment and adequately secure organizations.
James is also a researcher. His areas of research include: Continuous GRC, cyber analytics, Trusted Computing Group (TCG), Security Automation, Hardware & Software Security, ICS, SCADA, IOT, Malware Research, Full System Security Design Lifecycle and Leap Ahead technology.
Netflix viewing data architecture evolution - EBJUG Nov 2014Philip Fisher-Ogden
Netflix's architecture for viewing data has evolved as streaming usage has grown. Each generation was designed for the next order of magnitude, and was informed by learnings from the previous. From SQL to NoSQL, from data center to cloud, from proprietary to open source, look inside to learn how this system has evolved. (slides from a talk given at the East Bay Java Users Group MeetUp in Nov 2014)
Improved Applications with IPv6: an overviewCisco DevNet
A session in the DevNet Zone at Cisco Live, Berlin. Deployment of IPv6 is rapidly expanding. Support for IPv6 in applications is lagging behind. Vendors such as Apple are taking steps to rectify this situation and raise awareness to the problem. The session explains why this evolution happens now, including looking at Cisco collected adoption statistics and how IPv4 address exhaustion issues affect application performance. The majority of the session focuses on how applications should adopt and support IPv6. We start with simple dual-stack IPv4/IPv6 support in applications for different operating systems to achieve functional parity with IPv4. We then discuss some of the more interesting ways of delivering applications which are only feasible with IPv6 and we explain how they can help to improve application functionality. The session includes aspects such as utilizing multiple IPv6 addresses, "happy eyeball" techniques, using different type of IPv6 addresses and more. The session will discuss existing and evolving technologies.
Real Time Data Processing With Spark Streaming, Node.js and Redis with Visual...Brandon O'Brien
Contact:
https://www.linkedin.com/in/brandonjobrien
@hakczar
Code examples available at https://github.com/br4nd0n/spark-streaming and https://github.com/br4nd0n/spark-viz
A demo and explanation of building a streaming application using Spark Streaming, Node.js and Redis with a real time visualization. Includes discussion of internals of Spark and Spark streaming including RDD partitioning and code and data distribution and cluster resource allocation.
Introduction to Streaming Distributed Processing with StormBrandon O'Brien
Contact:
https://www.linkedin.com/in/brandonjobrien
@hakczar
Introducing streaming data concepts, Storm cluster architecture, Storm topology architecture, and demonstrate working example of a WordCount topology for SIGKDD Seattle chapter meetup.
Presented by Brandon O'Brien
Code example: https://github.com/OpenDataMining/brandonobrien
Meetup: http://www.meetup.com/seattlesigkdd/events/222955114/
Ready to build something with Cassandra but don't want to use something heavy like Java or C#? Let's take a quick tour though the DataStax Ruby and NodeJS drivers. You'll see how they are easy to use and just as powerful as any other driver. Connect, insert, query like a pro, and if there is a server failure? No problem. I'll show you how these drivers keep your app online despite server failures. This is a quick talk, but we'll have plenty of time to make you productive. It's that easy!
Hadoop summit - Scaling Uber’s Real-Time Infra for Trillion Events per DayAnkur Bansal
Building data pipelines is pretty hard! Building a multi-datacenter active-active real time data pipeline for multiple classes of data with different durability, latency and availability guarantees is much harder.
Real time infrastructure powers critical pieces of Uber (think Surge) and in this talk we will discuss our architecture, technical challenges, learnings and how a blend of open source infrastructure (Apache Kafka and Samza) and in-house technologies have helped Uber scale.
stackconf 2020 | Ignite talk: Opensource in Advanced Research Computing, How ...NETWAYS
Opensource software is becoming a pillar in our everyday life, leveraged by our cell phones, our transportation systems and on the websites we visit. In this quick talk, we will have a look on how Canada’s Advanced Research Computing (“ARC”) organizations use opensource software to deploy and operate some of the largest Supercomputers and Cloud deployments on Earth. We will briefly introduce the systems and dig deeper into the opensource technologies that together make the magic happen !
Engineering Leader opportunity @ Netflix - Playback Data SystemsPhilip Fisher-Ogden
Across the globe, 75M Netflix members love watching 125M hours per day of TV shows and movies. They love the ease of starting on one device and resuming on another, and the Playback Data Systems team makes that happen. We’re looking for a senior engineering manager to lead this high-impact team at Netflix.
Attributions for images:
https://www.flickr.com/photos/theholyllama/5738164504/ and https://www.flickr.com/photos/brewbooks/7780990192/, no changes made, https://creativecommons.org/licenses/by-sa/2.0/
https://www.flickr.com/photos/crschmidt/2956721498/, no changes made, https://creativecommons.org/licenses/by/2.0/
InfluxEnterprise Architecture Patterns by Tim Hall & Sam DillardInfluxData
In this InfluxDays NYC 2019 presentation, InfluxData VP of Products Tim Hall and Sales Engineer Sam Dillard discuss architecture patterns with InfluxEnterprise time series platform. They cover an overview of InfluxEnterprise, features, ingestion and query rates, deployment examples, replication patterns, and general advice. Presentation highlights include InfluxEnterprise cluster architecture and how to determine if you're ready for adopting InfluxEnterprise.
This is the talk I gave at the Seattle Spark Meetup in March, 2015. I discussed some Spark Streaming fundamentals, integration points with Kafka, Flume etc.
Teaching Apache Spark Clusters to Manage Their Workers Elastically: Spark Sum...Spark Summit
Devops engineers have applied a great deal of creativity and energy to invent tools that automate infrastructure management, in the service of deploying capable and functional applications. For data-driven applications running on Apache Spark, the details of instantiating and managing the backing Spark cluster can be a distraction from focusing on the application logic. In the spirit of devops, automating Spark cluster management tasks allows engineers to focus their attention on application code that provides value to end-users.
Using Openshift Origin as a laboratory, we implemented a platform where Apache Spark applications create their own clusters and then dynamically manage their own scale via host-platform APIs. This makes it possible to launch a fully elastic Spark application with little more than the click of a button.
We will present a live demo of turn-key deployment for elastic Apache Spark applications, and share what we’ve learned about developing Spark applications that manage their own resources dynamically with platform APIs.
The audience for this talk will be anyone looking for ways to streamline their Apache Spark cluster management, reduce the workload for Spark application deployment, or create self-scaling elastic applications. Attendees can expect to learn about leveraging APIs in the Kubernetes ecosystem that enable application deployments to manipulate their own scale elastically.
Lambda Architecture: The Best Way to Build Scalable and Reliable Applications!Tugdual Grall
Lambda Architecture is a useful framework to think about designing big data applications. This framework has been built initially at Twitter. In this presentation you will learn, based on concrete examples how to build deploy scalable and fault tolerant applications, with a focus on Big Data and Hadoop.
This presentation was delivered at the OOP conference, Munich, Feb 2016
Rental Cars and Industrialized Learning to Rank with Sean DownesDatabricks
Data can be viewed as the exhaust of online activity. With the rise of cloud-based data platforms, barriers to data storage and transfer have crumbled. The demand for creative applications and learning from those datasets has accelerated. Rapid acceleration can quickly accrue disorder, and disorderly data design can turn the deepest data lake into an impenetrable swamp.
In this talk, I will discuss the evolution of the data science workflow at Expedia with a special emphasis on Learning to Rank problems. From the heroic early days of ad-hoc Spark exploration to our first production sort model on the cloud, we will explore the process of industrializing the workflow. Layered over our story, I will share some best practices and suggestions on how to keep your data productive, or even pull your organization out of the data swamp.
Big Data Day LA 2015 - Big Data Day LA 2015 - Applying GeoSpatial Analytics u...Data Con LA
This session will explore how to apply GeoSpatial analytics using Apache Spark on high-velocity streaming (data-in-motion) and high-volume batch (data-at-rest). Demonstrations will be performed throughout the session to cement these concepts.
(BDT403) Netflix's Next Generation Big Data Platform | AWS re:Invent 2014Amazon Web Services
As Netflix expands their services to more countries, devices, and content, they continue to evolve their big data analytics platform to accommodate the increasing needs of product and consumer insights. This year, Netflix re-innovated their big data platform: they upgraded to Hadoop 2, transitioned to the Parquet file format, experimented with Pig on Tez for the ETL workload, and adopted Presto as their interactive querying engine. In this session, Netflix discusses their latest architecture, how they built it on the Amazon EMR infrastructure, the contributions put into the open source community, as well as some performance numbers for running a big data warehouse with Amazon S3.
Detecting and Catching the Bad Guys Using Deception
Traditional controls are well known for their short comings in the face of modern cyber-attacks. Cyber security technologies will make use of signature based, behavioral, Next Generation capabilities or attempt to augment capabilities by leveraging cloud based or on premise cyber analytics warehouse and threat intelligence feeds via indicator of compromise (IOC) or other mechanisms. Although the later efforts have increased organizational cyber capabilities, they only do so with proper investments in people, process and technology. Additionally, as attackers adapt to defenses, these controls begin to experience decreasing marginal rates of defensive capability.
Deception programs, architectures and technologies endeavor to augment existing cyber security capabilities through the use of honeypots or honey net (decoys) or breadcrumbs or broken glass (deceptions).
Advanced deception technologies are differentiated by the use of distributed deception technology which features agentless, simple deployment capabilities with lightweight deceptions that leverage operating system objects deceive attackers into triggering alerts. Normal users would never trigger the deceptions as an attacker would, resulting in high fidelity alerting with near-zero false positives. Such technology consequently serves to not only augment cyber security capabilities post-breach but provides a new, highly effective post-breach cyber security capability along with precise real-time forensics.
James Muren is a strategist and delivers workshops in cyber security strategy, GRC and security architecture that are used to develop long-term strategies and tactical roadmaps for customers that addresses security for legacy and cloud architectures. As a strategic management consultant and having built fully capable cyber programs in the past, he helps mentor and lead teams for programs & projects in information technology & cyber security. James is primarily focused on the business benefits of cyber security, and the demonstration of those benefits through metrics that can be quickly communicated to executive leadership. By properly integrating security controls within a regulatory and policy context, security programs such as breach and incident response, data governance, forensics, etc. can properly demonstrate value, receive proper investment and adequately secure organizations.
James is also a researcher. His areas of research include: Continuous GRC, cyber analytics, Trusted Computing Group (TCG), Security Automation, Hardware & Software Security, ICS, SCADA, IOT, Malware Research, Full System Security Design Lifecycle and Leap Ahead technology.
Rolling the Root Zone DNSSEC Key Signing Key, by Edward Lewis.
A presentation given at APNIC 42's DNS and INR Security session on Monday, 3 October 2016.
The Domain Name System (DNS) is a critical part of Internet infrastructure and the largest distributed Internet directory service. DNS translates names to IP addresses, a required process for web navigation, email delivery, and other Internet functions. However, the DNS infrastructure is not secure enough unless the security mechanisms such as Transaction Signatures (TSIG) and DNS Security Extensions (DNSSEC) are implemented. To guarantee the availability and the secure Internet services, it is important for networking professionals to understand DNS concepts, DNS Security, configurations, and operations.
This course will discuss the concept of DNS Operations in detail, mechanisms to authenticate the communication between DNS Servers, mechanisms to establish authenticity, and integrity of DNS data and mechanisms to delegate trust to public keys of third parties. Participant will be involved in Lab exercises and do configurations based on number of scenarios.
Using Riak for Events storage and analysis at Booking.comDamien Krotkine
At Booking.com, we have a constant flow of events coming from various applications and internal subsystems. This critical data needs to be stored for real-time, medium and long term analysis. Events are schema-less, making it difficult to use standard analysis tools.This presentation will explain how we built a storage and analysis solution based on Riak. The talk will cover: data aggregation and serialization, Riak configuration, solutions for lowering the network usage, and finally, how Riak's advanced features are used to perform real-time data crunching on the cluster nodes.
Deploying DNSSEC: A .LK Case Study
Sashika Suren (LK Domain Registry)
This session will explore LK Domain Registry’s technical solution for deploying DNSSEC support in the .LK registry. With a goal of making it easier for domain name holders to easily add DNSSEC, we will take a quick look at our DNSSEC implementation strategy, the status/progress of .LK signed domains, and our lessons learned and challenges for increasing the percentage of signed domain names.
23rd PITA AGM and Conference: DNS Security - A holistic view APNIC
Security Specialist Jamie Gillespie presents on DNS Security, examining the complex interactions of this system, from domain registration to name resolution, the security risks of each component, and the mitigation options currently available at 23rd PITA AGM and Annual Conference in Nadi, Fiji from 8 to 12 April 2019.
10 Ways to Scale Your Website Silicon Valley Code Camp 2019Dave Nielsen
Redis has 10 different data structures (String, Hash, List, Set, Sorted Set, Bit Array, Bit Field, Hyperloglog, Geospatial Index, Streams) plus Pub/Sub and many Redis Modules. In this talk, Dave will give 10 examples of how to use these data structures to scale your website. I will start with the basics, such as a cache and User session management. Then I demonstrate user generated tags, leaderboards and counting things with hyberloglog. I will with a demo of Redis Pub/Sub vs Redis Streams which can be used to scale your Microservices-based architecture.
Presenter: Adam Zeglin, CTO of Instaclustr
In this presentation we discuss a method of provisioning and running an Apache Cassandra deployment spilt between multiple heterogeneous data centers which, rather than allocating per-node public IPv4 addresses or configuring mesh VPNs, uses Port Address Translation (PAT) for node↔internet connectivity and is self- configuring and discoverable via DNS Service Discovery (DNS-SD or wide-area Bonjour). While Cassandra has built-in support for AWS EC2 multi-region/data centre topologies (via Ec2MultiRegionSnitch, etc), the existing solution requires the wasteful allocation of public IPv4 addresses per-node. Additionally there is little support for topologies that are either a mix of or deploy completely on alternative infrastructure providers. Our solution uses a single public IP address per data center, is provider-agnostic, doesn’t introduce the configuration and management overheads of a mesh VPN between data centres, and allows nodes to automatically discover each-other.
At this meetup Patrick McFadin, Solutions Architect at DataStax, will be discussing the most recently added features in Apache Cassandra 2.0, including: Lightweight transactions, eager retries, improved compaction, triggers, and CQL cursors. He'll also be touching on time series data with Apache Cassandra.
NoSQL – Data Center Centric Application EnablementDATAVERSITY
The growth of Datacenter infrastructure is trending out of bounds, along with the pace in user activity and data generation in this digital era. However, the nature of the typical application deployment within the data center is changing to accommodate new business needs. Those changes introduce complexities in application deployment architecture and design, which cascade into requirements for a new generation of database technology (NoSQL) destined to ease that complexity. This webcast will discuss the modern data centers data centric application, the complexities that must be dealt with and common architectures found to describe and prescribe new data center aware services. Well look at the practical issues in implementation and overview current state of art in NoSQL database technology solving the problems of data center awareness in application development.
Intellectual Property Protection―
Cross Roads between Ethics, Information Security, and Internal Audit
Richard (Rick) Brunner has more than 40 years experience in information security and technology, specializing in secure systems/application design and development, system architectures, information risks and controls, testing, and strategy and program management. Rick’s past assignment was as an Assistant Vice President, Security Strategy and Architecture at GM Financial and has worked in Healthcare, Finance, Human Resources, Military, and Intelligence. Rick has 32 years of military service, both active and reserves, rising to the rank of Colonel (0-6). He holds an Executive Jurist Doctorate degree, concentration in Law and Technology from Concord Law School; Master of Science degree in Computer Science, concentration in Information Systems Security from James Madison University; and a Bachelor of Science degree in Mathematics and Computer Science from University of Texas at San Antonio. Rick is an Assistant Faculty member at Collin College, instructing courses in their cyber security program and is an active member of Collin’s Cyber Security Advisory Board. Rick holds the following certifications:
• Certified Information Systems Security Professional (CISSP) (Certification Number: 375658)
• SABSA Chartered Security Architect - Foundation Certificate (SCF) (License SCF14020703)
• ITIL Foundation Certificate in IT Service Management (Certification Number: 37823)
Layered Security / Defense in Depth
One area that I have found that even seasoned security professionals have a problem with articulating is layered security (defense in depth). Most are familiar with their area of expertise (servers, networks, pen testing, etc.), but have never viewed security as a heterogeneous process. In my presentation I use a layered diagram to highlight what controls are in what layers, what controls interact across layers, and what a complete layered security model would look like vs. what a more typical company security model does look like.
Nathan Shepard
CISSP, CISM, CRISC, CISA
33 Years in IT.
21 Years in Information Security.
Information Security consulting at the corporate governance level.
Information Security management for outsourced InfoSec delivery.
Business Geekdom: 1 = 3 = 5
Each year a security team participates in several audits, meetings with the business and strategy meetings. Often times, security is seen as one imposing requirements that are either too difficult, impossible to manage or flat out ridiculous.
This is similar to a geek. A geek is defined, as, "an unfashionable or socially inept person." Is this socially ineptness actually just the lack of the ability to translate the passion of the security professional to the business professional?
In this presentation, I would like to cover how to create, establish and evangelize a framework that has one backend with several frontends. The backend is a common security control framework (not the UCF) and the front end translates to the various business units, audits and business strategies encountered in a security professionals profession each year.
Grant Gilliam is a Enterprise and Solutions Architect for CHRISTUS Health. Previously, Gilliam has been a security architect, senior security engineer and senior data security analyst. Industries worked in include healthcare, insurance, software and news media. Gilliam has also established and created his own business focusing in outsourcing non-competitive business tasks for allowing clients a strategic advantage over competitors by minimizing FTE and contractor headcount.
His educational background includes a Master of Science in Information Systems, focusing in Information Security, and Bachelor of Business Administration in Management Information Systems, both from Baylor University. The focus of his masters degree research was IT law and Intellectual Property. Gilliam also is a Certified Information Systems Security Professional, Certified Information Security Manager and Certified Information Systems Auditor.
Day in the Life of a Security Solutions Architect
I'd like to present my "Day in the Life of a Security Solutions Architect" at Hewlett Packard Enterprise. In this presentation, I'll go into detail of what exactly I do as a security architect, and my career progression which got me there. I'll speak about my daily activities, successful client engagements, skills required, etc. I'm happy to answer any questions from the audience, share insights, what I wish I had done earlier in my career, etc.
Marco Fernandes is a Security Solutions Architect at Hewlett Packard Enterprise. Prior to that I worked in IT in the defense industry and security consulting in the commercial world. I'm also President of the North Texas Cyber Security Association. I was born in Dallas, TX, and I obtained my Bachelor of Science in Business Computer Information Systems from the University of North Texas. I've In my free time I enjoy card games, reading, fitness, watching WWE wrestling, & helping my community.
Red, Amber, Green Status: The Human Dashboard
This session will outline the importance of presenting actionable metrics for the Security Awareness program. Oftentimes security programs are presented while omitting the most constant threat to Information Systems: the human. From a security awareness perspective, we will review analytics that include key performance indicators that may already be available to you; they just need to be added to the new human dashboard.
Laurianna Callaghan currently serves as a security consultant for Ana Academy, a Dallas based security training company. Previously, Laurianna worked with Dell where she was the creator of security analytics for a major healthcare customer which were presented at the 2016 IASAP conference. In addition, Laurianna has more than 21 years experience in various IT domains. She has served as the Director of Systems Engineering for a telemarketing firm, the UNIX/MVS Manager for a major airline and has IT experience in the healthcare, communications, transportation, education, retail, and other industry sectors. Laurianna holds both the CCNA Security and CISSP designations.
Hacking Performance Management, the Blue Green Game - With a live demo!
Dr. Branden R. Williams has almost twenty years of experience in technology and information security, both as a consultant and an executive. Branden co-founded a technology services company that provided the foundation to a prominent e-learning company. He has vast experience as a practitioner and consultant which included helping companies create user-centric security controls and models. His specialty is navigating complex landscapes—be it compliance, security, technology, or business—and finding innovative solutions that save companies money while reducing risk and improving performance. Along the way, he was a Consulting Director for VeriSign/AT&T, one of four CTOs at RSA, ISSA Distinguished Fellow, elected to the PCI Board of Advisors, and author of four books.
Assuming people are rational, we all do things to maximize our payoffs. It's why things like Enron, and the Sub Prime mortgage crisis happen. This demonstration will show you a key element to designing performance management systems that employees will hack to their advantage.
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesSanjeev Rampal
Talk presented at Kubernetes Community Day, New York, May 2024.
Technical summary of Multi-Cluster Kubernetes Networking architectures with focus on 4 key topics.
1) Key patterns for Multi-cluster architectures
2) Architectural comparison of several OSS/ CNCF projects to address these patterns
3) Evolution trends for the APIs of these projects
4) Some design recommendations & guidelines for adopting/ deploying these solutions.
ER(Entity Relationship) Diagram for online shopping - TAEHimani415946
https://bit.ly/3KACoyV
The ER diagram for the project is the foundation for the building of the database of the project. The properties, datatypes, and attributes are defined by the ER diagram.
This 7-second Brain Wave Ritual Attracts Money To You.!nirahealhty
Discover the power of a simple 7-second brain wave ritual that can attract wealth and abundance into your life. By tapping into specific brain frequencies, this technique helps you manifest financial success effortlessly. Ready to transform your financial future? Try this powerful ritual and start attracting money today!
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
3. About me
• University of North Texas Alum
• Class of 2017
• Majored in Computer Engineering
• Team Leader for Senior Design project for NASA
• President of UNT’s Club for Cyber Security
• National Cyber League
• Collegiate Cyber Defense Competition
• Goldman Sachs Analyst 2017 NAPA
• GS 2016 Summer Analyst (Intern)
#NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
32017-Nov-11th
4. Team IRIS
• Jacen R Kohler
• Tyler Alvarez
• Alex Fatum
#NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
42017-Nov-11th
5. How did I get a NASA Project?
Texas Space Grant Consortium
• Educational wing for NASA in Texas
• Open to all college students in Texas
• Other stated have equivilant programs
#NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
52017-Nov-11th
6. Current Situation
Internet in space uses static IPv4 addresses
Static addressing requires manually assigning addresses
Static addressing is not fault tolerant
duplicates are possible
The constantly varying propagation delays & line interfere with
traditional network handshakes
#NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
62017-Nov-11th
7. Current Situation
The current Tracking and Data Relay Satellite (TDRS) System are
merely repeaters and amplifies and have no routing capabilities
#NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
72017-Nov-11th
8. The Problem
The process should be automated
aka… Dynamic Addressing
Just like your phone on a coffee shop’s wifi
A protocol like this already exists
Dynamic Host Configuration Protocol (DHCP)
#NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
82017-Nov-11th
9. How to Solve this problem
Simulate a space network, down here on Earth including…
propagation delay
line-of-sight signal blocking
adequate complexity (enough parts to be realistic)
Find a way to implement DHCP in the simulated space network
#NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
92017-Nov-11th
10. #NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
102017-Nov-11th
Adequate Complexity
•Simulate a space network on earth
•10x Raspberry Pi 3
•Mission Control Center
•3x TDRS
•TDRS5, TDRS7, &TDRS9
•3x Spacecraft
• ISS, Hubble, Orion
•3x EVA Suits
12. #NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
122017-Nov-11th
Adequate Complexity
• Pull TLE data from CelesTrak
• Two Line Element (TLE)
• 3000 TLE data in the file that gets parsed
• Search for orbital element we need
• Extract the 6 Keplerian Elements from TLE
• 6 Keplerian elements are like an address
• Allow us to calculate time based locations
• Past, present, or future
13. #NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
132017-Nov-11th
Simulated Space Network
• One way link to TDRS in GeoSynchronous orbit takes
• Simulation must emulate these delays to be realistic
14. #NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
142017-Nov-11th
Simulated Space Network (cont.)
• Delays will be simulated using existing open source tools
designed for network testing
• tcconfig
+312.432ms
149460120
0
Hold for delay
Epoch time
received
Packet
Ingress
( distance / speed of light ) + time of insert
+312.437ms
149460120
1
+312.443ms
149460120
2
+312.449ms
149460120
3
+312.452ms
149460120
4
20. #NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
202017-Nov-11th
Project Management
• Agile methodologies
• Iterative Testing
• GitHub
• Slack
• Additional paperwork
• Status Reports
• Implementation Plan
• Test Plan
• Maintenance Plan
• User Guide
24. #NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
242017-Nov-11th
0
50
100
150
1494115200 1494201600 1494288000 1494374400 1494460800 1494547200 1494633600 1494720000
DHCPHandshakeDuration(milliseconds)
Epoch Time (seconds)
ISS & HST with TDRS5, TDRS7, & TDRS9 from 2017 May 07th to
13th
Traditional DHCP from Earth Pipelined DHCP from TDRS
26. Prerequisites
TDRSs must have compute power onboard
To act as DHCP Server
To calculate orbital mechanics
We launch a new one every year or so
#NTTXISSACSC5
Abusing Protocols for Dynamic Addressing in Spaces
262017-Nov-11th