Accumulo Summit 2015: From Big Data to Linked Data: Making Sense of Massive, Highly Connected, Multi-structured Datasets [Sponsored]

•Download as PPTX, PDF•

0 likes•455 views

Talk Abstract Linked Data Analysis is an approach designed to more efficiently and effectively interrogate and analyze highly connected Big Data. Linked Data Analysis leverages graph- style search, algorithms, and visualization to power pattern discovery, pattern matching, and anomaly detection. In this talk we will also discuss some primary use cases for Linked Data Analysis, including cyber incident detection and response Speakers Ely Kahn Co-founder and VP of Business Development, Sqrrl Ely Kahn is a co-founder and VP of Business Development for Sqrrl. Previously, Ely served in a variety of positions in the Federal Government, including Director of Cybersecurity at the National Security Staff in White House, Deputy Chief of Staff at the National Protection Programs Directorate in the Department of Homeland Security, and Director of Risk Management and Strategic Innovation in the Transportation Security Administration. Before his service in the Federal Government, Ely was a management consultant with Booz Allen Hamilton. Ely has a BA from Harvard University and a MBA from the Wharton School at the University of Pennsylvania. Adam Fuchs Chief Technology Officer, Sqrrl As the ChiefTechnology Officer and co-founder of Sqrrl,Adam Fuchs is responsible for ensuring that Sqrrl is leading the world in Big Data Infrastructure technology. Previously at the National Security Agency, Adam was an innovator and technical director for several database projects, handling some of the world’s largest and most diverse data sets. He is a co-founder of the Apache Accumulo project. Adam has a BS in Computer Science from the University of Washington and has completed extensive graduate-level course work at the University of Maryland.

Technology

Securely explore your data
SQRRL ENTERPRISE V2
Linked Data Analysis

© 2015 Sqrrl | All Rights Reserved
MOTIVATION: LOGS VS. LINKED DATA
LOGS
VS
.
LINKED DATA
2

© 2015 Sqrrl | All Rights Reserved
LINKED DATA
• Organizes data into entities
and relationships (links)
• More intuitive visualization
• Surfaces meaning & context
• Enables faster analysis
3

© 2015 Sqrrl | All Rights Reserved
SQRRL LINKED DATA ANALYSIS
4

© 2015 Sqrrl | All Rights Reserved
Sqrrl extends an open core of Apache Accumulo and Hadoop
TECHNOLOGY STACK
Visualization / API
Physical
Data Storage
Data Model
Processing
Interface
Audit
Cryptography
Labeling+Policy
Query Engine Bulk/Graph Processing
Raw Events Linked Data
HDFS Accumulo+
Commodity Hardware / Cloud
5

Version 7 of the Elastic Stack adds powerful new features to the popular open source platform for search, logging, and analytics. Come hear directly from Elastic engineers and architecture team members on powerful new additions like GIS functionality and frozen-tier search. Plus, hear about the full range of orchestration options for getting the most out of your deployments, however and wherever you choose to run them. This session is sponsored by Elastic.

Business Intelligence in Minutes with Amazon Athena and Amazon QuickSight

Amazon Web Services

Leveraging cloud database connectors to automate analytics in alteryx

Grazitti Interactive

SnapLogic Live: Big Data Integration

SnapLogic

SnapLogic Live: Powering Cloud Analytics

SnapLogic

Search based analytics are a critical function for any organization, and so it’s no surprise that Elasticsearch is the go-to engine to power these queries. However, the limitations of today’s traditional x86-based data analytics infrastructures often hinder or delay companies from getting the immediate insights they need to run a smart, data-driven business due to costly and time-consuming data preparation and indexing. To solve this, we’ll show you how to leverage FPGA/x86 heterogeneous compute technology to eliminate indexing and transformation, in addition to providing enhanced Elasticsearch functionality. By leveraging FPGA and x86-based heterogeneous architectures either on-premise or in the cloud via the Amazon F1 platform companies can eliminate data indexing and transformation, accelerate searches and extend Elasticsearch capabilities to: • Accelerate workflows with the ability to deploy pre-index and post-index searches • Speed search and analysis across unstructured data and JSON, XML, LOGs, CSV, TSV and other files with no ETL • Increase edit distance values up to 10 or more using Fuzzy Hamming or Levenshtein searches t-index searches • Speed search and analysis across unstructured data and JSON, XML, LOGs, CSV, TSV and other files with no ETL • Enhance wildcard searches to include leading wildcard characters

SnapLogic Live: ServiceNow Integration

SnapLogic

Hunting for Evil with the Elastic Stack

Elasticsearch

Paris Spark Meetup - Trifacta - 03_04_2017

Modern Data Stack France

Scaling ML-Based Threat Detection For Production Cyber Attacks

Databricks

Vulnerabilities such as Spectre and Meltdown continue to plague many production servers, based on Intel CPUs. Our solution involves software-based monitoring of hardware counters and sending that data to Apache Spark clusters for threat detection. We leverage Spark's support for support vector machine (SVM) inference. Our machine learning models are trained off-line by a data scientist within a Jupyter notebook environment. As new models are validated, they can be easily deployed to the Spark cluster from the notebook. We have standardized model export and import using the ONNX machine learning open file format. In our presentation, we will demo the full pipeline, from model training to deployment. We will discuss the various challenges when deploying ML-based cyber-threat detection at scale using Apache Spark. For example, we found that gaps in detection can occur when Spark models are updated. We will describe a novel data ingestion architecture, based on Apache Kafka, that we developed to deal with this issue.

Empower your security practitioners with the Elastic Stack

Elasticsearch

Alexander Pavlenko, Java Software Engineer, DataArt.

Alina Vilk

Combining Logs, Metrics, and Traces for Unified Observability

Elasticsearch

Elastic@Colruyt: Ensuring business continuity and improving efficiency

Elasticsearch

Discover how Colruyt Group, a supermarket retailer that represents 40+ brands (Spar, OKay, Bio-Planet NV, DeamLand, Collishop), ensures business continuity for more than 30,000 employees by using the Elastic Stack and Elastic Observability to monitor events and logs coming from core infrastructure and application components used by internal and external end users. With Elastic, the group reduces cost and increases productivity of the IT team, allowing them to provide excellent customer service both inside and outside the company.

xGem Data Stream Processing

Jorge Hirtz

Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...

Spark Summit

HP ships millions of PCs, Printers, and other devices every year to customers in all market segments. More customers are seeking services provided with our products enabling new opportunities for HP to create services from the data we can collect from our devices. Every device we ship is an IoT endpoint with powerful CPU to capture rich data. Insights from this data are used internally to improve our products and focus on customer needs. In this presentation, John will focus on HP’s journey to enabling Big Data analytics from within a large enterprise environment. He will review the challenges and how HP decided on AWS, Apache Spark and Databricks as the foundation for their entry into Big Data Analytics. John will also review how HP uses Spark to build analytic services from the data they generate from their devices.

Palestra de abertura: Evolução e visão do Elastic Observability

Elasticsearch

_Search? Made Simple: Elastic + App Search

Elasticsearch

New Relic Plugin for Hadoop | Blue Medora

Blue Medora

Load data from xml to Snowflake in minutes

syed_javed

Meetup introduction to elastic stack - search at scale - skilledfield slide...

Mouaz Alnouri

Now, without a doubt, given exponential data growth, there’s increased demand for relevant, real-time insights. We all know this. Elastic Stack is one of the best solutions with powerful search capabilities. It provides the opportunities to centralise data and consequently make actionable decisions. In this meetup we will walk through Elastic Stack, to help you better understand how to make the best use of the solution. This meetup will cover: -What is Elastic Stack and what problems it can solve -How can data analysts/scientists benefit from Elastic Stack -A demo

Science base usage analysis - AGU2016 - in21d08

Sky Bristol

ScienceBase is a research infrastructure developed and operated by the U.S. Geological Survey with users and uses across a number of other agency and organization partners. Over four years ago, we released an Application Programming Interface (API) as the foundation of the system and took on the mindset that our progress would be measured by the uptake of the API by others beyond ourselves in developing interesting applications. We now measure success more by someone finding ScienceBase, organizing their data and information, developing an innovative API-driven application and then serendipitous discovery through a science meeting. Because of the way we built the RESTful API, we can characterize what parts of the system are employed. Analysis of usage data helps us take the supposition out of what works and guides design and funding decisions. This analytics-based process facilitates regular adjustments to our thinking and allows us to test design decisions as hypotheses rather than untestable aspirations.

Countering Threats with the Elastic Stack at CERDEC/ARL

Elasticsearch

Elevating Tactical DDD Patterns Through Object Calisthenics

Dorra BARTAGUIZ

After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!

Pushing the limits of ePRTC: 100ns holdover for 100 days

Adtran

What's hot

SnapLogic Live: Workday Integration

SnapLogic

The Need for NoSQL - MarkLogic

GovLoop

Expanding Elastic: Learn how anyone can leverage heterogeneous compute to ext...

Ryft

SnapLogic Live: ServiceNow Integration

SnapLogic

Hunting for Evil with the Elastic Stack

Elasticsearch

Paris Spark Meetup - Trifacta - 03_04_2017

Modern Data Stack France

Scaling ML-Based Threat Detection For Production Cyber Attacks

Databricks

Empower your security practitioners with the Elastic Stack

Elasticsearch

Alexander Pavlenko, Java Software Engineer, DataArt.

Alina Vilk

Combining Logs, Metrics, and Traces for Unified Observability

Elasticsearch

Elastic@Colruyt: Ensuring business continuity and improving efficiency

Elasticsearch

xGem Data Stream Processing

Jorge Hirtz

Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...

Spark Summit

Palestra de abertura: Evolução e visão do Elastic Observability

Elasticsearch

_Search? Made Simple: Elastic + App Search

Elasticsearch

New Relic Plugin for Hadoop | Blue Medora

Blue Medora

Load data from xml to Snowflake in minutes

syed_javed

Meetup introduction to elastic stack - search at scale - skilledfield slide...

Mouaz Alnouri

Science base usage analysis - AGU2016 - in21d08

Sky Bristol

Countering Threats with the Elastic Stack at CERDEC/ARL

Elasticsearch

What's hot (20)

SnapLogic Live: Workday Integration

The Need for NoSQL - MarkLogic

Expanding Elastic: Learn how anyone can leverage heterogeneous compute to ext...

SnapLogic Live: ServiceNow Integration

Hunting for Evil with the Elastic Stack

Paris Spark Meetup - Trifacta - 03_04_2017

Scaling ML-Based Threat Detection For Production Cyber Attacks

Empower your security practitioners with the Elastic Stack

Alexander Pavlenko, Java Software Engineer, DataArt.

Combining Logs, Metrics, and Traces for Unified Observability

Elastic@Colruyt: Ensuring business continuity and improving efficiency

xGem Data Stream Processing

Unlocking Value in Device Data Using Spark: Spark Summit East talk by John La...

Palestra de abertura: Evolução e visão do Elastic Observability

_Search? Made Simple: Elastic + App Search

New Relic Plugin for Hadoop | Blue Medora

Load data from xml to Snowflake in minutes

Meetup introduction to elastic stack - search at scale - skilledfield slide...

Science base usage analysis - AGU2016 - in21d08

Countering Threats with the Elastic Stack at CERDEC/ARL

Recently uploaded

Elevating Tactical DDD Patterns Through Object Calisthenics

Dorra BARTAGUIZ

Pushing the limits of ePRTC: 100ns holdover for 100 days

Adtran

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Removing Uninteresting Bytes in Software Fuzzing

Aftab Hussain

Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process. In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds. - These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

Peter Spielvogel

Building better applications for business users with SAP Fiori. • What is SAP Fiori and why it matters to you • How a better user experience drives measurable business benefits • How to get started with SAP Fiori today • How SAP Fiori elements accelerates application development • How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities • How SAP Fiori paves the way for using AI in SAP apps

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

Enhancing Performance with Globus and the Science DMZ

Globus

Assure Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

Free Complete Python - A step towards Data Science

RinaMondal9

Quantum Computing: Current Landscape and the Future Role of APIs

Vlad Stirbu

Video Streaming: Then, Now, and in the Future

Alpen-Adria-Universität

In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.

Accelerate your Kubernetes clusters with Varnish Caching

Thijs Feryn

The Future of Platform Engineering

Jemma Hussein Allen

By Design, not by Accident - Agile Venture Bolzano 2024

Pierluigi Pugliese

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Prayukth K V

The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development. The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers: State of global ICS asset and network exposure Sectoral targets and attacks as well as the cost of ransom Global APT activity, AI usage, actor and tactic profiles, and implications Rise in volumes of AI-powered cyberattacks Major cyber events in 2024 Malware and malicious payload trends Cyberattack types and targets Vulnerability exploit attempts on CVEs Attacks on counties – USA Expansion of bot farms – how, where, and why In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East Why are attacks on smart factories rising? Cyber risk predictions Axis of attacks – Europe Systemic attacks in the Middle East Download the full report from here: https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

Alex Pruden

This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second). Paper: https://eprint.iacr.org/2023/1886

PHP Frameworks: I want to break free (IPC Berlin 2024)

Ralf Eggert

In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development. This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

Recently uploaded (20)

Elevating Tactical DDD Patterns Through Object Calisthenics

Pushing the limits of ePRTC: 100ns holdover for 100 days

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Removing Uninteresting Bytes in Software Fuzzing

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

Securing your Kubernetes cluster_ a step-by-step guide to success !

Enhancing Performance with Globus and the Science DMZ

Assure Contact Center Experiences for Your Customers With ThousandEyes

GraphRAG is All You need? LLM & Knowledge Graph

Free Complete Python - A step towards Data Science

Quantum Computing: Current Landscape and the Future Role of APIs

Video Streaming: Then, Now, and in the Future

Accelerate your Kubernetes clusters with Varnish Caching

The Future of Platform Engineering

By Design, not by Accident - Agile Venture Bolzano 2024

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

PHP Frameworks: I want to break free (IPC Berlin 2024)

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...