Collecting Endpoint Security Logs Through Big Data Technology - Dedi Dwianto
•
1 like•375 views
Big data technology can collect endpoint security logs from various data sources like clickstream data, shopping cart data, and sensor data. These logs record events in operating systems and software and have characteristics of big data. Open source tools like Logstash and Elasticsearch can be used to ingest, process, and analyze these logs. Logstash is a server-side data processing pipeline that takes in data from multiple sources, while Elasticsearch is a distributed search and analytics engine. These tools can be incorporated into a big data event log architecture using other Hadoop components like Apache Flume, YARN, HDFS, and analytics frameworks.
Report
Share
Report
Share
Download to read offline
Recommended
Elk - An introduction
So, what is the ELK Stack? "ELK" is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a "stash" like Elasticsearch. Kibana lets users visualize data with charts and graphs in Elasticsearch.
Piyali Kamra - Analytics and Data Visualization pipeline backed by AWS Glue &...
AWS Community Day Midwest 2020
Piyali Kamra - Analytics and Data Visualization pipeline backed by AWS Glue & Athena
Ronan Corkery, kdb+ developer at Kx Systems: “Kdb+: How Wall Street Tech can ...
Ronan Corkery, kdb+ developer at Kx Systems: “Kdb+: How Wall Street Tech can Speed up the World"
Bio:
Ronan Corkery is a kdb+ engineer who has been working with Kx and First Derivatives for the past 4 years. Currently based in Total Gas and Power he spent his first 2 year working with Morgan Stanley.
Abstract:
Ronan's presentation will focus on the vertical industries the formally only finance based technologies Kx offers has been moving into. He will present proven solutions as well as introducing the overall architecture that Kx uses as well as laying out potential opportunities to work with Kx.
Hasura 2.0 Webinar
Hasura 2.0 is our biggest release since we launched.
This webinar goes over the new capabilities in our v2 release:
- Connect to multiple databases
- Generate REST APIs
- Enhanced Authorisation
- Hasura Cloud & AWS VPC Peering
SC5 Hangout2 pilot 1 description
This document discusses using the BDE (Big Data Europe) platform to demonstrate downscaling of climatic and meteorological data, data ingestion and export, analytics capabilities, and data lineage tracking. The SC5 Pilot project ingests NetCDF files, performs WRF-based downscaling on institutional resources as requested, and maintains lineage records. Basic analytics like climate change indices are performed using Hive queries. Further development may investigate NetCDF transformations for WRF and test data lineage and downscaling customization. The BDE is expected to provide scalability, efficient resource use, and data reproducibility. Hands-on access to a Jupyter notebook is provided.
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
( ELK Stack Training - https://www.edureka.co/elk-stack-trai... )
This Edureka tutorial on What Is ELK Stack will help you in understanding the fundamentals of Elasticsearch, Logstash, and Kibana together and help you in building a strong foundation in ELK Stack. Below are the topics covered in this ELK tutorial for beginners:
1. Need for Log Analysis
2. Problems with Log Analysis
3. What is ELK Stack?
4. Features of ELK Stack
5. Companies Using ELK Stack
Building stateful apps using serverless
The document discusses building stateful applications using serverless architecture and provides 5 patterns for delegating state when using serverless: 1) Send full context in event payload, 2) Use backend as a service (BaaS) or managed services, 3) Run lightweight long-running proxies, 4) Apply state externally, and 5) Use fast transactional databases. It also provides examples for each pattern and discusses what constitutes a stateless application from a serverless perspective.
The IoT and big data
In the next five years, 15 to 40 billion additional connected devices are expected to hit the market. How can we handle such volumes and velocity of data?
Introduction to Dynamo storage systems, Riak, Cassandra, time series databases and edge analytics.
Recommended
Elk - An introduction
So, what is the ELK Stack? "ELK" is the acronym for three open source projects: Elasticsearch, Logstash, and Kibana. Elasticsearch is a search and analytics engine. Logstash is a server‑side data processing pipeline that ingests data from multiple sources simultaneously, transforms it, and then sends it to a "stash" like Elasticsearch. Kibana lets users visualize data with charts and graphs in Elasticsearch.
Piyali Kamra - Analytics and Data Visualization pipeline backed by AWS Glue &...
AWS Community Day Midwest 2020
Piyali Kamra - Analytics and Data Visualization pipeline backed by AWS Glue & Athena
Ronan Corkery, kdb+ developer at Kx Systems: “Kdb+: How Wall Street Tech can ...
Ronan Corkery, kdb+ developer at Kx Systems: “Kdb+: How Wall Street Tech can Speed up the World"
Bio:
Ronan Corkery is a kdb+ engineer who has been working with Kx and First Derivatives for the past 4 years. Currently based in Total Gas and Power he spent his first 2 year working with Morgan Stanley.
Abstract:
Ronan's presentation will focus on the vertical industries the formally only finance based technologies Kx offers has been moving into. He will present proven solutions as well as introducing the overall architecture that Kx uses as well as laying out potential opportunities to work with Kx.
Hasura 2.0 Webinar
Hasura 2.0 is our biggest release since we launched.
This webinar goes over the new capabilities in our v2 release:
- Connect to multiple databases
- Generate REST APIs
- Enhanced Authorisation
- Hasura Cloud & AWS VPC Peering
SC5 Hangout2 pilot 1 description
This document discusses using the BDE (Big Data Europe) platform to demonstrate downscaling of climatic and meteorological data, data ingestion and export, analytics capabilities, and data lineage tracking. The SC5 Pilot project ingests NetCDF files, performs WRF-based downscaling on institutional resources as requested, and maintains lineage records. Basic analytics like climate change indices are performed using Hive queries. Further development may investigate NetCDF transformations for WRF and test data lineage and downscaling customization. The BDE is expected to provide scalability, efficient resource use, and data reproducibility. Hands-on access to a Jupyter notebook is provided.
What Is ELK Stack | ELK Tutorial For Beginners | Elasticsearch Kibana | ELK S...
( ELK Stack Training - https://www.edureka.co/elk-stack-trai... )
This Edureka tutorial on What Is ELK Stack will help you in understanding the fundamentals of Elasticsearch, Logstash, and Kibana together and help you in building a strong foundation in ELK Stack. Below are the topics covered in this ELK tutorial for beginners:
1. Need for Log Analysis
2. Problems with Log Analysis
3. What is ELK Stack?
4. Features of ELK Stack
5. Companies Using ELK Stack
Building stateful apps using serverless
The document discusses building stateful applications using serverless architecture and provides 5 patterns for delegating state when using serverless: 1) Send full context in event payload, 2) Use backend as a service (BaaS) or managed services, 3) Run lightweight long-running proxies, 4) Apply state externally, and 5) Use fast transactional databases. It also provides examples for each pattern and discusses what constitutes a stateless application from a serverless perspective.
The IoT and big data
In the next five years, 15 to 40 billion additional connected devices are expected to hit the market. How can we handle such volumes and velocity of data?
Introduction to Dynamo storage systems, Riak, Cassandra, time series databases and edge analytics.
Kx brian
Kx is a time-series database company that provides the world's fastest time-series database called kdb+. Kdb+ is an in-memory column-oriented database with real-time and historical analytics capabilities. It can process over 30 million sensor readings per second and search tables of 4 billion records per second per core. Kx is used by the world's largest investment banks and companies across industries including healthcare, automotive, manufacturing and more for applications like high-frequency trading, sensor data analytics, and IoT.
An Intro to Elasticsearch and Kibana
Visualize some of Austin's open source data using Elasticsearch with Kibana. ObjectRocket's Steve Croce presented this talk on 10/13/17 at the DBaaS event in Austin, TX.
Introduction to ELK
This slide deck provides an overview of ElasticSearch, LogStash and Kibana stack. https://goo.gl/oZIrwS
SC7 Webinar 5 13/12/2017 UoA Presentation "Technical aspects of the 3rd secur...
This document summarizes the technical aspects of the third Secure Societies pilot project. The pilot uses multiple data sources including satellite imagery, news articles, and Twitter posts. It has two main workflows: a change detection workflow that analyzes satellite images to detect changes over time using image processing techniques, and an event detection workflow that monitors social media and news to cluster events. Both workflows store results as linked geospatial data for visualization and querying through an interface. The change detector component parallelizes change detection algorithms on images using Spark.
Spark for bay area big data developer
Global Big Data Conference is offering 3 day extensive Developer conference on Big Data. This is a fast paced, vendor agnostic, technical overview of the Big Data landscape.
Log analysis using elk
Log Management
Log Monitoring
Log Analysis
Need for Log Analysis
Problem with Log Analysis
Some of Log Management Tool
What is ELK Stack
ELK Stack Working
Beats
Different Types of Server Logs
Example of Winlog beat, Packetbeat, Apache2 and Nginx Server log analysis
Mimikatz
Malicious File Detection using ELK
Practical Setup
Conclusion
Building real apps on serverless
This document discusses building stateful applications on serverless platforms and provides several patterns for managing state in a serverless environment. The key patterns discussed are: 1) Sending the full context in event payloads to avoid database reads, 2) Using backend-as-a-service providers to manage stateful services like identity, media, and notifications, 3) Running long-running "state planes" to manage state, 4) Building workflows using state machine patterns, and 5) Using fast databases to directly manage state without delegating it. The document emphasizes that real applications always have state and the challenge is effectively delegating state when building on serverless platforms. It also discusses related topics like ensuring idempotency and implementing patterns
Tech Thursday - Beer & DevOps 24.11.
This document outlines an agenda for a DevOps event on auto-scaling. The agenda includes welcome remarks, presentations on auto-scaling use cases and a Kontena demo. There is also a scheduled auto-scale demo showing how resource groups can be provisioned and scaled up/down automatically based on metrics and policies in OpenStack Heat. The document argues that auto-scaling can be simplified using standardized metrics and workloads that scale out horizontally rather than vertically. It notes auto-scaling can be implemented in a few hours and at very low cost using services like Datadog to monitor metrics and trigger scaling policies.
Exploring MongoDB & Elasticsearch: Better Together
An Open Talk at DeveloperWeek Austin 2017 by Kimberly Wilkins (@dba_denizen), Principal Engineer - Databases at ObjectRocket. Featuring new use cases like Bitcoin, AI, IoT, and all the cool things.
Hadoop for bay area big data developer
Global Big Data Conference is offering 3 day extensive Developer conference on Big Data. This is a fast paced, vendor agnostic, technical overview of the Big Data landscape. No prior knowledge of databases or programming is assumed.
Cassandra for bay area big data developer
Global Big Data Conference is offering 3 day extensive Developer conference on Big Data. This is a fast paced, vendor agnostic, technical overview of the Big Data landscape. No prior knowledge of databases or programming is assumed.
Hunting for Evil with the Elastic Stack
Whether you are threat hunting or responding to a signature-based alert, learn how to use Elastic tools to tell the entire story and more efficiently root out adversaries in your environment.
See the video: https://www.elastic.co/elasticon/tour/2019/washington-dc/hunting-for-evil-with-the-elastic-stack
Big data for bay area big data developer
The document announces a three-day Big Data Developer Conference to take place from March 23-25 at the Hilton in Santa Clara. The conference will feature extensive sessions on big data tools and technologies like Hadoop, HDFS, MapReduce, NOSQL, Cassandra, HBase, Spark, and analytics. It is aimed at engineers, developers, managers, and others working with big data. The agenda outlines the daily tracks and sessions covering topics such as introductions to big data systems, workshops, use cases, security, and trends. Registration details are provided.
Scalable Data Analytics and Visualization with Cloud Optimized Services
This document discusses scalable data analytics and visualization using cloud optimized services. It provides examples of leveraging cloud technologies like AWS and Azure to build scalable implementations for processing and serving large geospatial and earth observation datasets. This includes architectures for hosting high resolution raster data and services, global hydrologic modeling, and near real-time flood forecasting using NOAA's National Water Model.
The Elastic ELK Stack
The ELK stack is an open source toolset for data analysis that includes Logstash, Elasticsearch, and Kibana. Logstash collects and parses data from various sources, Elasticsearch stores and indexes the data for fast searching and analytics, and Kibana visualizes the data. The ELK stack can handle large volumes of time-series data in real-time and provides actionable insights. Commercial plugins are also available for additional functionality like monitoring, security, and support.
Curse of Cardinality: A History and Evolution of Monitoring at Scale
This document summarizes the evolution of monitoring tools from the 1990s to present day. It discusses early tools like MRTG and RRDtool that helped prove the need for larger network links. Nagios emerged in the early 2000s and became popular but also led to expensive commercial clones. Graphite and StatsD provided great graphing in the late 2000s. Splunk's full-text search and interface grew in popularity. OpenTSDB aimed to be highly scalable but could cause outages. Prometheus emerged in 2015 and was adopted by CNCF, using a pull model and dimensional data. The document concludes that observability is the next focus area, with improved metrics, distributed tracing, and better use of logs.
Managing Large Scale Financial Time-Series Data with Graphs
Slides from a recent webinar by Objectivity showing how the ThingSpan platform is ideal for graph analytics to uncover patterns and insights within large, complex data sets in order to make efficient decisions.
PoolParty Search Server
PoolParty Semantic Search Server is described technologically. How to use SKOS thesauri to map data from different sources and how to generate a semantic index. How to build precise faceted search.
Suricate
Suricate is an open data science platform that allows users to:
1) Upload or stream data into the platform for processing and analytics.
2) Data scientists can then perform analytics using Python or other tools and visualize results.
3) The platform supports processing the results of analytics to trigger actions, thereby enabling the creation of continuously adjusted agile systems and strategies based on up-to-date insights.
London Borough of Harrow - Deliver GI with speed and efficiency
At Esri UK Annual Conference 2014
At Harrow Council GI focused projects are being more of the norm. In this example there were two departments trying to create a spatial database of council land ownership in Harrow, but problems were forecast. Which one would be the definitive layer? The two data sources would benefit each other so the projects were combined so that just one dataset was created. The problem was that users wanted to be able to have a record of land ownership and associate a large number of attributes with it. All this without the paper files and manual searching of the current process.
A GIS based solution was proposed, utilising the Flex API intranet platform previously custom built by Harrow. This allowed quick development time and simple integration. The overall outcome of the deeds project was the smooth transition from a laborious paper based system with no mapping to an intuitive, simple to use mapping system that answered questions in significantly quicker times, gave multiple users access and brought together two datasets into one.
Xavient - DiP
Data Ingestion Platform - DiP
Check out the real time data ingestion using Data Ingestion Platform (DiP) which harness the powers of Apache Apex, Apache Flink, Apache Spark and Apache Storm to give real time data ingestion and visualization.
DiP comes along with a UI which allows to switch between multiple data streaming engines and combines them under one single platform.
Log management with_logstash_and_elastic_search
This document proposes a log management solution using Logstash, Elasticsearch, and Kibana. Logstash is used to collect, parse, and index logs into Elasticsearch for centralized storage and real-time search. Kibana provides visualization and analytics dashboards. The solution offers scalability, reliability, searchability, and a low-cost and flexible open source approach to solving the challenges of gathering, analyzing, and gaining insights from large volumes of log data from diverse sources.
More Related Content
What's hot
Kx brian
Kx is a time-series database company that provides the world's fastest time-series database called kdb+. Kdb+ is an in-memory column-oriented database with real-time and historical analytics capabilities. It can process over 30 million sensor readings per second and search tables of 4 billion records per second per core. Kx is used by the world's largest investment banks and companies across industries including healthcare, automotive, manufacturing and more for applications like high-frequency trading, sensor data analytics, and IoT.
An Intro to Elasticsearch and Kibana
Visualize some of Austin's open source data using Elasticsearch with Kibana. ObjectRocket's Steve Croce presented this talk on 10/13/17 at the DBaaS event in Austin, TX.
Introduction to ELK
This slide deck provides an overview of ElasticSearch, LogStash and Kibana stack. https://goo.gl/oZIrwS
SC7 Webinar 5 13/12/2017 UoA Presentation "Technical aspects of the 3rd secur...
This document summarizes the technical aspects of the third Secure Societies pilot project. The pilot uses multiple data sources including satellite imagery, news articles, and Twitter posts. It has two main workflows: a change detection workflow that analyzes satellite images to detect changes over time using image processing techniques, and an event detection workflow that monitors social media and news to cluster events. Both workflows store results as linked geospatial data for visualization and querying through an interface. The change detector component parallelizes change detection algorithms on images using Spark.
Spark for bay area big data developer
Global Big Data Conference is offering 3 day extensive Developer conference on Big Data. This is a fast paced, vendor agnostic, technical overview of the Big Data landscape.
Log analysis using elk
Log Management
Log Monitoring
Log Analysis
Need for Log Analysis
Problem with Log Analysis
Some of Log Management Tool
What is ELK Stack
ELK Stack Working
Beats
Different Types of Server Logs
Example of Winlog beat, Packetbeat, Apache2 and Nginx Server log analysis
Mimikatz
Malicious File Detection using ELK
Practical Setup
Conclusion
Building real apps on serverless
This document discusses building stateful applications on serverless platforms and provides several patterns for managing state in a serverless environment. The key patterns discussed are: 1) Sending the full context in event payloads to avoid database reads, 2) Using backend-as-a-service providers to manage stateful services like identity, media, and notifications, 3) Running long-running "state planes" to manage state, 4) Building workflows using state machine patterns, and 5) Using fast databases to directly manage state without delegating it. The document emphasizes that real applications always have state and the challenge is effectively delegating state when building on serverless platforms. It also discusses related topics like ensuring idempotency and implementing patterns
Tech Thursday - Beer & DevOps 24.11.
This document outlines an agenda for a DevOps event on auto-scaling. The agenda includes welcome remarks, presentations on auto-scaling use cases and a Kontena demo. There is also a scheduled auto-scale demo showing how resource groups can be provisioned and scaled up/down automatically based on metrics and policies in OpenStack Heat. The document argues that auto-scaling can be simplified using standardized metrics and workloads that scale out horizontally rather than vertically. It notes auto-scaling can be implemented in a few hours and at very low cost using services like Datadog to monitor metrics and trigger scaling policies.
Exploring MongoDB & Elasticsearch: Better Together
An Open Talk at DeveloperWeek Austin 2017 by Kimberly Wilkins (@dba_denizen), Principal Engineer - Databases at ObjectRocket. Featuring new use cases like Bitcoin, AI, IoT, and all the cool things.
Hadoop for bay area big data developer
Global Big Data Conference is offering 3 day extensive Developer conference on Big Data. This is a fast paced, vendor agnostic, technical overview of the Big Data landscape. No prior knowledge of databases or programming is assumed.
Cassandra for bay area big data developer
Global Big Data Conference is offering 3 day extensive Developer conference on Big Data. This is a fast paced, vendor agnostic, technical overview of the Big Data landscape. No prior knowledge of databases or programming is assumed.
Hunting for Evil with the Elastic Stack
Whether you are threat hunting or responding to a signature-based alert, learn how to use Elastic tools to tell the entire story and more efficiently root out adversaries in your environment.
See the video: https://www.elastic.co/elasticon/tour/2019/washington-dc/hunting-for-evil-with-the-elastic-stack
Big data for bay area big data developer
The document announces a three-day Big Data Developer Conference to take place from March 23-25 at the Hilton in Santa Clara. The conference will feature extensive sessions on big data tools and technologies like Hadoop, HDFS, MapReduce, NOSQL, Cassandra, HBase, Spark, and analytics. It is aimed at engineers, developers, managers, and others working with big data. The agenda outlines the daily tracks and sessions covering topics such as introductions to big data systems, workshops, use cases, security, and trends. Registration details are provided.
Scalable Data Analytics and Visualization with Cloud Optimized Services
This document discusses scalable data analytics and visualization using cloud optimized services. It provides examples of leveraging cloud technologies like AWS and Azure to build scalable implementations for processing and serving large geospatial and earth observation datasets. This includes architectures for hosting high resolution raster data and services, global hydrologic modeling, and near real-time flood forecasting using NOAA's National Water Model.
The Elastic ELK Stack
The ELK stack is an open source toolset for data analysis that includes Logstash, Elasticsearch, and Kibana. Logstash collects and parses data from various sources, Elasticsearch stores and indexes the data for fast searching and analytics, and Kibana visualizes the data. The ELK stack can handle large volumes of time-series data in real-time and provides actionable insights. Commercial plugins are also available for additional functionality like monitoring, security, and support.
Curse of Cardinality: A History and Evolution of Monitoring at Scale
This document summarizes the evolution of monitoring tools from the 1990s to present day. It discusses early tools like MRTG and RRDtool that helped prove the need for larger network links. Nagios emerged in the early 2000s and became popular but also led to expensive commercial clones. Graphite and StatsD provided great graphing in the late 2000s. Splunk's full-text search and interface grew in popularity. OpenTSDB aimed to be highly scalable but could cause outages. Prometheus emerged in 2015 and was adopted by CNCF, using a pull model and dimensional data. The document concludes that observability is the next focus area, with improved metrics, distributed tracing, and better use of logs.
Managing Large Scale Financial Time-Series Data with Graphs
Slides from a recent webinar by Objectivity showing how the ThingSpan platform is ideal for graph analytics to uncover patterns and insights within large, complex data sets in order to make efficient decisions.
PoolParty Search Server
PoolParty Semantic Search Server is described technologically. How to use SKOS thesauri to map data from different sources and how to generate a semantic index. How to build precise faceted search.
Suricate
Suricate is an open data science platform that allows users to:
1) Upload or stream data into the platform for processing and analytics.
2) Data scientists can then perform analytics using Python or other tools and visualize results.
3) The platform supports processing the results of analytics to trigger actions, thereby enabling the creation of continuously adjusted agile systems and strategies based on up-to-date insights.
London Borough of Harrow - Deliver GI with speed and efficiency
At Esri UK Annual Conference 2014
At Harrow Council GI focused projects are being more of the norm. In this example there were two departments trying to create a spatial database of council land ownership in Harrow, but problems were forecast. Which one would be the definitive layer? The two data sources would benefit each other so the projects were combined so that just one dataset was created. The problem was that users wanted to be able to have a record of land ownership and associate a large number of attributes with it. All this without the paper files and manual searching of the current process.
A GIS based solution was proposed, utilising the Flex API intranet platform previously custom built by Harrow. This allowed quick development time and simple integration. The overall outcome of the deeds project was the smooth transition from a laborious paper based system with no mapping to an intuitive, simple to use mapping system that answered questions in significantly quicker times, gave multiple users access and brought together two datasets into one.
What's hot (20)
SC7 Webinar 5 13/12/2017 UoA Presentation "Technical aspects of the 3rd secur...
SC7 Webinar 5 13/12/2017 UoA Presentation "Technical aspects of the 3rd secur...
Exploring MongoDB & Elasticsearch: Better Together
Exploring MongoDB & Elasticsearch: Better Together
Scalable Data Analytics and Visualization with Cloud Optimized Services
Scalable Data Analytics and Visualization with Cloud Optimized Services
Curse of Cardinality: A History and Evolution of Monitoring at Scale
Curse of Cardinality: A History and Evolution of Monitoring at Scale
Managing Large Scale Financial Time-Series Data with Graphs
Managing Large Scale Financial Time-Series Data with Graphs
London Borough of Harrow - Deliver GI with speed and efficiency
London Borough of Harrow - Deliver GI with speed and efficiency
Similar to Collecting Endpoint Security Logs Through Big Data Technology - Dedi Dwianto
Xavient - DiP
Data Ingestion Platform - DiP
Check out the real time data ingestion using Data Ingestion Platform (DiP) which harness the powers of Apache Apex, Apache Flink, Apache Spark and Apache Storm to give real time data ingestion and visualization.
DiP comes along with a UI which allows to switch between multiple data streaming engines and combines them under one single platform.
Log management with_logstash_and_elastic_search
This document proposes a log management solution using Logstash, Elasticsearch, and Kibana. Logstash is used to collect, parse, and index logs into Elasticsearch for centralized storage and real-time search. Kibana provides visualization and analytics dashboards. The solution offers scalability, reliability, searchability, and a low-cost and flexible open source approach to solving the challenges of gathering, analyzing, and gaining insights from large volumes of log data from diverse sources.
Airline traffic management analysis
• The objective is airline traffic management analysis is to maintain and analyze it based on various factors
• Tools used: Horton Works Sandbox, Tableau
Streaming analytics
Stream computing is a processing paradigm that brings the analytics to the data, rather than storing the data first.
The hidden engineering behind machine learning products at Helixa
Data Orchestration Summit 2020 organized by Alluxio
https://www.alluxio.io/data-orchestration-summit-2020/
The hidden engineering behind machine learning products at Helixa
Gianmario Spacagna, (Helixa)
About Alluxio: alluxio.io
Engage with the open source community on slack: alluxio.io/slack
AWS Webcast - Amazon Kinesis and Apache Storm
Join us for a for a Amazon Kinesis tutorial webinar. In this session we will provide a reference architecture and instructions for building a system that performs real-time sliding-windows analysis over streaming clickstream data. We will use Amazon Kinesis for managed ingestion of streaming data at scale with the ability to replay past data, and run sliding-window computation using Apache Storm. We’ll demonstrate in the webinar on how to build the system and deploy on AWS and walkthrough all the steps from ingestion, processing, and storing to visualizing of the data in real-time.
ExtraHop Product Overview Datasheet
The ExtraHop platform enables organizations to perform multidimensional analysis and visualizations on their streaming wire data.
TB8568_8568_Presentation
HPE Operations Analytics provides a scalable platform for analyzing large volumes of IT operational data from multiple sources to improve root cause analysis, anomaly detection, and predictive analytics capabilities. It helps reduce mean time to repair issues by automatically correlating disparate metrics and logs to identify relationships and root causes. HPE IT uses Operations Analytics to gain insights from its operational data across many domains including servers, applications, databases, networks and private clouds.
Elasticsearch features and ecosystem
Around elasticsearch ecosystem - some components and all work together. With some real dashboard examples from ELK of lesegais.ru/portal
Putting Hadoop To Work In The Enterprise
The document discusses how Hadoop can be used in enterprises by providing untethered access to large amounts of raw data for data scientists. It outlines different user needs that Hadoop can address, including data artists, business analysts, and business consumers. The document also presents Teradata's approach to helping Hadoop succeed in enterprises by providing products, services, and support around big data analytics, data integration, and simplifying Hadoop use.
xGem Data Stream Processing
This document provides an overview of data stream processing. It discusses what streaming data is and examples like IoT sensors, social media, and website monitoring. It also outlines the typical components of a streaming data pipeline including collecting and ingesting data from various sources, processing the data in real-time, storing the processed data, and serving it for analytics, search, and dashboards. Key streaming technologies mentioned include Apache Kafka, Apache NiFi, and various stream processing frameworks. It also introduces Stishovite as a console for managing an entire streaming data platform built from open source components.
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
The document discusses using a data lake approach with EMC Isilon storage to address various business use cases. It describes how the solution provides shared storage for multiple workloads through multi-protocol support, enables data protection and isolation of client data, and allows testing applications across Hadoop distributions through a common platform. Examples are given of how this approach supports an enterprise data hub, data warehouse offloading, data integration, and enrichment services.
Stream Processing – Concepts and Frameworks
More and more data sources today provide a constant stream of data, from IoT devices to Social Media streams. It is one thing to collect these events in the velocity they arrive, without losing any single message. An Event Hub and a data flow engine can help here. It’s another thing to do some (complex) analytics on the data. There is always the option to first store in a data sink of choice and later analyze it. Storing even a high-volume event stream is feasible and not a challenge anymore. But this adds to the end-to-end latency and it takes minutes if not hours to present results. If you need to react fast, you simply can’t afford to first store the data. You need to do process it directly on the data stream. This is called Stream Processing or Stream Analytics. In this talk I will present the important concepts, a Stream Processing solution should support and then dive into some of the most popular frameworks available on the market and how they compare.
Dataservices: Processing (Big) Data the Microservice Way
Apache Big Data 2017, Miami (Florida/USA): Talk by Josef Adersberger (@adersberger, CTO at QAware)
Abstract:
We see a big data processing pattern emerging using the Microservice approach to build an integrated, flexible, and distributed system of data processing tasks. We call this the Dataservice pattern. In this presentation we'll introduce into Dataservices: their basic concepts, the technology typically in use (like Kubernetes, Kafka, Cassandra and Spring) and some architectures from real-life.
Real-Time Analytics with Confluent and MemSQL
This document discusses enabling real-time analytics for IoT applications. It describes how industries like auto, transportation, energy, warehousing and logistics, and healthcare need real-time analytics to handle streaming data from IoT sensors. It also discusses how Confluent's Kafka stream processing platform can be used to build applications that ingest IoT data at high speeds, transform the data, and power real-time analytics and user interfaces. MemSQL's in-memory database is presented as a fast and scalable storage option to support real-time analytics on the large volumes of IoT data.
Big Data Modeling Challenges and Machine Learning with No Code
Presented at SF BAY ACM_202001015_by_Karthik Chinnusamy
What are the Big Data model challenges in today's field? With a few best practice recommendations and Machine Learning approaches, I will use Knime to show the modeling advantages for Big Data with the following themes:
.Performance: Good data models can help us quickly query the required data and reduce I/O throughput.
.Cost: Good data models can significantly reduce unnecessary data redundancy, reuse computing results, and reduce the storage and computing costs for the big data system.
.Efficiency: Good data models can greatly improve user experience and increase the efficiency of data utilization.
.Quality: Good data models make data statistics more consistent and reduce the possibility of computing errors.
I will also describe tools for Sources, Ingestion, Exploration, Modeling and Machine Learning.
Azure Data Explorer deep dive - review 04.2020
Modern Data Science Lifecycle with ADX & Azure
This document discusses using Azure Data Explorer (ADX) for data science workflows. ADX is a fully managed analytics service for real-time analysis of streaming data. It allows for ad-hoc querying of data using Kusto Query Language (KQL) and integrates with various Azure data ingestion sources. The document provides an overview of the ADX architecture and compares it to other time series databases. It also covers best practices for ingesting data, visualizing results, and automating workflows using tools like Azure Data Factory.
Turn data into business with OPENDATASOFT
OpenDataSoft is a turnkey full-featured platform designed for data valorisation :
- data-driven innovation
- business development
- communication
- business monitoring
On a single platform, OpenDataSoft customers can turn data into business through
- open data
- API for dev / monetization / SEO
- datavisualisations
Elasticsearch + Cascading for Scalable Log Processing
Supreet Oberoi's presentation on "Large scale log processing with Cascading & Elastic Search". Elasticsearch is becoming a popular platform for log analysis with its ELK stack: Elasticsearch for search, Logstash for centralized logging, and Kibana for visualization. Complemented with Cascading, the application development platform for building Data applications on Apache Hadoop, developers can correlate at scale multiple log and data streams to perform rich and complex log processing before making it available to the ELK stack.
Using Elastic to Monitor Everything - Christoph Wurm, Elastic - DevOpsDays Te...
"Elasticsearch has come a long way: Started as a distributed search engine in 2009, it's now the tool of choice for even the largest websites (e.g. Facebook, Github, Ebay). Half-way to 2016 the ELK stack helped it become firmly embedded in many centralised log management systems (e.g. Netflix, Uber).
We're now midair in the next step, with the first folks using it for metrics. NASA is using it to monitor the Curiosity rover, Blizzard and Riot to monitor vast online gaming worlds.
This talk will focus on what makes this transition from more unstructured to structured data possible.
"
Similar to Collecting Endpoint Security Logs Through Big Data Technology - Dedi Dwianto (20)
The hidden engineering behind machine learning products at Helixa
The hidden engineering behind machine learning products at Helixa
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Dataservices: Processing (Big) Data the Microservice Way
Dataservices: Processing (Big) Data the Microservice Way
Big Data Modeling Challenges and Machine Learning with No Code
Big Data Modeling Challenges and Machine Learning with No Code
Elasticsearch + Cascading for Scalable Log Processing
Elasticsearch + Cascading for Scalable Log Processing
Using Elastic to Monitor Everything - Christoph Wurm, Elastic - DevOpsDays Te...
Using Elastic to Monitor Everything - Christoph Wurm, Elastic - DevOpsDays Te...
More from idsecconf
idsecconf2023 - Mochammad Riyan Firmansyah - Takeover Cloud Managed Router vi...
Cloud Managed Router merupakan hasil dari kombinasi antara perangkat router konvensional dengan teknologi cloud management, yang dikembangkan agar memudahkan pengguna untuk dapat mengatur perangkat router dari jarak jauh. Namun tentu saja dengan penerapan yang kurang tepat, maka hal ini bisa dimanfaatkan oleh orang yang tidak bertanggung jawab, bahkan dapat beresiko akses perangkat router diambil alih. Pada topik ini saya akan sedikit menceritakan bagaimana resiko tersebut bisa terjadi.
idsecconf2023 - Neil Armstrong - Leveraging IaC for Stealthy Infrastructure A...
This document provides an agenda and details for a presentation by Neil Armstrong on leveraging infrastructure as code (IaC) for stealthy red team operations. The presentation will cover background on IaC and red team operations, goals and challenges of the project, designs for segmented network, command and control (C2), phishing, and security information and event management (SIEM) infrastructure, implementation and results, and conclusions. The project aims to create reliable, flexible, and stealthy red team infrastructure using IaC to streamline deployments and allow repeatable, disposable operations.
idsecconf2023 - Mangatas Tondang, Wahyu Nuryanto - Penerapan Model Detection ...
Dalam dunia keamanan siber, sinergi antara berbagai proses memiliki peran yang sangat penting. Salah satu proses atau framework yang tengah menjadi sorotan dan menarik perhatian luas adalah Detection Engineering. Proses Detection Engineering ini bertujuan untuk meningkatkan struktur dan pengorganisasian dalam pembuatan detection use case atau rules di Security Operation Center (SOC). Detection Engineering bisa dikatakan masih baru dalam dunia keamanan siber, sehingga terdapat banyak peluang untuk membuat keseluruhan prosesnya menjadi lebih baik. Salah satu hal yang masih terlupakan adalah integrasi antara proses Detection Engineering dan Threat Modeling. Biasanya, Threat Modeling lebih berfokus pada solusi pencegahan dan mitigasi resiko secara langsung dan melupakanan komponen deteksi ketika pencegahan dan mitigasi tersebut gagal dalam menjalankan fungsinya. Dalam makalah ini, kami memperkenalkan paradigma baru dengan mengintegrasikan Detection Engineering ke dalam proses Threat Modeling. Pendekatan ini menjadikan Detection sebagai langkah proaktif tambahan, yang dapat menjadi lapisan pertahanan ekstra ketika kontrol pencegahan dan mitigasi akhirnya gagal dalam menghadapi ancaman sesungguhnya.
idsecconf2023 - Rama Tri Nanda - Hacking Smart Doorbell.pdf
Smart doorbell atau bel pintar telah menjadi populer dalam sistem keamanan rumah pintar. Namun, banyak dari perangkat ini masih menggunakan protokol yang tidak aman untuk berkomunikasi, protokol yang rentan terhadap serangan keamanan seperti jamming, sniffing dan replay attack. Penelitian ini bertujuan untuk menganalisis kelemahan penggunaan protokol komunikasi pada smart doorbell, serta menginvestigasi potensi pemanfaatan Software Defined Radio (SDR) dan modul arduino dalam mengamati komunikasi gelombang elektronik pada frekuensi 433 MHz. Selain itu penelitian ini ditujukan untuk mengidentifikasi potensi risiko yang dihadapi oleh pengguna pengkat IoT, serta memberikan pandangan tentang perlindungan yang lebih baik.
idsecconf2023 - Akshantula Neha, Mohammad Febri Ramadlan - Cyber Harmony Auto...
Modern organizations are facing the severe challenge of effectively countering threats and mitigating Indicators of Compromise (IOCs) within their network environments. The increasing complexity and volume of cyber threats has highlighted the urgency of building robust mechanisms to block specific IOCs independently. While some organizations have adopted Endpoint Detection and Response (EDR) systems, these solutions often have limitations and require manual processes to collect and examine IOCs from multiple sources. These operational barriers prevent organizations from achieving a proactive and efficient defense posture, an obstacle that is particularly important due to the critical role that IOC blocking plays in containing the spread of threats and limiting potential damage. Hence, the need for a solution that orchestrates automated IOC blocking, utilizing tools such as AlienVault Open Threat Exchange (OTX), VirusTotal, CrowdStrike, and Slack. In this presentation, we examine the importance of automated IOC blocking and its potential to strengthen network security, while highlighting the critical role that these tools play in mitigating evolving cyber threats.
idsecconf2023 - Aan Wahyu - Hide n seek with android app protections and beat...
Pembahasan ini bertujuan untuk memberikan edukasi tentang mekanisme perlindungan yang diterapkan pada aplikasi android seperti root detection, ssl pinning, anti emulation, tamper detection dan bagaimana teknik yang digunakan untuk melakukan mekanisme bypass proteksi yang diimplementasikan dengan bantuan reverse engineering menggunakan tool seperti frida, ghidra, objection, magisk, dan sebagainya.
idsecconf2023 - Satria Ady Pradana - Launch into the Stratus-phere Adversary ...
Adversary Simulation pada lingkungan cloud memiliki karakteristik unik sehingga memerlukan pendekatan khusus. Stratus menawarkan fleksibilitas dalam melakukan simulasi attack secara native pada lingkungan cloud. Presentasi ini akan memberikan penjelasan tentang penggunaan Stratus dalam adversary simulation dan bagaimana mengembangkan skenario khusus sesuai kebutuhan.
Ali - The Journey-Hack Electron App Desktop (MacOS).pdf
Semakin berkembangnya teknologi di aplikasi Desktop terdapat celah keamanan yang dapat menyebabkan dampak langsung atau tidak langsung pada kerahasiaan, Integritas Data yang di bangun menggunakan Framework dari Electron khusus nya aplikasi Desktop di Sistem Operasi MAC. Dalam materi yang di persentasikan akan membahas celah keamanan Security Misconfiguration,RCE,Code Injection, Bypass File Quarantine dan juga bagaiman cara intercept Aplikasi Electron Desktop di system operasi macOS
Muh. Fani Akbar - Infiltrate Into Your AWS Cloud Environment Through Public E...
This document discusses how to infiltrate an AWS cloud environment through publicly exposed services like Elastic Container Registry (ECR), Systems Manager (SSM) documents, Elastic Block Store (EBS) snapshots, Relational Database Service (RDS) snapshots, and Amazon Machine Images (AMI). It outlines attack flows showing how an attacker could access credentials, source code, personal information and other sensitive data from these publicly shared resources. The document recommends mitigation steps like encrypting shared resources, frequent credential rotation, using least privilege access, inventorying all resources, and proper Identity and Access Management (IAM) configuration.
Rama Tri Nanda - NFC Hacking Hacking NFC Reverse Power Supply Padlock.pdf
Near Field Communication (NFC) saat ini adalah teknologi yang umumnya di gunakan untuk media pembayaran serta akses kontrol untuk keamanan ruangan dan gedung. Tidak terbatas untuk hal itu saja, teknologi NFC juga kerap di implementasikan untuk perangkat IoT. Beberapa perangkat menggunakan NFC tag untuk menyimpan informasi guna sinkronisasi dengan perangkat smartphone. Penggunaan teknologi NFC awalnya dianggap aman karna mengharuskan alat baca dengan tag berada dalam poisisi yang sangat dekat. Sehingga dianggap sulit untuk melakukan penyadapan informasinya. Seiring waktu banyak penilitian mengungkapkan bahwa komunikasi ISO 1443-3 ini bisa di intip dan di terjemahkan ke dalam bentuk perintah serta respon aslinya. Proxmark3 adalah salah satu alat yang dikembangkan untuk keperluan tersebut. Namun ada kondisi dimana perangkat proxmark tidak dapat di fungsikan maksimal lantaran berkurangnya sensititifitas pembaca dan tag ketika ada objek berada diantara keduanya. Di paper ini saya ingin menyajikan hasil penelitian saya tentang penggunaan Dynamic Instrumentation Frida untuk memantau penggunaan modul java nfc dalam platform Android dan menggunakannya untuk melakukan lockpicking pada gembok pintar berbasis NFC.
Arief Karfianto - Proposed Security Model for Protecting Patients Data in Ele...
This paper is a documentation of proposed security management for Electronic Health Records which includes security planning and policy, security program, risk management, and protection mechanism. Planning and policy are developed to provide a basic principle of security management at a hospital. The security program in this document includes Risk-Adaptable Access Control (RAdAC) and the implementation of security education, training and awareness (SETA). Regarding risk management, we perform risk identification, inventory of assets, information assets classification, and information assets value assessment, threat identification, and vulnerability assessment. For protection mechanism, we propose biometrics and signature as the authentication methods. The use of firewalls, intrusion detection system and encrypted data transmission is also suggested for securing data, application and network.
Nosa Shandy - Clickjacking That Worthy-Google Bug Hunting Story.pdf
Menceritakan pengalaman bug hunting kerentanan clickjacking pada beberapa produk Google dan membahas beberapa teknik untuk melakukan bypass terhadap kerentanan tersebut. Serta menjelaskan clickjacking yang benar berdasarkan pengalaman pribadi
Baskoro Adi Pratomo - Evaluasi Perlindungan Privasi Pengguna pada Aplikasi-Ap...
Evaluasi privasi pengguna pada aplikasi Android menemukan bahwa: (1) sebagian besar aplikasi mengumpulkan data pribadi pengguna seperti lokasi, kontak, dan aktivitas; (2) data tersebut dapat dibagikan ke pihak ketiga dan tidak mudah dihapus; (3) hanya beberapa aplikasi yang memiliki petugas perlindungan data yang tersedia. Analisis ini menunjukkan pentingnya perlindungan privasi pengguna yang lebih
Utian Ayuba - Profiling The Cloud Crime.pdf
Cloud criminals are motivated by curiosity, recognition, thrill-seeking, and more. They exhibit aggressive and rude behaviors online due to reduced social cues and anonymity. Common traits include narcissism, neuroticism, and social awkwardness. While organizations cannot control criminal motivations and traits, they can minimize opportunities by strengthening security with talent, budgets, technologies, and diligence.
Proactive cyber defence through adversary emulation for improving your securi...
Organization using Adversary Emulation plan to develop an attack emulation and/or simulation and execute it against enterprise infrastructure. These activities leverage real-world attacks and TTPs by Threat Actor, so you can identify and finding the gaps in your defense before the real adversary attacking your infrastructure. Adversary Emulation also help security team to get more visibility into their environment. Performing Adversary Emulation continuously to strengthen and improve your defense over the time.
Perkembangan infrastruktur kunci publik di indonesia - Andika Triwidada
UU-ITE pasal 11 melegalkan Tanda Tangan Elektronik, membuat kedudukannya setara dengan tanda tangan basah. Implementasinya mengandalkan Infrastruktur Kunci Publik yang melibatkan beberapa organisasi dan jalinan trust. Akan di bahas gambaran umum implementasi IKP di Indonesia dan berbagai layanan yang telah beroperasi, serta sebagian aspek keamanannya.
Pentesting react native application for fun and profit - Abdullah
React Native merupakan framework untuk membuat aplikasi native menggunakan Javascript. Aplikasi dibundling menjadi satu file .bundle yang berisi seluruh logika aplikasi. Hal ini menjadikan aplikasi rentan terhadap manipulasi kode. Beberapa celah keamanan yang mungkin ada meliputi manipulasi endpoint API, mengeksploitasi fitur Firebase, menemukan kredensial akun, atau penyimpanan data secara tidak aman.
Hacking oximeter untuk membantu pasien covid19 di indonesia - Ryan fabella
Pandemi covid-19 melonjak pada gelombang ke-2 di. Untuk mengantisipasi itu pemerintah membagikan oximeter ke puskesmas. Oximeter yang ada dipasaran mengharuskan tenaga kesehatan untuk kontak langsung dengan pasien. Dengan menggunakan Hacked Oxymeter ini dapat mengurangi intensitas bertemu dengan pasien dan mengurangi resiko terpapar covid-19. Secara metodologi, hacking oximeter ini membaca output komunikasi serial pada alat oximeter untuk kemudian diolah oleh mikrokontroler dan dikirim ke MQTT broker untuk diteruskan ke klien yang membutuhkan. Alat ini digunakan oleh pasien yang sedang isoman di hotel, fasilitas Kesehatan atau rumah sakit darurat/lapangan
Vm escape: case study virtualbox bug hunting and exploitation - Muhammad Alif...
Eksploitasi kerentanan pada hypervisor semakin banyak diperbincangkan di beberapa tahun ini, dimulai dari kompetisi hacking Pwn2Own pada 2017 yang mengadakan kategori Virtual Machine dalam ajang lombanya, dan juga teknologi-teknologi terkini yang banyak menggunakan hypervisor seperti Cloud Computing, Malware Detection, dll. Hal tersebut menjadi ketertarikan bagi sebagian hacker, security researcher untuk mencari kelemahan dan mengeksploitasi hypervisor. Tulisan ini menjelaskan mengenai proses Vulnerability Research dan VM Escape exploitation pada VirtualBox.
Devsecops: membangun kemampuan soc di dalam devsecops pipeline - Dedi Dwianto
Proses DevSecOps saat ini banyak digunakan dikalangan industri yang membutuhkan kecepatan baik dalam pengembangan maupun implementasi. Setiap tahapan pada pipeline DevSecOps merupakan tahapan yang harus diperhatikan dan masuk kedalam pantauan SOC (Security Operation Center). Untuk itu diperlukan kemampuan SOC untuk bisa memantau setiap pipeline DevSecOps sehingga dapat memberikan gambaran kondisi keamanan pada organisasi
More from idsecconf (20)
idsecconf2023 - Mochammad Riyan Firmansyah - Takeover Cloud Managed Router vi...
idsecconf2023 - Mochammad Riyan Firmansyah - Takeover Cloud Managed Router vi...
idsecconf2023 - Neil Armstrong - Leveraging IaC for Stealthy Infrastructure A...
idsecconf2023 - Neil Armstrong - Leveraging IaC for Stealthy Infrastructure A...
idsecconf2023 - Mangatas Tondang, Wahyu Nuryanto - Penerapan Model Detection ...
idsecconf2023 - Mangatas Tondang, Wahyu Nuryanto - Penerapan Model Detection ...
idsecconf2023 - Rama Tri Nanda - Hacking Smart Doorbell.pdf
idsecconf2023 - Rama Tri Nanda - Hacking Smart Doorbell.pdf
idsecconf2023 - Akshantula Neha, Mohammad Febri Ramadlan - Cyber Harmony Auto...
idsecconf2023 - Akshantula Neha, Mohammad Febri Ramadlan - Cyber Harmony Auto...
idsecconf2023 - Aan Wahyu - Hide n seek with android app protections and beat...
idsecconf2023 - Aan Wahyu - Hide n seek with android app protections and beat...
idsecconf2023 - Satria Ady Pradana - Launch into the Stratus-phere Adversary ...
idsecconf2023 - Satria Ady Pradana - Launch into the Stratus-phere Adversary ...
Ali - The Journey-Hack Electron App Desktop (MacOS).pdf
Ali - The Journey-Hack Electron App Desktop (MacOS).pdf
Muh. Fani Akbar - Infiltrate Into Your AWS Cloud Environment Through Public E...
Muh. Fani Akbar - Infiltrate Into Your AWS Cloud Environment Through Public E...
Rama Tri Nanda - NFC Hacking Hacking NFC Reverse Power Supply Padlock.pdf
Rama Tri Nanda - NFC Hacking Hacking NFC Reverse Power Supply Padlock.pdf
Arief Karfianto - Proposed Security Model for Protecting Patients Data in Ele...
Arief Karfianto - Proposed Security Model for Protecting Patients Data in Ele...
Nosa Shandy - Clickjacking That Worthy-Google Bug Hunting Story.pdf
Nosa Shandy - Clickjacking That Worthy-Google Bug Hunting Story.pdf
Baskoro Adi Pratomo - Evaluasi Perlindungan Privasi Pengguna pada Aplikasi-Ap...
Baskoro Adi Pratomo - Evaluasi Perlindungan Privasi Pengguna pada Aplikasi-Ap...
Proactive cyber defence through adversary emulation for improving your securi...
Proactive cyber defence through adversary emulation for improving your securi...
Perkembangan infrastruktur kunci publik di indonesia - Andika Triwidada
Perkembangan infrastruktur kunci publik di indonesia - Andika Triwidada
Pentesting react native application for fun and profit - Abdullah
Pentesting react native application for fun and profit - Abdullah
Hacking oximeter untuk membantu pasien covid19 di indonesia - Ryan fabella
Hacking oximeter untuk membantu pasien covid19 di indonesia - Ryan fabella
Vm escape: case study virtualbox bug hunting and exploitation - Muhammad Alif...
Vm escape: case study virtualbox bug hunting and exploitation - Muhammad Alif...
Devsecops: membangun kemampuan soc di dalam devsecops pipeline - Dedi Dwianto
Devsecops: membangun kemampuan soc di dalam devsecops pipeline - Dedi Dwianto
Recently uploaded
Mind map of terminologies used in context of Generative AI
Mind map of common terms used in context of Generative AI.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
“I’m still / I’m still / Chaining from the Block”
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Securing your Kubernetes cluster_ a step-by-step guide to success !
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
UiPath Test Automation using UiPath Test Suite series, part 5
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
National Security Agency - NSA mobile device best practices
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
GridMate - End to end testing is a critical piece to ensure quality and avoid...
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
DevOps and Testing slides at DASA Connect
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications.
Recently uploaded (20)
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GridMate - End to end testing is a critical piece to ensure quality and avoid...
GridMate - End to end testing is a critical piece to ensure quality and avoid...
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Large Language Model (LLM) and it’s Geospatial Applications
Large Language Model (LLM) and it’s Geospatial Applications
Collecting Endpoint Security Logs Through Big Data Technology - Dedi Dwianto
- 1. BIG DATA TECHNOLOGY COLLECTING ENDPOINT SECURITY LOGS Dedi Dwianto, M.Kom,C|EH,OSCP,eMPAT,CREST|CRT,ISO 27001:LA - 2018
- 2. TEXT D
- 5. BIGDATA DATA SOURCES ▸ Clickstream data ▸ Shopping cart data ▸ Social networking data ▸ Sensor data
- 7. LOGS LOGS ▸ A file that records either events that occur in an operating system or other software runs
- 8. LOGS
- 9. TOOLS OPEN SOURCE LOGS MANAGEMENT ▸ Logstash An open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously ▸ Elasticsearch An open-source, RESTful, distributed search and analytics engine built on Apache Lucene
- 10. TOOLS LOGSTASH
- 11. TOOLS LOGSTASH
- 12. TOOLS ARCHITECTURE BIG DATA EVENT LOG APP (APACHE FLUME) REST API ELASTIC SEARCHLOGSTASH YARN (PIG/HIVE/SPARK) HDFS