The Beats are a friendly army of lightweight agents that installed on your servers capture operational data and ship it to Elasticsearch for analysis. They are open source, written in Golang, and maintained by Elastic, the company behind Elasticsearch, Logstash, and Kibana.
This talk will present the first three Beats: Topbeat for system level metrics, Filebeat for log files and Packetbeat for wire data. It will also demonstrate how to combine them with Logstash and Kibana in one advanced monitoring solution, unifying log management, metrics monitoring and system stats. Finally, you will learn how to create a new Beat from scratch using Golang and the libbeat framework to capture any type of information and ship it to Elasticsearch.
A talk about Open Source logging and monitoring tools, using the ELK stack (ElasticSearch, Logstash, Kibana) to aggregate logs, how to track metrics from systems and logs, and how Drupal.org uses the ELK stack to aggregate and process billions of logs a month.
During this brief walkthrough of the setup, configuration and use of the toolset we will show you how to find the trees from the forest in today's modern cloud environments and beyond.
Slides for a college course based on "Incident Response & Computer Forensics, Third Edition" by by Jason Luttgens, Matthew Pepe, and Kevin Mandia.
Teacher: Sam Bowne
Twitter: @sambowne
Website: https://samsclass.info/121/121_F16.shtml
We're talking about serious log crunching and intelligence gathering with Elastic, Logstash, and Kibana.
ELK is an end-to-end stack for gathering structured and unstructured data from servers. It delivers insights in real time using the Kibana dashboard giving unprecedented horizontal visibility. The visualization and search tools will make your day-to-day hunting a breeze.
During this brief walkthrough of the setup, configuration, and use of the toolset, we will show you how to find the trees from the forest in today's modern cloud environments and beyond.
A talk about Open Source logging and monitoring tools, using the ELK stack (ElasticSearch, Logstash, Kibana) to aggregate logs, how to track metrics from systems and logs, and how Drupal.org uses the ELK stack to aggregate and process billions of logs a month.
During this brief walkthrough of the setup, configuration and use of the toolset we will show you how to find the trees from the forest in today's modern cloud environments and beyond.
Slides for a college course based on "Incident Response & Computer Forensics, Third Edition" by by Jason Luttgens, Matthew Pepe, and Kevin Mandia.
Teacher: Sam Bowne
Twitter: @sambowne
Website: https://samsclass.info/121/121_F16.shtml
We're talking about serious log crunching and intelligence gathering with Elastic, Logstash, and Kibana.
ELK is an end-to-end stack for gathering structured and unstructured data from servers. It delivers insights in real time using the Kibana dashboard giving unprecedented horizontal visibility. The visualization and search tools will make your day-to-day hunting a breeze.
During this brief walkthrough of the setup, configuration, and use of the toolset, we will show you how to find the trees from the forest in today's modern cloud environments and beyond.
Nagios Conference 2013 - Janice Singh - Visualization of Monitoring Data at t...Nagios
Janice Singh's presentation on Visualization of Monitoring Data at the NASA Advanced Supercomputing Facilityuting facility.
The presentation was given during the Nagios World Conference North America held Sept 20-Oct 2nd, 2013 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
Logging is one of those things that everyone complains about, but doesn't dedicate time to. Of course, the first rule of logging is "do it". Without that, you have no visibility into system activities when investigations are required. But, the end goal is much, much more than this. Almost all applications require security audit logs for compliance; application logs for visibility across all cloud properties; and application tracing for tracking usage patterns and business intelligence. The latter is that magic sauce that helps businesses learn about their customer or in some cases the data is FOR the customer. Without a strategy this can get very messy, fast. In this session Michele will discuss design patterns for a sound logging and audit strategy; considerations for security and compliance; the benefits of a noSQL approach; and more.
ArcBlock Technical Learning Series Presents IPFS.
If there's a missing piece in current blockchain stack, that'll be a decentralized, public verifiable file system. Ideally before decentralizing computing, we shall decentralize the data. IPFS filled in this area, and it has a great potential to push web to the true web3 - decentralized web. This talk will talk about what problem IPFS is trying to solve, how it solves the problem, and how to use IPFS in our applications.
https://www.arcblock.io
https://hack.arcblock.io/learning
Chinmay Kolhatkar: Engineer, DataTorrent & Committer, Apache Apex
For ease of use and deployment, Apache Apex leverages Apache Bigtop. Apex, being part of bigtop stack, can be easily deployed in both debian and rpm based cluster system and run validation tests for installation. This talk will cover a demo on how to install apex-bigtop and use it. It also covers a test sandbox docker environment, having pre-installed bigtop-hadoop and bigtop-apex, for quickly getting started with apex.
IPFS is a distribution protocol that enables the creation of completely distributed applications through content addressing. A very ambitious open source project in Go, IPFS adopts a peer-to-peer hypermedia protocol to protect against a single point of failure. This presentation aims to highlight the design and ideas of IPFS and also touches upon a real world use case.
OSDC 2016 - Introduction to Testing Puppet Modules by David SchmittNETWAYS
Are you a puppet module author? Do other people use your puppet code? Do you want to know what your change broke? Would future-you still know what you expected here? Using automated tests can help you with this. There are tools to help you answer all this for puppet modules, but only few use them. Join this talk to get an introduction to the rspec-puppet and beaker-rspec test frameworks, learn how to run automated tests on your puppet modules, and learn how to write good tests.
Nagios Conference 2013 - Janice Singh - Visualization of Monitoring Data at t...Nagios
Janice Singh's presentation on Visualization of Monitoring Data at the NASA Advanced Supercomputing Facilityuting facility.
The presentation was given during the Nagios World Conference North America held Sept 20-Oct 2nd, 2013 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
Logging is one of those things that everyone complains about, but doesn't dedicate time to. Of course, the first rule of logging is "do it". Without that, you have no visibility into system activities when investigations are required. But, the end goal is much, much more than this. Almost all applications require security audit logs for compliance; application logs for visibility across all cloud properties; and application tracing for tracking usage patterns and business intelligence. The latter is that magic sauce that helps businesses learn about their customer or in some cases the data is FOR the customer. Without a strategy this can get very messy, fast. In this session Michele will discuss design patterns for a sound logging and audit strategy; considerations for security and compliance; the benefits of a noSQL approach; and more.
ArcBlock Technical Learning Series Presents IPFS.
If there's a missing piece in current blockchain stack, that'll be a decentralized, public verifiable file system. Ideally before decentralizing computing, we shall decentralize the data. IPFS filled in this area, and it has a great potential to push web to the true web3 - decentralized web. This talk will talk about what problem IPFS is trying to solve, how it solves the problem, and how to use IPFS in our applications.
https://www.arcblock.io
https://hack.arcblock.io/learning
Chinmay Kolhatkar: Engineer, DataTorrent & Committer, Apache Apex
For ease of use and deployment, Apache Apex leverages Apache Bigtop. Apex, being part of bigtop stack, can be easily deployed in both debian and rpm based cluster system and run validation tests for installation. This talk will cover a demo on how to install apex-bigtop and use it. It also covers a test sandbox docker environment, having pre-installed bigtop-hadoop and bigtop-apex, for quickly getting started with apex.
IPFS is a distribution protocol that enables the creation of completely distributed applications through content addressing. A very ambitious open source project in Go, IPFS adopts a peer-to-peer hypermedia protocol to protect against a single point of failure. This presentation aims to highlight the design and ideas of IPFS and also touches upon a real world use case.
OSDC 2016 - Introduction to Testing Puppet Modules by David SchmittNETWAYS
Are you a puppet module author? Do other people use your puppet code? Do you want to know what your change broke? Would future-you still know what you expected here? Using automated tests can help you with this. There are tools to help you answer all this for puppet modules, but only few use them. Join this talk to get an introduction to the rspec-puppet and beaker-rspec test frameworks, learn how to run automated tests on your puppet modules, and learn how to write good tests.
OSDC 2016 - Bareos Backup Integration with Standard Open Source Tools by Maik...NETWAYS
Bareos is a reliable network open source software to backup, archive and restore files from all major operating systems. Bareos backups to disk, tape (-libraries) or cloud storages, it has a new web UI, a new Python plugin interface and many more other new features.
Bareos can be easily integrated into common open source datacenter toolchains and facilities like Icinga, Salt, Puppet, Ceph and others.
This session gives an overview of Bareos and its interfaces.
OSDC 2016 - Hybrid Cloud - A Cloud Migration Strategy by Schlomo SchapiroNETWAYS
Do you use Cloud? Why? What about the 15 year legacy of your data center? How many Enterprise vendors tried to sell you their "Hybrid Cloud" solution? What actually is a Hybrid Cloud?
Cloud computing is not just a new way of running servers or Docker containers. The interesting part of any Cloud offering are managed services that provide solutions to difficult problems. Prime examples are messaging (SNS/SQS), distributed storage (S3), managed databases (RDS) and especially turn-key solutions like managed Hadoop (EMR).
Hybrid Cloud is usually understood as a way to unify or standardize server hosting across private data centers and Public Cloud vendors. Some Hybrid Cloud solutions even go as far as providing a unified API that abstracts away all the differences between different platforms. Unfortunately that approach focuses on the lowest common denominator and effectively prevents using the advanced services that each Cloud vendor also offers. However, these services are the true value of Public Cloud vendors.
Another approach to integrating Public Cloud and private data centers is using services from both worlds depending on the problems to solve. Don't hide the cloud technologies but make it simple to use them - both from within the data center and the cloud instances. Create a bridge between the old world of the data center and the new world of the Public Cloud. A good bridge will motivate your developers to move the company to the cloud.
Based upon recent developments at ImmobilienScout24, this talk tries to suggest a sustainable Cloud migration strategy from private data centers through a Hybrid Cloud into the AWS Cloud.
Bridging the security model of the data center with the security model of AWS.
Integrating the AWS identity management (IAM) with the existing servers in the data center.
Secure communication between services running in the data center and in AWS.
Deploying data center servers and Cloud resources together.
Service discovery for services running both in the data center and AWS.
Most of the tools used are Open Source and this talk will show how they come together to support this strategy:
AWS credential provider for employees and data center servers: http://immobilienscout24.github.io/afp/
Cloud Formation automation: https://github.com/ImmobilienScout24/cfn-sphere
Compliancy with European privacy laws: https://github.com/ImmobilienScout24/aws-monocyte
OSDC 2016 - An Introduction to Software Defined Networking (SDN) by Martin Lo...NETWAYS
Clouds and massively scalable setups impose new requirements on datacenters; hardly anywhere does this become as obvious as in the storage and networking areas. While a number of Software Defined Storage (SDS) solutions are established by now, Software Defined Networking (SDN) is still a new topic for many administrators and IT managers. This presentation will give a basic introduction into Software Defined Networking; it will explain the basic concepts behind SDN and why it is required in modern datacenters. In addition, it will give a quick overview over the market and compare three solutions, namely Open vSwitch, Midonet by Midokura and OpenContrail by Juniper.
The job of a Product Manager within the fashion and retail industry is to oversee the product planning and execution throughout the fashion product lifecycle.
Oversee the Product Development team and production team as well as working closely with all stakeholders.
OSDC 2016 - Ingesting Logs with Style by Pere Urbon-BayesNETWAYS
The log shipping scene been between us for a long time: from syslog, rsyslog to nowadays Fluentd, Flume and Logstash. Logstash been pushing hard to introduce new features that make the experience better for everyone. At the end of the day, a healthy shipper means a happy sysadmin. The latest Logstash includes persistence to reduce the chance of data loss, monitoring to find how everything is going and configuration management to make your life a lot easier. But wait, there’s more! Offline support, improved shutdown semantics, etc … features that will make your logs shipped and you a rested sysadmin.
In this talk we’ll see this features in action thought a real live sensor monitoring example. By the end of the session, you will be able to use the full power of Logstash in your own deployments.
Slides for a college course based on "Incident Response & Computer Forensics, Third Edition" by by Jason Luttgens, Matthew Pepe, and Kevin Mandia.
Teacher: Sam Bowne
Twitter: @sambowne
Website: https://samsclass.info/121/121_F16.shtml
Slides for a college course based on "Incident Response & Computer Forensics, Third Edition" by by Jason Luttgens, Matthew Pepe, and Kevin Mandia, at City College San Francisco.
Website: https://samsclass.info/152/152_F18.shtml
Managing your Black Friday Logs - Antonio Bonuccelli - Codemotion Rome 2018Codemotion
Monitoring an entire application is not a simple task, but with the right tools it is not a hard task either. However, events like Black Friday can push your application to the limit, and even cause crashes. As the system is stressed, it generates a lot more logs, which may crash the monitoring system as well. In this talk I will walk through the best practices when using the Elastic Stack to centralize and monitor your logs. I will also share some tricks to help you with the huge increase of traffic typical in Black Fridays.
Managing Your Security Logs with ElasticsearchVic Hargrave
The ELK stack (Elasticsearch-Logstash-Kibana) provides a cost effective alternative to commercial SIEMs for ingesting and managing OSSEC alert logs. This presentation will show you how to construct a low cost SIEM based on ELK that rivals the capabilties of commercials SIEMs.
The presentation belonging to the ALTEN Playground of november 2, 2017. More information on this playground can be found here: https://www.gitbook.com/book/matthijsmali/user-metrics/
OSMC 2016 - Monitor your infrastructure with Elastic Beats by Monica SarbuNETWAYS
Monica ist Mit-Schöpferin von Elastic Beats. Bevor sie Beats erfand, arbeitete sie als Core Developer für IPTEGO, einem Start-Up Unternehmen aus Berlin, das eine komplette Monitoring und Trouble-Shooting Solution für VoIP Netzwerke anbietet. Das Produkt wurde weltweit verkauft, und wird derzeit von großen Firmen der Telekommunikationsbranche verwendet.
OSMC 2016 | Monitor your Infrastructure with Elastic Beats by Monica SarbuNETWAYS
Beats sind eine freundliche Armee von leichtgewichtigen Agenten die, wenn sie auf dem Server installiert sind, Betriebsdaten erfassen und sie zur Analyse an Elasticsearch senden.
Sie sammeln die Logdaten ihrer Server und erhalten so Statistiken von CPU, Disk- und Speicherauslastung. Durch regelmäßige Abfragen sammeln sie Metriken von externen Systemen wie MySQL, Docker und Zookeeper und können die Kommunikation zwischen den Servern durch sniffen der entsprechenden Netzwerkverbindungen visualisieren.
Dieser Vortrag erläutert wie Sie Beats mit Elasticsearch und Kibana in einer kompletten Open Source Monitoring Lösung kombinieren können und sie ihnen helfen ihre verzweigte Infrastruktur zu überwachen und Fehler zu beheben.
Adding Support for Networking and Web Technologies to an Embedded SystemJohn Efstathiades
These are the slides for a presentation we gave at Device Developer Conference 2014 in the UK. The presentation discusses the work done, experiences, and lessons learnt from adding an open source TCP/IP network stack and web server to an existing industrial control system running on an ARM Cortex M3-based processor from TI.
The presentation covers the following:
· Integrating the network stack into the existing software base
· Configuring and using the network stack and web server
· Adding support for HTTP basic authentication to restrict user access
· Using HTTP to remotely access the target system and retrieve operational data
· Debugging hints and tips
· Pitfalls to avoid and other lessons learnt
Workshop: Big Data Visualization for SecurityRaffael Marty
Big Data is the latest hype in the security industry. We will have a closer look at what big data is comprised of: Hadoop, Spark, ElasticSearch, Hive, MongoDB, etc. We will learn how to best manage security data in a small Hadoop cluster for different types of use-cases. Doing so, we will encounter a number of big-data open source tools, such as LogStash and Moloch that help with managing log files and packet captures.
As a second topic we will look at visualization and how we can leverage visualization to learn more about our data. In the hands-on part, we will use some of the big data tools, as well as a number of visualization tools to actively investigate a sample data set.
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDogRedis Labs
Think you have big data? What about high availability
requirements? At DataDog we process billions of data points every day including metrics and events, as we help the world
monitor the their applications and infrastructure. Being the world’s monitoring system is a big responsibility, and thanks to
Redis we are up to the task. Join us as we discuss how the DataDog team monitors and scales Redis to power our SaaS based monitoring offering. We will discuss our usage and deployment patterns, as well as dive into monitoring best practices for production Redis workloads
Lessons Learned Running InfluxDB Cloud and Other Cloud Services at Scale by T...InfluxData
In this session, Tim will cover principles, learnings, and practical advice from operating multiple cloud services at scale, including of course our InfluxDB Cloud service. What do we monitor, what do we alert on, and how did we architect it all? What are our underlying architectural and operational principles?
Attackers don’t just search for technology vulnerabilities, they take the easiest path and find the human vulnerabilities. Drive by web attacks, targeted spear phishing, and more are commonplace today with the goal of delivering custom malware. In a world where delivering custom advanced malware that handily evades signature and blacklisting approaches, and does not depend on application software vulnerabilities, how do we understand when are environments are compromised? What are the telltale signs that compromise activity has started, and how can we move to arrest a compromise in progress before the attacker laterally moves and reinforces their position? The penetration testing community knows these signs and artifacts of advanced malware presence, and it is up to us to help educate defenders on what to look for.
Blackhat USA 2016 - What's the DFIRence for ICS?Chris Sistrunk
Digital Forensics and Incident Response (DFIR) for IT systems has been around quite a while, but what about Industrial Control Systems (ICS)? This talk will explore the basics of DFIR for embedded devices used in critical infrastructure such as Programmable Logic Controllers (PLCs), Remote Terminal Units (RTUs), and controllers. If these are compromised or even have a misoperation, we will show what files, firmware, memory dumps, physical conditions, and other data can be analyzed in embedded systems to determine the root cause.
This talk will show examples of what and how to collect forensics data from two popular RTUs that are used in Electric Substations: the General Electric D20MX and the Schweitzer Engineering Labs SEL-3530 RTAC.
This talk will not cover Windows or *nixbased devices such as Human Machine Interfaces (HMIs) or gateways.
Lessons Learned: Running InfluxDB Cloud and Other Cloud Services at Scale | T...InfluxData
In this session, Tim will cover principles, learnings, and practical advice from operating multiple cloud services at scale, including of course our InfluxDB Cloud service. What do we monitor, what do we alert on, and how did we architect it all? What are our underlying architectural and operational principles?
Docker Logging and analysing with Elastic StackJakub Hajek
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. What aspects should be considered while you design your logging solutions?
Similar to OSDC 2016 - Unifying Logs and Metrics Data with Elastic Beats by Monica Sarbu (20)
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
Games are powerful teaching tools, fostering hands-on engagement and fun. But they require careful consideration to succeed. Join me to explore factors in running and selecting games, ensuring they serve as effective teaching tools. Learn to maintain focus on learning objectives while playing, and how to measure the ROI of gaming in education. Discover strategies for pitching gaming to leadership. This session offers insights, tips, and examples for coaches, team leads, and enterprise leaders seeking to teach from simple to complex concepts.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Why React Native as a Strategic Advantage for Startup Innovation.pdfayushiqss
Do you know that React Native is being increasingly adopted by startups as well as big companies in the mobile app development industry? Big names like Facebook, Instagram, and Pinterest have already integrated this robust open-source framework.
In fact, according to a report by Statista, the number of React Native developers has been steadily increasing over the years, reaching an estimated 1.9 million by the end of 2024. This means that the demand for this framework in the job market has been growing making it a valuable skill.
But what makes React Native so popular for mobile application development? It offers excellent cross-platform capabilities among other benefits. This way, with React Native, developers can write code once and run it on both iOS and Android devices thus saving time and resources leading to shorter development cycles hence faster time-to-market for your app.
Let’s take the example of a startup, which wanted to release their app on both iOS and Android at once. Through the use of React Native they managed to create an app and bring it into the market within a very short period. This helped them gain an advantage over their competitors because they had access to a large user base who were able to generate revenue quickly for them.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?XfilesPro
Worried about document security while sharing them in Salesforce? Fret no more! Here are the top-notch security standards XfilesPro upholds to ensure strong security for your Salesforce documents while sharing with internal or external people.
To learn more, read the blog: https://www.xfilespro.com/how-does-xfilespro-make-document-sharing-secure-and-seamless-in-salesforce/
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Your Digital Assistant.
Making complex approach simple. Straightforward process saves time. No more waiting to connect with people that matter to you. Safety first is not a cliché - Securely protect information in cloud storage to prevent any third party from accessing data.
Would you rather make your visitors feel burdened by making them wait? Or choose VizMan for a stress-free experience? VizMan is an automated visitor management system that works for any industries not limited to factories, societies, government institutes, and warehouses. A new age contactless way of logging information of visitors, employees, packages, and vehicles. VizMan is a digital logbook so it deters unnecessary use of paper or space since there is no requirement of bundles of registers that is left to collect dust in a corner of a room. Visitor’s essential details, helps in scheduling meetings for visitors and employees, and assists in supervising the attendance of the employees. With VizMan, visitors don’t need to wait for hours in long queues. VizMan handles visitors with the value they deserve because we know time is important to you.
Feasible Features
One Subscription, Four Modules – Admin, Employee, Receptionist, and Gatekeeper ensures confidentiality and prevents data from being manipulated
User Friendly – can be easily used on Android, iOS, and Web Interface
Multiple Accessibility – Log in through any device from any place at any time
One app for all industries – a Visitor Management System that works for any organisation.
Stress-free Sign-up
Visitor is registered and checked-in by the Receptionist
Host gets a notification, where they opt to Approve the meeting
Host notifies the Receptionist of the end of the meeting
Visitor is checked-out by the Receptionist
Host enters notes and remarks of the meeting
Customizable Components
Scheduling Meetings – Host can invite visitors for meetings and also approve, reject and reschedule meetings
Single/Bulk invites – Invitations can be sent individually to a visitor or collectively to many visitors
VIP Visitors – Additional security of data for VIP visitors to avoid misuse of information
Courier Management – Keeps a check on deliveries like commodities being delivered in and out of establishments
Alerts & Notifications – Get notified on SMS, email, and application
Parking Management – Manage availability of parking space
Individual log-in – Every user has their own log-in id
Visitor/Meeting Analytics – Evaluate notes and remarks of the meeting stored in the system
Visitor Management System is a secure and user friendly database manager that records, filters, tracks the visitors to your organization.
"Secure Your Premises with VizMan (VMS) – Get It Now"
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
7. 7
All kinds of operational data
• Filebeat
• collects logs
• Winlogbeat
• collects Windows event logs
• Packetbeat
• collects insides from the
network packets
not released
• Topbeat
• collects system statistics like
CPU usage, disk usage,
memory usage per process,
etc
• Metricbeat
• collects metrics by
interrogating periodically
external services
9. ‹#›
In Elasticsearch .. you are storing
the raw value … You have the
ability to ask and answer questions
that you didn’t think about when
the data was stored!
Felix Barnsteiner
13. Sniffing the network traffic
13
• Copy traffic at OS or hardware level
• ZERO latency overhead
• Not in the request/response path,
cannot break your application
Client
Server
sniff sniff
15. Monitor the network traffic with OSS tools
15
1 2 3 4
ssh to each of your
server
start a trace using
tcpdump on each
of your server
download trace
from each server to
a common location
merge all traces
5
analyze it with
Wireshark
16. The Problem
16
1 2 3
you have lots of
servers
challenging to see
the traffic
exchanged
between your
servers
Packetbeat makes
it easy
17. Packetbeat overview
It does all of this in real time directly on the target servers
17
1 2 3 4
capture network
traffic
decodes network
traffic
correlates request
& response into
transactions
extract
measurements
5
send data to
Elasticsearch
19. Packetbeat: Configuration
19
# Network interfaces where to sniff the data
interfaces:
device: any
# Specify the type of your network data
protocols:
dns:
ports: [53]
http:
ports: [80, 8080, 8081, 5000, 8002]
mysql:
ports: [3306]
…
31. ‹#› 31
Packetbeat flows
• Look into data for which we don’t
understand the application layer
protocol
• TLS
• Protocols we don’t yet support
• Get data about IP / TCP / UDP layers
• number of packets
• retransmissions
• inter-arrival time
flows:
# network flow timeout
timeout: 30s
# reporting period
period: 10s
36. 36
Filebeat overview
• Simple log forwarder that
sends the log lines to
Elasticsearch
• Successor of Logstash
Forwarder
• It remembers how far it read,
so it never loses log line
• Reads the log files line by
line
• It doesn’t parse the log lines!
37. Filebeat: Parse logs with Logstash
37
• Filebeat sends out unparsed log
lines
• Use filters like Grok, mutate,
geoip to parse the log lines
• Combine the filters with
conditionals or create custom
filters in ruby
• Forward data to other systems
using the Logstash output
plugins
Filebeat
Elasticsearch
Logstash
Other
systems
38. Filebeat: Parse logs with Ingest Node
38
• Ingest node plugin is available
starting with Elasticsearch 5.0.0-
alpha1
• Filebeat sends out unparsed log
lines directly to Elasticsearch
• Use Ingest Node processors to
parse the log lines
• Easier to setup
Filebeat
Elasticsearch
39. Filebeat: Configuration
Configure prospectors to forward the log lines
39
filebeat:
# List of prospectors to fetch data.
prospectors:
# Type of files: log or stdin
- input_type: log
# Files that should be crawled and fetched.
paths:
- “/var/log/apache2/*”
# File encoding: plain, utf-8, big5, gb18030, …
encoding: plain
42. ‹#›
Multiline
42
multiline:
# Sticks together all lines
# that don’t start with a [
pattern: ^[
negate: true
match: after
Filebeat extra power
• Sticks together related log lines in a
single event
• For all those long exceptions
• Can also be done by Logstash, but it’s
sometimes easier to configure the
patterns closer to the source
45. ‹#› 45
json:
keys_under_root: false
message_key: “message”
overwrite_keys: false
add_error_key: false
Filebeat extra power JSON logs
• application logs in JSON format
• you don’t have to choose what data to
include in the log line
• don’t need to use grok filters from
Logstash to parse the application logs
48. ‹#›
Basic filtering
48
# Only send lines starting with
# ERR or WARN
include_lines: [“^ERR”, “^WARN”]
# Exclude lines containing
# a keyword
exclude_lines: [“Request received”]
# Exclude files all together
exclude_files: [“.gz$”]
Filebeat extra power
• Because removing stuff at the source
is more efficient
• Flexible Whitelist + Blacklist regexp
log line filtering
• Efficient log files filtering (excluded
files are never opened)
• Works on multiline too
50. 50
Winlogbeat overview
• Sends out unparsed
Windows event logs
• Remembers how far it read,
so it never loses any
Windows event logs
• Use Ingest Node or Logstash
to parse the Windows event
logs
51. Winlogbeat: Configuration
Specify the event logs that you want to monitor
51
winlogbeat:
#list of event logs to monitor
event_logs:
- name: Application
- name: Security
- name: System
54. 54
Topbeat overview
• Like the Unix top command
but instead of printing the
system statistics on the
screen it sends them
periodically to Elasticsearch
• Works also on Windows
55. Topbeat: Exported data
55
• system load
• total CPU usage
• CPU usage per core
• Swap, memory usage
System wide
• state
• name
• command line
• pid
• CPU usage
• memory usage
Per process
• available disks
• used, free space
• mounted points
Disk usage
56. Topbeat configuration
Specify the system statistics that you want to monitor
56
topbeat:
# how often to send system statistics
period: 10
# specify the processes to monitor
procs: [".*"]
# Statistics to collect (all enabled by default)
stats:
system: true
process: true
filesystem: true
67. Metricbeat: how it works
67
1 2 3
Periodically polls
monitoring APIs
of various
services
Groups
performance
data into
documents
Ships them to
Elasticsearch
68. Metricbeat: A module for each metric type
68
Metricbeat
apache
module
mysql
module
redis
module
system
module +
69. 69
Metricbeat: It is also a library!
• Use the Metricbeat infrastructure,
to create a standalone Beat
• You can create a Beat with a single
module that exports your custom
data
• Can use the built in Metricbeat
modules
Metricbeat
df module
github.com/ruflin/df2beat
70. Metricbeat module vs standalone Beat
70
• Contributed via PR to the
elastic/beats Github
repository
• Officially supported
• Supports common
systems
• Docker based integration
tests
Metricbeat module
• In a separate Github
repository
• Supported by the
community
• Supports specialized
systems
• Optional Docker based
integration tests
Standalone Beat
71. Provide a platform to make it
easier to build custom Beats
on top of it
71
73. libbeat
73
• Written in Go
• Provide common functionality for
reading configuration files, for
handling CLI arguments, for logging
• Makes sure reliably send the data out
• Provide things like encryption,
authentication with certificates
• Has support for different outputs:
Elasticsearch, Logstash, Redis, Kafka
libbeat
Outputs
76. Official vs Community Beats
76
• In the elastic/beats
Github repository
• Officially supported
• Synced releases with the
whole stack
Official Beats
• In another Github
repository
• Supported by the
community
• Releases at any time
Community Beats
78. ‹#› 78
input:
# Loop every 5 seconds
period: 5
# Use raw sockets for ping
# Requires root!
privileged: true
# Whether to perform IPv4/v6 pings
useipv4: true
useipv6: false
# List targets under the tag
# you want assigned to
targets:
# tag: google
google:
- google.com.au
- google.com
You know, for pings
• Sends ICMP (v4 or v6) pings
periodically to a list of hosts
• Can send also UDP pings (no root
required)
• Resolves DNS
• Records RTT
Pingbeat
80. ‹#› 80
Execbeat
execbeat:
execs:
# Each - Commands to execute.
-
# Cron expression
# Default is every 1 minute.
cron: "@every 10s"
# The command to execute
command: echo
args: "Hello World"
document_type: jolokia
fields:
host: test2
• Accepts cron expressions
• Sends stdout and stderr to Elastic
search
• Use Logstash and Grok to further
parse the output
Run any command
82. ‹#› 82
Dockerbeat
Docker Monitoring
• Uses the Docker API
• Exports per container stats about:
• CPU
• Memory
• Disk
• Network
• IO access
• Log
input:
# In seconds, defines how often to
# read server statistics
period: 5
# Define the docker socket path
# By default, this will get the
# unix:///var/run/docker.sock
socket:
88. 88
Beats Packer
• Cross-compiles to all our
supported platforms
• Produces RPMs, DEBs,
• Same tools that we use to build
the official Elastic Beats
• Can be executed from Travis CI
89. Multiple data types, one view in Kibana
89
• metrics
• flows
• logs
• system stats
• transactions
• transactions
• metrics
• metrics
• logs
• logs
• system stats
• flows
• flows
• metrics
• logs
90. Monitor MySQL with Elastic Stack
90
Metricbeat
mysql …
Filebeat
log …
Packetbeat
mysql …
Elasticsearch
Kibana
stats queries
slow queries
91. Monitor web server with Elastic Stack
91
Metricbeat
mysql apache
Filebeat
log …
Packetbeat
mysql http
Elasticsearch
Kibana
mysql & apache stats
queries & HTTP transactions
slow queries apache logs
93. ‹#› 93
Want to hear more about
Logstash?
Don’t miss Ingest Logs with
Style by Pere Urbon-Bayes
Thursday 12:00pm - 1:00pm in
MOA 05
94. ‹#›
Q&A
Find us on:
• github.com/elastic/beats
• discuss.elastic.co
• @elastic #elasticbeats
• #beats on freenode
Or Here. In Real Life!
95. ‹#›
Please attribute Elastic with a link to elastic.co
Except where otherwise noted, this work is licensed under
http://creativecommons.org/licenses/by-nd/4.0/
Creative Commons and the double C in a circle are
registered trademarks of Creative Commons in the United States and other countries.
Third party marks and brands are the property of their respective holders.
95