Who Watches the Watchers?
•Download as PPTX, PDF•
0 likes•538 views
Security and asset integrity are becoming an increasingly important part of the asset management world. Detecting both intentional and unintentional asset or metadata modification is an important part of ensuring the overall integrity of an asset system. This talk will discuss how Pixar cross-check and cross-index their Perforce asset data and metadata with Pixar's Templar System so as to detect both intentional and unintentional file and metadata modification or tampering.
Report
Share
Report
Share
Recommended
WHOIS the Master
Presents a WHOIS database search engine tool I wrote to allow pentesters to access network information for specified targets. First presented at BSidesDE 2010
Artificial Assistants: How can I help you? by Christopher Currin
Chatbots are not equal; with different forms permeating our lives more and more. Virtual assistants are increasingly relevant for businesses and our day-to-day lives. Chatbots have become ubiquitous for interactions, yet ‘reasonable’ intelligence remains elusive.
In this talk, we explore and explain their underlying architectures and capabilities to understand what makes them work, their weaknesses, and future improvements.
Design principles from a technology and human perspective will be disseminated with examples of current production systems and their impact. Furthermore, the audience will have the opportunity to advance these best practices.
Resources will be made available, so the technology is relevant, practical, and accessible.
Commuter solutions_ElyseMarr
This document discusses ways to help employees reduce stress during their commutes. It first provides statistics on how commuting impacts employee health. It then outlines a two-part approach to implementing a "Calming Commuter's Club" that 1) disseminates methods and tracks impacts, and 2) provides calming technologies and practices for commutes via various modes of transportation. Existing commuter benefit programs and platforms that employers can use are also reviewed, as well as mindfulness techniques and existing calming solutions that could help destress commuters.
[Europe merge world tour] Perforce Server Update
This document outlines Perforce's roadmap to improve its distributed development capabilities, called "100X". Current solutions like filtered replication and task streams will be enhanced. The "100X" initiative aims to optimize failover, reduce network load, allow horizontal scaling, and improve concurrency. It includes advanced replication techniques like proxy replicas, filtered/chained commits, and Git replication. The roadmap's goals for 2014 are horizontal scaling of read operations, high availability and failover, and improved replication throughput. These changes will make remote sites faster and allow better support of large, distributed teams.
White Paper: Release This! - Tools for a Smooth Release Cycle
This document discusses an alternative branching strategy used by Guerrilla Games where developers work on the main branch and changes are selectively integrated to release branches. Key points:
- Developers work on main branch while a release branch is used for testing and bug fixes. Changes are integrated from main to release under control of release managers.
- Tools were created to track changes, show dependencies, and facilitate integrations to improve process scalability and visibility for all developers.
- The strategy provides more control over release cycles by decoupling development from releases and allowing targeted testing of features on release branches. Maintaining transparency of changes is important for its success.
[India Merge World Tour] Git Fusion
Git Fusion provides an enterprise version of Git that combines the flexibility and workflows of Git with the security, scalability, and access control of Perforce's versioning system. It allows developers to continue using Git while providing release managers, administrators, and teams collaboration capabilities for managing large or complex projects. Key features include global replication without limits, flexible workflows, consistent access control and security, and dynamic remapping of repositories to integrate teams and codebases.
Tendències Enotur, núm 2. Monogràfic sobre enoturisme
Tendències Enotur, núm 2. Monogràfic sobre enoturismeParque Científico y Tecnológico de Turismo y Ocio
rEl número 2 del butlletí 'Tendènces Enotur' és editat des del PCT de Turisme i Oci a Vila-seca per part del Consorci per a la Millora de la Competitivitat del Tuisme i l'Oci a les comarques de Tarragona, amb el cofinançament del FEDER Eix 1 i la Diputació de Tarragona. Recommended
WHOIS the Master
Presents a WHOIS database search engine tool I wrote to allow pentesters to access network information for specified targets. First presented at BSidesDE 2010
Artificial Assistants: How can I help you? by Christopher Currin
Chatbots are not equal; with different forms permeating our lives more and more. Virtual assistants are increasingly relevant for businesses and our day-to-day lives. Chatbots have become ubiquitous for interactions, yet ‘reasonable’ intelligence remains elusive.
In this talk, we explore and explain their underlying architectures and capabilities to understand what makes them work, their weaknesses, and future improvements.
Design principles from a technology and human perspective will be disseminated with examples of current production systems and their impact. Furthermore, the audience will have the opportunity to advance these best practices.
Resources will be made available, so the technology is relevant, practical, and accessible.
Commuter solutions_ElyseMarr
This document discusses ways to help employees reduce stress during their commutes. It first provides statistics on how commuting impacts employee health. It then outlines a two-part approach to implementing a "Calming Commuter's Club" that 1) disseminates methods and tracks impacts, and 2) provides calming technologies and practices for commutes via various modes of transportation. Existing commuter benefit programs and platforms that employers can use are also reviewed, as well as mindfulness techniques and existing calming solutions that could help destress commuters.
[Europe merge world tour] Perforce Server Update
This document outlines Perforce's roadmap to improve its distributed development capabilities, called "100X". Current solutions like filtered replication and task streams will be enhanced. The "100X" initiative aims to optimize failover, reduce network load, allow horizontal scaling, and improve concurrency. It includes advanced replication techniques like proxy replicas, filtered/chained commits, and Git replication. The roadmap's goals for 2014 are horizontal scaling of read operations, high availability and failover, and improved replication throughput. These changes will make remote sites faster and allow better support of large, distributed teams.
White Paper: Release This! - Tools for a Smooth Release Cycle
This document discusses an alternative branching strategy used by Guerrilla Games where developers work on the main branch and changes are selectively integrated to release branches. Key points:
- Developers work on main branch while a release branch is used for testing and bug fixes. Changes are integrated from main to release under control of release managers.
- Tools were created to track changes, show dependencies, and facilitate integrations to improve process scalability and visibility for all developers.
- The strategy provides more control over release cycles by decoupling development from releases and allowing targeted testing of features on release branches. Maintaining transparency of changes is important for its success.
[India Merge World Tour] Git Fusion
Git Fusion provides an enterprise version of Git that combines the flexibility and workflows of Git with the security, scalability, and access control of Perforce's versioning system. It allows developers to continue using Git while providing release managers, administrators, and teams collaboration capabilities for managing large or complex projects. Key features include global replication without limits, flexible workflows, consistent access control and security, and dynamic remapping of repositories to integrate teams and codebases.
Tendències Enotur, núm 2. Monogràfic sobre enoturisme
Tendències Enotur, núm 2. Monogràfic sobre enoturismeParque Científico y Tecnológico de Turismo y Ocio
rEl número 2 del butlletí 'Tendènces Enotur' és editat des del PCT de Turisme i Oci a Vila-seca per part del Consorci per a la Millora de la Competitivitat del Tuisme i l'Oci a les comarques de Tarragona, amb el cofinançament del FEDER Eix 1 i la Diputació de Tarragona. Protecting privacy with fuzzy-feeling test data
Testing systems that contain personally identifying information (PII) can be tricky; we need to be very careful not to allow into our test systems, but we also need production-like test data. In this presentation, I outline a system I wrote for protecting user privacy by removing PII from production data automatically.
Logging for hackers SAINTCON
SAINTCON
LOG-MD
LOG-MD.com
Malware Archaeology
MalwareArchaeology
What you need to know to catch hackers using proper Windows logging
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Video (at YouTube) - http://bit.ly/19TNSTF
Big Data Security Analytics, Data Science and Machine Learning are a few of the new buzzwords that have invaded out industry of late. Most of what we hear are promises of an unicorn-laden, silver-bullet panacea by heavy-handed marketing folks, evoking an expected pushback from the most enlightened members of our community.
This talk will help parse what we as a community need to know and understand about these concepts and help understand where the technical details and actual capabilities of those concepts and also where they fail and how they can be exploited and fooled by an attacker.
The talk will also share results of the author's current ongoing research (on MLSec Project) of applying machine learning techniques to information secuirty monitoring.
Logging for Hackers - What you need to know to catch them
This document discusses how to detect malware using existing Windows logs. It begins with an introduction to the presenter and his tools for malware analysis and log management. Several examples of malware artifacts and their associated log entries are described. The document emphasizes that command line logging is critical for detection. It also introduces the free, open-source Log-MD tool which analyzes log settings and harvests security events to help identify infected systems.
NotaCon 2011 - Networking for Pentesters
This document provides introductions and background information for two presenters, JP Bourget and Rob Fuller, for a presentation on networking for pentesters. JP Bourget has a background in IT, computer security and information assurance. He currently works in network and security management and also teaches networking and security classes. Rob Fuller has 10 years experience in information security and IT, including work as a network attack operator and penetration tester. The presentation agenda includes topics like networking, information operations, vulnerability hunting, exploitation, persistence and pivoting.
Logging for Hackers v1.0
A look at the types malicious artifacts from Advanced and Commodity attacks, what unique artifacts to look for and how logging caught them for a Windows environment and how LOG-MD can help.
MalwareArchaeology.com
LOG-MD.com
Library Linked Data
This document discusses the Digital Public Library of America (DPLA) and linked library data. It begins by asking questions about what the DPLA is, where its materials and metadata are coming from, and what problems it may encounter. It then discusses that libraries have metadata in many forms beyond catalogs and that standards need to account for computers' abilities while allowing flexibility. Unique identifiers, controlled vocabularies, and machine-readable data are important. The document proposes several ways to connect library data, such as metadata standards or metasearch, and discusses issues with each. It introduces linked data using URIs, RDF triples, and vocabularies as a way to integrate data while allowing different implementations.
Version Your Cloud: Using Perforce to Manage Your Object Storage
Jeppesen uses a combination of their own data centers and a variety of 3rd party cloud providers to support their distribution of navigation data to most of the world’s pilots. But before any data leaves the building it goes through Perforce. Learn why Jeppesen chose to put Perforce at the heart of their global data distribution system, and how you too could leverage Perforce to help provide atomic changesets and true history tracking to your cloud-scale object stores.
Introduction to Malware Analysis
This document provides an overview of malware analysis, including both static and dynamic analysis techniques. Static analysis involves examining a file's code and components without executing it, such as identifying file types, checking hashes, and viewing strings. Dynamic analysis involves executing the malware in a controlled environment and monitoring its behavior and any system changes. Dynamic analysis tools discussed include Process Explorer, Process Monitor, and Autoruns to track malware processes, files accessed, and persistence mechanisms. Both static and dynamic analysis are needed to fully understand malware behavior.
Windows IR made easier and faster v1.0
Windows IR made easier and faster Find the head of the snake using Logs, AutoRuns, Large Registry Keys, Locked Files, IP/WhoIs and Netflow
Malware Archaeology
LOG-MD
BSidesNOLA
C*ollege Credit: Is My App a Good Fit for Cassandra?
The document discusses factors to consider when determining if Cassandra is a good fit for an application. It outlines planning questions around data structure, query patterns, and infrastructure requirements. It then provides comparisons of Cassandra to other databases like MongoDB and Redis, highlighting advantages and disadvantages of each for different use cases. The document aims to help users strategically evaluate which database solution aligns best with their specific application needs.
ANALYZE'15 - Bulk Malware Analysis at Scale
This document discusses extracting malware configurations at scale. It describes the large number of new malware samples seen daily and challenges with keeping up through traditional analysis methods. The author proposes automating extraction of configurations from remote access trojans (RATs) to more efficiently gather intelligence. Tools are introduced that can statically extract configurations from over 30 RAT types. A system is designed to intake large volumes of samples, use these tools to extract configurations, and output the data to a database for further analysis. The goal is to gain more insight into actors through common configuration elements like command and control domains even as they change tools.
Defcon 22-wesley-mc grew-instrumenting-point-of-sale-malware
This document discusses improving the communication of malware analysis by supplementing written reports with demonstrations of the malware's capabilities. It presents a case study of instrumenting and demonstrating point-of-sale (POS) malware called JackPOS. Key points covered include setting up the malware's command and control infrastructure, instrumenting the malware's code to monitor its behavior, and demonstrating how it searches process memory for stolen credit card data. The goal is to make malware analysis more reproducible and educational for other analysts.
CNIT 152 11 Analysis Methodology
This document provides an overview of the incident response analysis methodology. It discusses establishing objectives, understanding the situation and resources needed. Leadership is important to define objectives and prevent miscommunication. The analysis should focus on answering realistic questions within the defined scope. All data sources like operating systems, applications, user data, and networks should be understood. Various analysis methods are described like reviewing anomalies, host artifacts, malware analysis, tools, and manual review. The results should be periodically evaluated for progress and completeness in answering questions.
When Security Tools Fail You
When your security tools fail you, and what you can do about it. This discusses actual tool fail backgrounds, what failed and what you can do to detect and/or mitigate the issues(s) another way
HackerHurricane
MalwareArchaeology
Malware Archaeology
LOG-MD
11 Analysis Methodology
This document provides an overview of the incident response analysis methodology process. It discusses defining objectives, understanding the situation and available resources, identifying leadership, avoiding impossible tasks like proving a negative, asking why to define scope, knowing where data is stored, accessing raw data, selecting analysis methods like searching for malware or using tools like VirusTotal, manual review, filtering data, statistical analysis using tools like Sawmill, string searching, analyzing unallocated space, and file carving. It stresses periodically evaluating results to ensure progress and only making definitive statements if supported by evidence.
Blazing Data With Redis (and LEGOS!)
There are many fast data stores, and then there is Redis. Learn about this excellent NoSQL solution that is a powerful in-memory key-value store. Learn how to solve traditionally difficult problems with Redis, and how you can benefit from 100,000 reads/writes a second on commodity hardware. We’ll discuss how and when to use the different datatypes and commands to fit your needs. We’ll discuss the different PHP libraries with their pros and cons. We’ll then show some live examples on how to use it for a chatroom, and how Redis manages a billion data points for our dating matching system. Finally, we’ll discuss some of the upcoming features in the near future, such as clustering and scripting.
Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)
Follow along with the R Markdown file at http://rpubs.com/alexcpsec/tiq-test-Summer2014-2
Source code available at:
https://github.com/mlsecproject/tiq-test
https://github.com/mlsecproject/tiq-test-Summer2014
https://github.com/mlsecproject/combine
---------
Full Abstract:
Threat Intelligence feeds are now being touted as the saving grace for SIEM and log management deployments, and as a way to supercharge incident detection and even response practices. We have heard similar promises before as an industry, so it is only fair to try to investigate. Since the actual number of breaches and attacks worldwide is unknown, it is impossible to measure how good threat intelligence feeds really are, right? Enter a new scientific breakthrough developed over the last 300 years: statistics!
This presentation will consist of a data-driven analysis of a cross-section of threat intelligence feeds (both open-source and commercial) to measure their statistical bias, overlap, and representability of the unknown population of breaches worldwide. Are they a statistical good measure of the population of "bad stuff" happening out there? Is there even such a thing? How tuned to your specific threat surface are those feeds anyway? Regardless, can we actually make good use of them even if the threats they describe have no overlap with the actual incidents you have been seeing in your environment?
We will provide an open-source tool for attendees to extract, normalize and export data from threat intelligence feeds to use in their internal projects and systems. It will be pre-configured with current OSINT network feed and easily extensible for private or commercial feeds. All the statistical code written and research data used (from the open-source feeds) will be made available in the spirit of reproducible research. The tool itself will be able to be used by attendees to perform the same type of tests on their own data.
Join Alex and Kyle on a journey through the actual real-world usability of threat intelligence to find out which mix of open source and private feeds are right for your organization.
Ask a Malware Archaeologist
The document discusses how to detect malware through effective logging. It recommends enabling command line logging for events like cmd.exe, powershell, and other processes to see details of malware behavior. The speaker advocates building a "malware matrix" of indicators and monitoring important log events. Effective logging of files, registry, network connections and other activities on internet-facing systems can help detect malware, as demonstrated by the speaker's analysis of real world advanced persistent threats. Logs are crucial for both incident response and prevention when properly configured.
EDR, ETDR, Next Gen AV is all the rage, so why am I ENRAGED?
This document discusses the presenter's testing of various EDR and EPP solutions using three malware samples. Key findings include:
1) Many solutions failed to detect infections, even those detected by the presenter's IPS. Detection was weakest for "fileless" Kovter and morphing Dridex malware.
2) Solutions provided inadequate details to fully remediate infections. The presenter's own LOG-MD tool outperformed EDR solutions in revealing infection artifacts.
3) Based on the results, the presenter recommends that EDR tools integrate capabilities to remotely run third-party tools like LOG-MD for more thorough investigations. Simpler consoles are also needed to distribute workload across security
How to Organize Game Developers With Different Planning Needs
Different skills have different needs when it comes to planning. For a coder it may make perfect sense to plan work in two-week sprints, but for an artist, an asset may take longer than two weeks to complete.
How do you allow different skills to plan the way that works best for them? Some studios may choose to open up for flexibility – do whatever you like! But that tends to cause issues with alignment and siloes of data, resulting in loss of vision. Lost vision in the sense that it is difficult to understand, but also — and maybe more importantly — the risk of losing the vision of what the game will be.
With the right approach, however, you can avoid these obstacles. Join backlog expert Johan Karlsson to learn:
-The balance of team autonomy and alignment.
-How to use the product backlog to align the project vision.
-How to use tools to support the flexibility you need.
Looking for a planning and backlog tool? You can try Hansoft for free.
Regulatory Traceability: How to Maintain Compliance, Quality, and Cost Effic...
How do regulations impact your product requirements? How do you ensure that you identify all the needed requirements changes to meet these regulations?
Ideally, your regulations should live alongside your product requirements, so you can trace among each related item. Getting to that point can be quite an undertaking, however. Ultimately you want a process that:
-Saves money
-Ensures quality
-Avoids fines
If you want help achieving these goals, this webinar is for you. Watch Tom Totenberg, Senior Solutions Engineer for Helix ALM, show you:
-How to import a regulation document into Helix ALM.
-How to link to requirements.
-How to automate impact analysis from regulatory updates.
More Related Content
Similar to Who Watches the Watchers?
Protecting privacy with fuzzy-feeling test data
Testing systems that contain personally identifying information (PII) can be tricky; we need to be very careful not to allow into our test systems, but we also need production-like test data. In this presentation, I outline a system I wrote for protecting user privacy by removing PII from production data automatically.
Logging for hackers SAINTCON
SAINTCON
LOG-MD
LOG-MD.com
Malware Archaeology
MalwareArchaeology
What you need to know to catch hackers using proper Windows logging
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Video (at YouTube) - http://bit.ly/19TNSTF
Big Data Security Analytics, Data Science and Machine Learning are a few of the new buzzwords that have invaded out industry of late. Most of what we hear are promises of an unicorn-laden, silver-bullet panacea by heavy-handed marketing folks, evoking an expected pushback from the most enlightened members of our community.
This talk will help parse what we as a community need to know and understand about these concepts and help understand where the technical details and actual capabilities of those concepts and also where they fail and how they can be exploited and fooled by an attacker.
The talk will also share results of the author's current ongoing research (on MLSec Project) of applying machine learning techniques to information secuirty monitoring.
Logging for Hackers - What you need to know to catch them
This document discusses how to detect malware using existing Windows logs. It begins with an introduction to the presenter and his tools for malware analysis and log management. Several examples of malware artifacts and their associated log entries are described. The document emphasizes that command line logging is critical for detection. It also introduces the free, open-source Log-MD tool which analyzes log settings and harvests security events to help identify infected systems.
NotaCon 2011 - Networking for Pentesters
This document provides introductions and background information for two presenters, JP Bourget and Rob Fuller, for a presentation on networking for pentesters. JP Bourget has a background in IT, computer security and information assurance. He currently works in network and security management and also teaches networking and security classes. Rob Fuller has 10 years experience in information security and IT, including work as a network attack operator and penetration tester. The presentation agenda includes topics like networking, information operations, vulnerability hunting, exploitation, persistence and pivoting.
Logging for Hackers v1.0
A look at the types malicious artifacts from Advanced and Commodity attacks, what unique artifacts to look for and how logging caught them for a Windows environment and how LOG-MD can help.
MalwareArchaeology.com
LOG-MD.com
Library Linked Data
This document discusses the Digital Public Library of America (DPLA) and linked library data. It begins by asking questions about what the DPLA is, where its materials and metadata are coming from, and what problems it may encounter. It then discusses that libraries have metadata in many forms beyond catalogs and that standards need to account for computers' abilities while allowing flexibility. Unique identifiers, controlled vocabularies, and machine-readable data are important. The document proposes several ways to connect library data, such as metadata standards or metasearch, and discusses issues with each. It introduces linked data using URIs, RDF triples, and vocabularies as a way to integrate data while allowing different implementations.
Version Your Cloud: Using Perforce to Manage Your Object Storage
Jeppesen uses a combination of their own data centers and a variety of 3rd party cloud providers to support their distribution of navigation data to most of the world’s pilots. But before any data leaves the building it goes through Perforce. Learn why Jeppesen chose to put Perforce at the heart of their global data distribution system, and how you too could leverage Perforce to help provide atomic changesets and true history tracking to your cloud-scale object stores.
Introduction to Malware Analysis
This document provides an overview of malware analysis, including both static and dynamic analysis techniques. Static analysis involves examining a file's code and components without executing it, such as identifying file types, checking hashes, and viewing strings. Dynamic analysis involves executing the malware in a controlled environment and monitoring its behavior and any system changes. Dynamic analysis tools discussed include Process Explorer, Process Monitor, and Autoruns to track malware processes, files accessed, and persistence mechanisms. Both static and dynamic analysis are needed to fully understand malware behavior.
Windows IR made easier and faster v1.0
Windows IR made easier and faster Find the head of the snake using Logs, AutoRuns, Large Registry Keys, Locked Files, IP/WhoIs and Netflow
Malware Archaeology
LOG-MD
BSidesNOLA
C*ollege Credit: Is My App a Good Fit for Cassandra?
The document discusses factors to consider when determining if Cassandra is a good fit for an application. It outlines planning questions around data structure, query patterns, and infrastructure requirements. It then provides comparisons of Cassandra to other databases like MongoDB and Redis, highlighting advantages and disadvantages of each for different use cases. The document aims to help users strategically evaluate which database solution aligns best with their specific application needs.
ANALYZE'15 - Bulk Malware Analysis at Scale
This document discusses extracting malware configurations at scale. It describes the large number of new malware samples seen daily and challenges with keeping up through traditional analysis methods. The author proposes automating extraction of configurations from remote access trojans (RATs) to more efficiently gather intelligence. Tools are introduced that can statically extract configurations from over 30 RAT types. A system is designed to intake large volumes of samples, use these tools to extract configurations, and output the data to a database for further analysis. The goal is to gain more insight into actors through common configuration elements like command and control domains even as they change tools.
Defcon 22-wesley-mc grew-instrumenting-point-of-sale-malware
This document discusses improving the communication of malware analysis by supplementing written reports with demonstrations of the malware's capabilities. It presents a case study of instrumenting and demonstrating point-of-sale (POS) malware called JackPOS. Key points covered include setting up the malware's command and control infrastructure, instrumenting the malware's code to monitor its behavior, and demonstrating how it searches process memory for stolen credit card data. The goal is to make malware analysis more reproducible and educational for other analysts.
CNIT 152 11 Analysis Methodology
This document provides an overview of the incident response analysis methodology. It discusses establishing objectives, understanding the situation and resources needed. Leadership is important to define objectives and prevent miscommunication. The analysis should focus on answering realistic questions within the defined scope. All data sources like operating systems, applications, user data, and networks should be understood. Various analysis methods are described like reviewing anomalies, host artifacts, malware analysis, tools, and manual review. The results should be periodically evaluated for progress and completeness in answering questions.
When Security Tools Fail You
When your security tools fail you, and what you can do about it. This discusses actual tool fail backgrounds, what failed and what you can do to detect and/or mitigate the issues(s) another way
HackerHurricane
MalwareArchaeology
Malware Archaeology
LOG-MD
11 Analysis Methodology
This document provides an overview of the incident response analysis methodology process. It discusses defining objectives, understanding the situation and available resources, identifying leadership, avoiding impossible tasks like proving a negative, asking why to define scope, knowing where data is stored, accessing raw data, selecting analysis methods like searching for malware or using tools like VirusTotal, manual review, filtering data, statistical analysis using tools like Sawmill, string searching, analyzing unallocated space, and file carving. It stresses periodically evaluating results to ensure progress and only making definitive statements if supported by evidence.
Blazing Data With Redis (and LEGOS!)
There are many fast data stores, and then there is Redis. Learn about this excellent NoSQL solution that is a powerful in-memory key-value store. Learn how to solve traditionally difficult problems with Redis, and how you can benefit from 100,000 reads/writes a second on commodity hardware. We’ll discuss how and when to use the different datatypes and commands to fit your needs. We’ll discuss the different PHP libraries with their pros and cons. We’ll then show some live examples on how to use it for a chatroom, and how Redis manages a billion data points for our dating matching system. Finally, we’ll discuss some of the upcoming features in the near future, such as clustering and scripting.
Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)
Follow along with the R Markdown file at http://rpubs.com/alexcpsec/tiq-test-Summer2014-2
Source code available at:
https://github.com/mlsecproject/tiq-test
https://github.com/mlsecproject/tiq-test-Summer2014
https://github.com/mlsecproject/combine
---------
Full Abstract:
Threat Intelligence feeds are now being touted as the saving grace for SIEM and log management deployments, and as a way to supercharge incident detection and even response practices. We have heard similar promises before as an industry, so it is only fair to try to investigate. Since the actual number of breaches and attacks worldwide is unknown, it is impossible to measure how good threat intelligence feeds really are, right? Enter a new scientific breakthrough developed over the last 300 years: statistics!
This presentation will consist of a data-driven analysis of a cross-section of threat intelligence feeds (both open-source and commercial) to measure their statistical bias, overlap, and representability of the unknown population of breaches worldwide. Are they a statistical good measure of the population of "bad stuff" happening out there? Is there even such a thing? How tuned to your specific threat surface are those feeds anyway? Regardless, can we actually make good use of them even if the threats they describe have no overlap with the actual incidents you have been seeing in your environment?
We will provide an open-source tool for attendees to extract, normalize and export data from threat intelligence feeds to use in their internal projects and systems. It will be pre-configured with current OSINT network feed and easily extensible for private or commercial feeds. All the statistical code written and research data used (from the open-source feeds) will be made available in the spirit of reproducible research. The tool itself will be able to be used by attendees to perform the same type of tests on their own data.
Join Alex and Kyle on a journey through the actual real-world usability of threat intelligence to find out which mix of open source and private feeds are right for your organization.
Ask a Malware Archaeologist
The document discusses how to detect malware through effective logging. It recommends enabling command line logging for events like cmd.exe, powershell, and other processes to see details of malware behavior. The speaker advocates building a "malware matrix" of indicators and monitoring important log events. Effective logging of files, registry, network connections and other activities on internet-facing systems can help detect malware, as demonstrated by the speaker's analysis of real world advanced persistent threats. Logs are crucial for both incident response and prevention when properly configured.
EDR, ETDR, Next Gen AV is all the rage, so why am I ENRAGED?
This document discusses the presenter's testing of various EDR and EPP solutions using three malware samples. Key findings include:
1) Many solutions failed to detect infections, even those detected by the presenter's IPS. Detection was weakest for "fileless" Kovter and morphing Dridex malware.
2) Solutions provided inadequate details to fully remediate infections. The presenter's own LOG-MD tool outperformed EDR solutions in revealing infection artifacts.
3) Based on the results, the presenter recommends that EDR tools integrate capabilities to remotely run third-party tools like LOG-MD for more thorough investigations. Simpler consoles are also needed to distribute workload across security
Similar to Who Watches the Watchers? (20)
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Logging for Hackers - What you need to know to catch them
Logging for Hackers - What you need to know to catch them
Version Your Cloud: Using Perforce to Manage Your Object Storage
Version Your Cloud: Using Perforce to Manage Your Object Storage
C*ollege Credit: Is My App a Good Fit for Cassandra?
C*ollege Credit: Is My App a Good Fit for Cassandra?
Defcon 22-wesley-mc grew-instrumenting-point-of-sale-malware
Defcon 22-wesley-mc grew-instrumenting-point-of-sale-malware
Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)
Measuring the IQ of your Threat Intelligence Feeds (#tiqtest)
EDR, ETDR, Next Gen AV is all the rage, so why am I ENRAGED?
EDR, ETDR, Next Gen AV is all the rage, so why am I ENRAGED?
More from Perforce
How to Organize Game Developers With Different Planning Needs
Different skills have different needs when it comes to planning. For a coder it may make perfect sense to plan work in two-week sprints, but for an artist, an asset may take longer than two weeks to complete.
How do you allow different skills to plan the way that works best for them? Some studios may choose to open up for flexibility – do whatever you like! But that tends to cause issues with alignment and siloes of data, resulting in loss of vision. Lost vision in the sense that it is difficult to understand, but also — and maybe more importantly — the risk of losing the vision of what the game will be.
With the right approach, however, you can avoid these obstacles. Join backlog expert Johan Karlsson to learn:
-The balance of team autonomy and alignment.
-How to use the product backlog to align the project vision.
-How to use tools to support the flexibility you need.
Looking for a planning and backlog tool? You can try Hansoft for free.
Regulatory Traceability: How to Maintain Compliance, Quality, and Cost Effic...
How do regulations impact your product requirements? How do you ensure that you identify all the needed requirements changes to meet these regulations?
Ideally, your regulations should live alongside your product requirements, so you can trace among each related item. Getting to that point can be quite an undertaking, however. Ultimately you want a process that:
-Saves money
-Ensures quality
-Avoids fines
If you want help achieving these goals, this webinar is for you. Watch Tom Totenberg, Senior Solutions Engineer for Helix ALM, show you:
-How to import a regulation document into Helix ALM.
-How to link to requirements.
-How to automate impact analysis from regulatory updates.
Efficient Security Development and Testing Using Dynamic and Static Code Anal...
Be sure to register for a demo, if you would like to see how Klocwork can help ensure that your code is secure, reliable, and compliant.
https://www.perforce.com/products/klocwork/live-demo
Understanding Compliant Workflow Enforcement SOPs
If it’s not documented, it didn’t happen.
When it comes to compliance, if you’re doing the work, you need to prove it. That means having well-documented SOPs (standard operating procedures) in place for all your regulated workflows.
It also means logging your efforts to enforce these SOPs. They show that you took appropriate action in any number of scenarios, which can be related to regulations, change requests, firing of an employee, logging an HR compliant, or anything else that needs a structured workflow.
But when do you need to do this, and how do you go about it?
In this webinar, Tom Totenberg, our Helix ALM senior solutions engineer, clarifies workflow enforcement SOPs, along with a walkthrough of how Perforce manages GDPR (General Data Protection Regulation) requests. He’ll cover:
-What are SOPs?
-Why is it important to have this documentation?
-Example: walking through our internal Perforce GDPR process.
-What to beware of.
-Building the workflow in ALM.
Branching Out: How To Automate Your Development Process
If you could ship 20% faster, what would it mean for your business? What could you build? Better question, what’s slowing your teams down?
Teams struggle to manage branching and merging. For bigger teams and projects, it gets even more complex. Tracking development using a flowchart, team wiki, or a white board is ineffective. And attempts to automate with complex scripting are costly to maintain.
Remove the bottlenecks and automate your development your way with Perforce Streams –– the flexible branching model in Helix Core.
Join Brad Hart, Chief Technology Officer and Brent Schiestl, Senior Product Manager for Perforce version control to learn how Streams can:
-Automate and customize development and release processes.
-Easily track and propagate changes across teams.
-Boost end user efficiency while reducing errors and conflicts.
-Support multiple teams, parallel releases, component-based development, and more.
How to Do Code Reviews at Massive Scale For DevOps
Code review is a critical part of your build process. And when you do code review right, you can streamline your build process and achieve DevOps.
Most code review tools work great when you have a team of 10 developers. But what happens when you need to scale code review to 1,000s of developers? Many will struggle. But you don’t need to.
Join our experts Johan Karlsson and Robert Cowham for a 30-minute webinar. You’ll learn:
-The problems with scaling code review from 10s to 100s to 1,000s of developers along with other dimensions of scale (files, reviews, size).
-The solutions for dealing with all dimensions of scale.
-How to utilize Helix Swarm at massive scale.
Ready to scale code review and streamline your build process? Get started with Helix Swarm, a code review tool for Helix Core.
How to Spark Joy In Your Product Backlog
By now many of us have had plenty of time to clean and tidy up our homes. But have you given your product backlog and task tracking software as much attention?
To keep your digital tools organized, it is important to avoid hoarding on to inefficient processes. By removing the clutter in your product backlog, you can keep your teams focused.
It’s time to spark joy by cleaning up your planning tools!
Join Johan Karlsson — our Agile and backlog expert — to learn how to:
-Apply digital minimalism to your tracking and planning.
-Organize your work by category.
-Motivate teams by transitioning to a cleaner way of working.
TRY HANSOFT FREE
Going Remote: Build Up Your Game Dev Team
Everyone’s working remote as a result of the coronavirus (COVID-19). And while game development has always been done with remote teams, there’s a new challenge facing the industry.
Your audience has always been mostly at home – now they may be stuck there. And they want more games to stay happy and entertained.
So, how can you enable your developers to get files and feedback faster to meet this rapidly growing demand?
In this webinar, you’ll learn:
-How to meet the increasing demand.
-Ways to empower your remote teams to build faster.
-Why Helix Core is the best way to maximize productivity.
Plus, we’ll share our favorite games keeping us happy in the midst of a pandemic.
Shift to Remote: How to Manage Your New Workflow
The spread of coronavirus has fundamentally changed the way people work. Companies around the globe are making an abrupt shift in how they manage projects and teams to support their newly remote workers.
Organizing suddenly distributed teams means restructuring more than a standup. To facilitate this transition, teams need to update how they collaborate, manage workloads, and maintain projects.
At Perforce, we are here to help you maintain productivity. Join Johan Karlsson — our Agile expert — to learn how to:
Keep communication predictable and consistent.
-Increase visibility across teams.
-Organize projects, sprints, Kanban boards and more.
-Empower and support your remote workforce.
Hybrid Development Methodology in a Regulated World
In a regulated industry, collaboration can be vital to building quality products that meet compliance. But when an Agile team and a Waterfall team need to work together, it can feel like mixing oil with water.
If you're used to Agile methods, Waterfall can feel slow and unresponsive. From a Waterfall perspective, pure Agile may lack accountability and direction. Misaligned teams can slow progress, and expose your development to mistakes that undermine compliance.
It's possible to create the best of both worlds so your teams can operate together harmoniously. This is how to develop products quickly, and still make regulators happy.
Join ALM Solutions Engineer Tom Totenberg in this webinar to learn how teams can:
- Operate efficiently with differing methodologies.
- Glean best practices for their tailored hybrid.
- Work together in a single environment.
Watch the webinar, and when you're ready for a tool to help you with the hybrid, know that you can try Helix ALM for free.
Better, Faster, Easier: How to Make Git Really Work in the Enterprise
There's a lot of reasons to love Git. (Git is awesome at what it does.) Let’s look at the 3 major use cases for Git in the enterprise:
1. You work with third party or outsourced development teams.
2. You use open source in your products.
3. You have different workflow needs for different teams.
Making the best of Git can be difficult in an enterprise environment. Trying to manage all the moving parts is like herding cats.
So, how do you optimize your teams’ use of Git — and make it all fit into your vision of the enterprise SDLC?
You’ll learn about:
-The challenges that accompany each use case — third parties, open source code, different workflows.
-Ways to solve these problems.
-How to make Git better, faster, and easier — with Perforce
Easier Requirements Management Using Diagrams In Helix ALM
Sometimes requirements need visuals. Whether it’s a diagram that clarifies an idea or a screenshot to capture information, images can help you manage requirements more efficiently. And that means better quality products shipped faster.
In this webinar, Helix ALM Professional Services Consultant Gerhard Krüger will demonstrate how to use visuals in ALM to improve requirements. Learn how to:
-Share information faster than ever.
-Drag and drop your way to better teamwork.
-Integrate various types of visuals into your requirements.
-Utilize diagram and flowchart software for every need.
-And more!
Immediately apply the information in this webinar for even better requirements management using Helix ALM.
How To Master Your Mega Backlog
It’s common practice to keep a product backlog as small as possible, probably just 10-20 items. This works for single teams with one Product Owner and perhaps a Scrum Master.
But what if you have 100 Scrum teams managing a complex system of hardware and software components? What do you need to change to manage at such a massive scale?
Join backlog expert Johan Karlsson to learn how to:
-Adapt Agile product backlog practices to manage many backlogs.
-Enhance collaboration across disciplines.
-Leverage backlogs to align teams while giving them flexibility.
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...
In Part 3, we will look at what the future might hold for embedded programming languages and development tools. And, we will look at the future for software safety and security standards.
How to Scale With Helix Core and Microsoft Azure
This document discusses how to scale Helix Core using Microsoft Azure. It begins by explaining the benefits of using Helix Core and Azure together, such as high performance, scalability, security integration, and availability. It then covers computing, storage, and security options on Azure, including virtual machine types and operating system choices. Next, it describes how to set up global deployments with Helix Core on Azure using techniques like proxies, replicas, and the Perforce federated architecture. It concludes with examples of advanced topologies like build servers, hybrid cloud/on-premises implementations, and multi-cloud considerations.
Achieving Software Safety, Security, and Reliability Part 2
In Part 2, we will focus on the automotive industry, as it leads the way in enforcing safety, security, and reliability standards as well as best practices for software development. We will then examine how other industries could adopt similar practices.
Should You Break Up With Your Monolith?
Modernizing an application’s architecture is often a necessary multi-year project in the making. The goal –– to stabilize code, detangle dependencies, and adopt a toolset that ignites innovation.
Moving your monolith repository to a microservices/component based development model might be on trend. But is it right for you?
Before you break up with anything, it is vital to assess your needs and existing environment to construct the right plan. This can minimize business risks and maximize your development potential.
Join Tom Tyler and Chuck Gehman to learn more about:
-Why you need to plan your move with the right approach.
-How to reduce risk when refactoring your monolithic repository.
-What you need to consider before migrating code.
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
In part one of our three-part webinar series, we examine common software development challenges, review the safety and security standards adopted by different industries, and examine the best practices that can be applied to any software development team.
What's New in Helix ALM 2019.4
The features you’ve been waiting for! Helix ALM’s latest update expands usability and functionality to bring solid improvements to your processes.
Watch Helix ALM Senior Product Manager Paula Rome demonstrate how new features:
-Simplify workflows.
-Expand report analysis.
-Boost productivity in the Helix ALM web client.
All this and MORE packed into an exciting 30 minutes! Get inspired. Be extraordinary with the new Helix ALM.
Free Yourself From the MS Office Prison
Companies that track requirements, create traceability matrices, and complete audits - especially for compliance - run into many problems using only Word and Excel to accomplish these tasks.
Most notably, manual processes leave employees vulnerable to making costly mistakes and wasting valuable time.
These outdated tracking procedures rob organizations of benefiting from four keys to productivity and efficiency:
-Automation
-Collaboration
-Visibility
-Traceability
However, modern application lifecycle management (ALM) tools solve all of these problems, linking and organizing information into a single source of truth that is instantly auditable.
Gerhard Krüger, senior consultant for Helix ALM, explains how the right software supports these fundamentals, generating improvements that save time and money.
More from Perforce (20)
How to Organize Game Developers With Different Planning Needs
How to Organize Game Developers With Different Planning Needs
Regulatory Traceability: How to Maintain Compliance, Quality, and Cost Effic...
Regulatory Traceability: How to Maintain Compliance, Quality, and Cost Effic...
Efficient Security Development and Testing Using Dynamic and Static Code Anal...
Efficient Security Development and Testing Using Dynamic and Static Code Anal...
Branching Out: How To Automate Your Development Process
Branching Out: How To Automate Your Development Process
How to Do Code Reviews at Massive Scale For DevOps
How to Do Code Reviews at Massive Scale For DevOps
Hybrid Development Methodology in a Regulated World
Hybrid Development Methodology in a Regulated World
Better, Faster, Easier: How to Make Git Really Work in the Enterprise
Better, Faster, Easier: How to Make Git Really Work in the Enterprise
Easier Requirements Management Using Diagrams In Helix ALM
Easier Requirements Management Using Diagrams In Helix ALM
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...
Achieving Software Safety, Security, and Reliability Part 3: What Does the Fu...
Achieving Software Safety, Security, and Reliability Part 2
Achieving Software Safety, Security, and Reliability Part 2
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
Achieving Software Safety, Security, and Reliability Part 1: Common Industry ...
Recently uploaded
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
Programming Foundation Models with DSPy - Meetup Slides
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
HCL Notes and Domino License Cost Reduction in the World of DLAU
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Essentials of Automations: The Art of Triggers and Actions in FME
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
How to Get CNIC Information System with Paksim Ga.pptx
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Building Production Ready Search Pipelines with Spark and Milvus
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Recently uploaded (20)
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Who Watches the Watchers?
- 1. # Mark Harrison Tech Lead, Asset Management Pixar
- 2. #
- 3. #
- 4. #
- 5. # • Data • Metadata (file sizes, checksums, owners, etc)
- 6. # • Very Good Track Record • P4 verify • P4 checkpoint • No Known Undetected Failures!
- 7. # • What if it does mess up? – Undetected error? – Metadata goes odd? • It’s your job on the line!
- 8. # • People are Great! • People hardly ever make mistakes! • People are always looking out for your best interests!
- 9. # • Some People are Bad! • Some, just incompetent! • Some, both! • How can you tell? It doesn’t matter! • Intentional Data/Metadata Corruption – Bad guy made a src code mod • Example, modded code, blame on msundy – Crazy man put his picture in a film
- 10. # • P4 Verify? – Good but not good enough – Intentional hacking – Plain old Bug
- 11. # • “Who shall guard the self-same guardians?” • Need to have two systems that cross-check and validate • Each system serves as a check on the other – Catches (we hope!!) bugs and other oddities – Catches (we hope!!) intentional data modification • Is two enough? – Down that path lies insanity
- 12. # • We mirror all Perforce metadata (including checksums, sizes) in the Templar database • Perforce is still the “system of record” • But, we can see if the system of record ever changes • A Bad Guy will have to subvert both Perforce and Templar to do his evil deeds
- 13. # • Templar database initially mirrored Perforce data and metadata for “offline verify” • Basically a “p4 verify” that bypassed the P4D – Didn’t cause a slowdown • Takeaway: – Have a good abstraction, it can be used in multiple places
- 14. # SQL> desc p4_files • ID generic asset id (234363343) • P4PATH p4 path ("//ts3/myfile.jpg") • HEADVER the head revision (3) • REPOS repository name ("ts3") SQL> desc p4_versions • ID generic asset id (234363343) • VERSIONOF this revision is a version of file x • P4PATH p4 path ("//ts3/myfile.jpg") • VERSION version number ("3") • ISDELETED true if deleted version • REPOS repository name ("ts3”)
- 15. # SQL> desc asset_metadata • ID generic asset id (234363343) • md5sum checksum as per P4 • filesize filesize as per P4 • etc… • For each asset ID, we store the metadata we care about • Not just P4, but other asset types as well • All keyed by asset ID • Updates are by P4 checkin trigger
- 16. # • So far, things are good • Never detected a P4 error • Never detected an intentional modification • (Maybe the bad guys are just that good!) • But we can correlate our results with the Backup System Metadata • All Systems Agree!!
- 17. # • Good to know that our systems are working • Good to know that our colleagues seem trustworthy. • We don’t have to trust each other to like each other.
- 18. # Mark Harrison mh@pixar.com
- 19. # Mark is in charge of Pixar’s Data Management Group where he has a 50 year charter to store all data and metadata related to the Studio's feature films and related work. Prior to that he lived in China and was the Chief Software Architect of the China Internet. His software supported the growth from 200K users to the current base of 350M users. He studied Computer Science, worked in Texas, and wrote a couple of computer books. (Speaker photo here)
- 20. # RESOURCES CD Report: info.perforce.com/cd-report White Paper: perforce.com/white-paper-link Template: perforce.com/template