Git-annex allows users to manage files with git without checking file contents into the repository. It uses git for version control and file tracking, but stores file contents separately. File contents can be distributed across multiple drives, servers, and locations for redundancy and access from different devices. Git-annex also supports various workflows like archiving data, consuming media, and syncing files while traveling.
Advanced Persistent Threat (APT) attacks are highly organised and are launched for prolonged periods. APT attacks exhibit discernible attributes or patterns.
Introduction to the rapid prototyping with python and linux for embedded systemsNaohiko Shimizu
Short seminar for engineers who want to get the knowledge of embedded system with Linux. This slide uses RaspberryPi2 and Python for the laboratory. Make your board and try with this slide.
All of the sample codes and cross compiling environment can be downloaded from IP ARCH, Inc., my company. You can also download LiveCygwin which is ready made package just extract and do, based on Cygwin.
Requirement:
Windwos based PC with Ethernet connector. We will use peer to peer connection between PC and Raspberry Pi.
Appropriate right to control your PC to turn on the network share.
RaspberryPi B+ or RaspberryPi2 B
An ethernet cable for connect between PC and Pi
A micro USB cable for the Pi's power
A laboratory board which schematics is in the slide. You may make it with bread board.
Your effort :-)
Advanced Persistent Threat (APT) attacks are highly organised and are launched for prolonged periods. APT attacks exhibit discernible attributes or patterns.
Introduction to the rapid prototyping with python and linux for embedded systemsNaohiko Shimizu
Short seminar for engineers who want to get the knowledge of embedded system with Linux. This slide uses RaspberryPi2 and Python for the laboratory. Make your board and try with this slide.
All of the sample codes and cross compiling environment can be downloaded from IP ARCH, Inc., my company. You can also download LiveCygwin which is ready made package just extract and do, based on Cygwin.
Requirement:
Windwos based PC with Ethernet connector. We will use peer to peer connection between PC and Raspberry Pi.
Appropriate right to control your PC to turn on the network share.
RaspberryPi B+ or RaspberryPi2 B
An ethernet cable for connect between PC and Pi
A micro USB cable for the Pi's power
A laboratory board which schematics is in the slide. You may make it with bread board.
Your effort :-)
All the content of this website is informative and non-commercial, does not imply a commitment to develop, launch or schedule delivery of any feature or functionality, should not rely on it in making decisions, incorporate or take it as a reference in a contract or academic matters. Likewise, the use, distribution and reproduction by any means, in whole or in part, without the authorization of the author and / or third-party copyright holders, as applicable, is prohibited.
HDFS tiered storage: mounting object stores in HDFSDataWorks Summit
Most users know HDFS as the reliable store of record for big data analytics. HDFS is also used to store transient and operational data when working with cloud object stores, such as Microsoft Azure, and on-premises object stores, such as Western Digital’s ActiveScale. In these settings, applications often manage data stored in multiple storage systems or clusters, requiring a complex workflow for synchronizing data between filesystems for business continuity planning (BCP) and/or supporting hybrid cloud architectures to achieve the required business goals for durability, performance, and coordination.
To resolve this complexity, HDFS-9806 has added a PROVIDED storage tier to HDFS allowing mounting external namespaces, both object stores and other HDFS clusters. Building on this functionality, we can now allow remote namespaces to be synchronized with HDFS, enabling asynchronous writes to the remote storage and the possibility to synchronously and transparently read data back to a local application wanting to access file data which is stored remotely. This talk, which corresponds to the work in progress under HDFS-12090, will present how the Hadoop admin can manage storage tiering between clusters and how that is then handled inside HDFS through the snapshotting mechanism and asynchronously satisfying the storage policy.
Speaker
Thomas Demoor, Object Storage Architect, Western Digital
Ewan Higgs, Software Engineer, Western Digital
GDPR compliance application architecture and implementation using Hadoop and ...DataWorks Summit
The General Data Protection Regulation (GDPR) is a legislation designed to protect personal data of European Union citizens and residents. The main requirement is to log personal data accesses/changes in customer-specific applications. These logs can then be audited by owning entities to provide reporting to end users indicating usage of their personal data. Users have the ""right to be forgotten,â€Âmeaning their personal data can be purged from the system at their request. The regulation goes into effect on May 25,2018 with significant fines for non-compliance.
This session will provide insight on how to approach/implement a GDPR compliance solution using Hadoop and Streaming for any enterprise with heavy volumes of data.This session will delve into deployment strategies, architecture of choice (Kafka,NiFi. and Hive ACID with streaming), implementation best practices, configurations, and security requirements. Hortonworks Professional Services System Architects helped the customer on ground to design, implement, and deploy this application in production.
Speaker
Saurabh Mishra, Hortonworks, Systems Architect
Arun Thangamani, Hortonworks, Systems Architect
This tutorial is designed for anyone who needs to work with data stored in HDF5 files. The tutorial will cover functionality and useful features of the HDF5 utilities h5dump, h5diff, h5repack, h5stat, h5copy, h5check and h5repart. We will also introduce a prototype of the new h52jpeg conversion tool and recently released h5perf_serial tool used for performance studies. We will briefly introduce HDFView. Details of the HDFView and HDF-Java will be discussed in a separate tutorial.
Peer-to-peer networks are right solution for distributing mostly static content to multiple clients while exploiting bandwidth of all clients to share file chunks and available bandwidth.
Torrent is currently most used P2P network protocol, but in this day and age it suffers from lack of official specification and has stigma attached to it. However, if you want to distribute software updates or create multiple disaster recovery sites it makes sense to create private torrent network which will to it's job.
In this presentation I will try to explain what it takes to create your own peer-to-peer cloud based on torrent protocol.
Talk given by Ryan Michela, Principal Software Engineer at Salesforce, at Silicon Valley Code Camp on October 2016
The Git distributed version control system is basically magic. It tracks all your files, branches, merges, and changes. You can work online, offline, with or without a central server. Git has unlocked the power of distributed teams and become one of the foundational technologies of modern software development. Yet how does it work? In this presentation we deep dive into the hidden internals of the Git distributed version control system to find out how they really work. We will start with the history of Git - where it came from and how it came to be. Then we will cover how Git represents objects in it's data store, and what is really going on in your .git directory. Next we will break down a commit and look at how Git represents your commit history. Finally, we will take a look at branching and merging, including what really happens when you rebase a branch. By the end you will be a Git wizard, able to see through the magic and control it for yourself.
As the popularity of Git grows, questions around security and code quality are moving center stage. Learn why the combination of Git/Gerrit and TeamForge form the industry’s leading enterprise-grade solution to manage Git based development programs, both on-premises and in the cloud.
In this webinar, Johannes Nicolai will provide an overview of Git/Gerrit- and why enterprises choose to use the Git-TeamForge integration for unmatched security, scalability and compliance, as well as introduce and demonstrate the power of Gerrit 2.8 with TeamForge:
How you can ensure meeting regulatory and corporate compliance mandates with TeamForge’s tamper-proof audit trails.
How TeamForge provides 100% history protection for accidentally deleted branches and helps you meet the most stringent compliance standards.
How Git, Gerrit and Jenkins reduces the number of manual code reviews by automatically pre-validating builds via Jenkins.
How to graphically design your own review workflows using CollabNet’s Quality Gate Wizard for Gerrit
CONFidence 2018: Attacking web servers via run time configuration (Eldar "Wir...PROIDEA
A few years ago I released a number of .htaccess based attacks against the Apache web server. Since then there has been several changes in the space. Newer Apache versions have new modules, directives and changes to default behavior. On the defense side a common prevention approach has emerged and several solutions claim they can detect malicious .htaccess files. In this workshop I will cover how .htaccess files work, document the changes affecting .htaccess files, and bust some defense myths. We will be exploiting real world bugs along the way to showcase several of these issues. We then bring the existing attacks up to a modern standard and release some new techniques for creating web backdoors that do not need direct access execute code and do not leave any forensic traces in the logs. I will also release similar attacks against non-Apache servers.
These are slides of a GitHub training for the Roslyn team. The recording can be found here:
http://channel9.msdn.com/Blogs/dotnet/github-for-the-roslyn-team
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
FOSDEM 2012 git-annex talk
1. Intro Use cases Technical details Outro
git-annex
manage files with git, without checking their contents in
Richard Hartmann,
RichiH@{freenode,OFTC,IRCnet},
richih.mailinglist@gmail.com
2012-02-05
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
2. Intro Use cases Technical details Outro
Outline
1 Intro
2 Use cases
3 Technical details
4 Outro
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
3. Intro Use cases Technical details Outro
Outline
1 Intro
2 Use cases
3 Technical details
4 Outro
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
4. Intro Use cases Technical details Outro
Who am I?
Project & Network Operations Manager at Globalways AG
freenode & OFTC staff
Passionate about FLOSS
I am not the author of git-annex, but an interested early
adopter
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
5. Intro Use cases Technical details Outro
What is git?
Version control system
Distributed
No need for central repository
Commit while offline
Full history of all files in every checkout
Best version control system available (imo...)
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
6. Intro Use cases Technical details Outro
What is git-annex?
Based on git
No need to check files into git
Still able to check files into git if you want
Able to maintain full history, but does not do so by default
Written with low bandwidth and flaky connections in mind
Various work-flows
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
7. Intro Use cases Technical details Outro
Outline
1 Intro
2 Use cases
3 Technical details
4 Outro
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
8. Intro Use cases Technical details Outro
The Archivist
Put data into git-annex
Distribute data among any number of drives, tapes,
remotes, etc
Store offline media in a safe place
Maintain full information about number and location of all
copies
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
9. Intro Use cases Technical details Outro
Media consumption
Import podcasts, videos, and slides
Sync or export to consumption devices
Consume media
Drop consumed media from annex
Deletion propagates through all annexes over time
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
10. Intro Use cases Technical details Outro
The Nomad
Keep copies of data on www
Optionally sync between several local devices for backup
Add data locally and/or remotely while on the road
Sync data between local and remote once at an Internet
´
cafe or similar
Perfect for photos while travelling
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
11. Intro Use cases Technical details Outro
Outline
1 Intro
2 Use cases
3 Technical details
4 Outro
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
12. Intro Use cases Technical details Outro
Internal workings 1/2
Written in Haskell, so strong typing etc internally
Uses rsync to transfer data
Moves files into .git/annex/objects
Makes files read-only
Puts symlink in place of file
Stores location data in branch git-annex
User adds and commits symlinks to master branch
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
13. Intro Use cases Technical details Outro
Internal workings 2/2
Read-only files force you to git annex unlock prior to
changing them
Ensures that you will git annex add all unlocked files
git-annex can then discard or keep old data, depending on
setup
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
14. Intro Use cases Technical details Outro
Data integrity
Set minimal number of required copies per suffix, directory,
etc
SHA1, SHA2-{224,256,384,512} for integrity
All remotes and special remotes can be verified
remotes verify locally and transmit the result
special remotes transfer all data to verify
Verification takes required amount of copies into account
git fsck; git annex fsck
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
15. Intro Use cases Technical details Outro
Special remotes 1/2
Stores data in non-git-annex remotes
Tracks all data stored in special remotes
Supports encryption for storage on untrusted
machines/media
Hook system lets you write to and read from arbitrary
remotes
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
16. Intro Use cases Technical details Outro
Special remotes 2/2
bup
directory
rsync
S3, Swift, etc
Tahoe-LAFS
web (media.ccc.de, Project Gutenberg, archive.org, etc)
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
17. Intro Use cases Technical details Outro
Outline
1 Intro
2 Use cases
3 Technical details
4 Outro
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
18. Intro Use cases Technical details Outro
Where to get it
cabal install git-annex --bindir=$HOME/bin
Native packages for
Debian
Ubuntu
FreeBSD
Arch Linux
NixOS
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
19. Intro Use cases Technical details Outro
Further reading
https://github.com/RichiH/talks
http://git-annex.branchable.com/
http://www.slideshare.net/RichiH/
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex
20. Intro Use cases Technical details Outro
Thanks!
Thanks for listening!
Questions? Follow me outside when my time-slot is over.
See slide footer for further contact Information.
Richard Hartmann, RichiH@{freenode,OFTC,IRCnet}, richih.mailinglist@gmail.com
git-annex