FASTEN presentation at FOSDEM 2022 : Making Dependency Management Intelligent

•

0 likes•24 views

Fasten Project

The FASTEN project was presented for the third time at FOSDEM, on February 4-5, 2022, online, by Amir Mir from TUDelft.

FASTEN
Making Dependency Management Intelligent
Amir M. Mir (TU Delft)
@amir_mir93
s.a.m.mir@tudelft.nl

Open Source Software (OSS)
● Open-source libraries as a building block for creating new software

Package Dependency Networks (PDNs)
● Version constraints make these networks more complicated

Recent Failures with PDNs
● Leftpad in 2016
● Equifax data breach in 2017
● Log4j in 2021

Issues with PDNs
● The update problem
● The compliance problem
● The deprecation problem
● The lack of incentive problem

Existing Solutions to the Issues of PDNs
● Services like GitHub, Dependabot
● Problems:
○ No support for assessing updates
○ No help with impact assessment
○ False positives

The Root Cause
Current Solutions
Call Dependency
Networks (CDNs)

The FASTEN Project
● Fine-Grained Analysis of Software Ecosystems as Network
● Aims at solving the issues of PDNs by making package management robust and
intelligent
● A centralized service to host the graphs and serve the analyses

The FASTEN Solution
● More precise license compliance
○ Am I linking to GPL code?
● More precise risk profiling
○ Does this vulnerability affect my package?
● More precise change impact analysis
○ How many packages will I break if I change this function?
○ Can I safely update the dependencies of my package?
● Integration with package managers

Overview of the FASTEN Architecture
Data streams
Package repositories
Vulnerability information
FASTEN server
Call graph generators
Analysis layer
Security Change impact
Compliance Quality and Risk
Storage layer
REST
API
Web
UI
Continuous
Integration
servers

Examples of FASTEN Workflow
Update with confidence
Before FASTEN After FASTEN

Examples of FASTEN Workflow
Deciding to use a library
Before FASTEN After FASTEN

Questions?
https://www.fasten-project.eu/
https://github.com/fasten-project
https://twitter.com/fastenproject
The FASTEN project has received funding from the European Union’s Horizon 2020 research
and innovation programme under grant agreement No 825328.

Presentation by Amir Mir, TUDelft. As recent events, such as the leftpad incident and the Equifax data breach, have demonstrated, dependencies on networks of external libraries can introduce projects to significant operational and compliance risks as well as difficult to assess security implications. FASTEN introduces fine-grained, method-level, tracking of dependencies on top of existing dependency management networks. In our talk, we will present how FASTEN works on top of the Rust/Cargo and Java/Maven ecosystems.

FASTEN: Intelligent Software Package Management

Amir M. Mir

FASTEN presentation at OSS2021, by Michele Scarlato, Endocode, May 12, 2021, ...

Fasten Project

The FASTEN project wants to support DevOps teams and help developers tracking, managing and mastering dependencies. FASTEN’s goal is to develop a toolchain that is provisioning and collecting project information, security alerts, and repositories from well-known and widely used services. It merges this information into a data stream, performs analysis, stores it, and, consequently, builds a call-graph for each analyzed project. The gathered information is made available through a REST API and Web UI and performs continuous integration to provide developers with updated and sanitized versions of their dependencies. One part of this toolchain will be an Open Source license analysis. This analysis should perform a verification and compatibility check on licenses used in Open Source projects and facilitate development from a user perspective as well as create industry-relevant information on license infringements. This functionality shall be presented in this talk. FASTEN has received funding from the European Union's Horizon 2020 research and innovation programme. It is carried out by a Consortium composed of AUEB, TUDelft, University of Milan-Bicocca, Endocode, OW2, SIG, and XWIKI.

Data Versioning and Reproducible ML with DVC and MLflow

Databricks

Machine Learning development involves comparing models and storing the artifacts they produced. We often compare several algorithms to select the most efficient ones. We assess different hyper-parameters to fine-tune the model. Git helps us store multiple versions of our code. Additionally, we need to keep track of the datasets we are using. This is important not only for audit purposes but also for assessing the performances of the models, developed at a later time. Git is a standard code versioning tool in software development. It can be used to store your datasets but it does not offer an optimal solution.

Open Source Big Graph Analytics on Neo4j with Apache Spark

Kenny Bastani

In this talk I will introduce you to a Docker container that provides an easy way to do distributed graph processing using Apache Spark GraphX and a Neo4j graph database. You’ll learn how to analyze big data graphs that are exported from Neo4j and consequently updated from the results of a Spark GraphX analysis. The types of analysis I will be talking about are PageRank, connected components, triangle counting, and community detection.

Driving the future of PostgreSQL adoption

Umair Shahid

Inauguration lecture Martin Pinzger, University of Klagenfurt, Austria

Martin Pinzger

Slides of my inauguration lecture at the University of Klagenfurt in Austria. In this talk I outline several challenges of evolving software systems and present several ideas and findings from my research to address them. In particular, I show how we can use the history of software projects to identify critical parts of a software system and how we can use visualization techniques to help software engineers to understand the implementation of large, complex software systems including large spreadsheets.

Surviving microservices

Francesco Degrassi

With microservices gone mainstream a few years ago, many organizations have now adopted them; even though all are paying the price in terms of training, solution complexity and operational costs, few are reaping the promised benefits. Lower velocity, quality and performance issues, along with an overall lack of visibility are what we hear about most often. In this session, working from our experience as advisors to software development teams, we’ll walk you through some of the symptoms you might experience, their possible causes and some potential solutions.

Modern applicatons require modern monitoring solutions that can react fast on changes in the monitored applications (think of autoscaling, updates). And after many years our old monitoring system, based on Nagios and Cacti, was not holding up anymore. This talk tells the story of your journey from our old system through defining our requirements and multiple tool evaluations (Zabbix, Prometheus, Icinga2) to our current impementation based on Icinga2. I will also show some of our implementation details and how we solved problems in our deployment.

presentation_SB_v01Salvatore Balzano

Streaming is a Detail

HostedbyConfluent

"We can all agree that streaming is super cool. And for a while now, the adoption conversation has been largely led with an all-in mentality. But that’s silly. The only concerns end users have are: -The freshness of their data -Latency they require to meet their SLAs from source to consumption -All while maintaining data quality and governance. Luckily, the industry has realized this and we have seen a shift of streaming capabilities surfacing as an in-database technology, via objects as trivial to analytics engineers as views - materialized that is. With this convergence of streaming capabilities and batch level accessibility, this is when ELT tools like dbt can join in and expand out the adoption story. dbt is the T in ELT, Extract Load and Transform. In dbt, analytics engineers design models - SQL (and occasional python) statements that encapsulate business logic. At runtime, dbt will wrap that logic in a DDL statement and send it over to the data platform to execute. In this session, we’ll discuss how we see streaming at dbt Labs. We will dive into how we are extending dbt to support low-latency scenarios and the recent additions we have made to make batch and streaming allies in a DAG rather than archenemies."

Webinar slides: DevOps Tutorial: how to automate your database infrastructure

Severalnines

Join our guest speaker Riaan Nolan of mukuru.com, the First Puppet Labs Certified Professional in South Africa, as he walks us through the facets of DevOps integrations and the mission-critical advantages that database automation can bring to your database infrastructure. Infrastructure automation isn’t easy, but it’s not rocket science either. Done right, it is a worthwhile investment, but deciding on which tools to invest in can be a confusing and overwhelming process. Riaan will share some of his secrets on how to proceed with this and he knows what he’s talking about: he saves the companies he works for substantial amounts on their monthly IT bills, typically around 50%. Don’t miss out on this opportunity to understand how you can find efficiencies for your database infrastructure and do watch this webinar to understand the key pain points, which indicate that it’s time to invest in database automation. AGENDA DevOps and databases - what are the challenges Managing databases in a DevOps environment - Requirements from microservice environments - Automated deployments - Performance monitoring - Backups - Schema changes - Version upgrades - Automated failover - Integration with ChatOps and other tools Data distribution - Database hosting in cloud environments - Managing data flows Cloud Automation on AWS SPEAKERS Riaan Nolan was the First Puppet Labs Certified Professional in South Africa. Riaan uses Amazon EC2, VPC and Autoscale with Cloudformation to spin up complete stacks with Autoscaling Fleets. He saves companies substantial amounts on their monthly IT bills, typically around 50% - yes, at one company that meant $500k+ per year. And he’s participated in a number of community tech related forums. He uses next generation technologies such as AWS, Cloudformation, Autoscale, Puppet, GlusterFS, NGINX, Magento and PHP to power huge eCommerce stores. His specialties are Puppet Automation, Cloud Deployments, eCommerce, eMarketing, Specialized Linux Services, Windows, Process making, Budgets, Asset Tracking, Procurement. - Devops Lead, Mukuru - Expert Live Systems Administrator, foodpanda | Hellofood - Senior Systems Administrator / Infrastructure Lead, Rocket Internet GmbH - Senior Technology Manager, Africa Internet Accelerator Art van Scheppingen is a Senior Support Engineer at Severalnines. He’s a pragmatic MySQL and Database expert with over 15 years experience in web development. He previously worked at Spil Games as Head of Database Engineering, where he kept a broad vision upon the whole database environment: from MySQL to Couchbase, Vertica to Hadoop and from Sphinx Search to SOLR. He regularly presents his work and projects at various conferences (Percona Live, FOSDEM) and related meetups.

Software Quality Concerns in the Italian Bank Sector: the Emergence of a Me...

Daniel Russo

ngStockholm #8 at NetEnt - Micro Frontend Architecture

Ishaan Puniani

5 Years of Jenkins and DevOps Trends and What That Means For the Future of t...

DevOps.com

With 5 years of research and over 5000 survey respondents, there's no better way of understanding the evolving nature of the industry than this year's DevOps and Jenkins Community survey results. Join us, and a panel of industry experts, as we dig through historical data and retrospective insights to formulate meaningful predictions on the unfolding landscape of DevOps and the Jenkins Community in the years to come.

Effects of Ownership on Software Quality

Md. Shafiuzzaman Hira

What's new in the latest source{d} releases!

source{d}

Oprim - .Net Core Development Company in Canada

OprimSolutions1

Building Business on Top of Open Source

Open Networking Summit

RNUG 2020: Domino Application Strategy: Key insights for successful moderniza...

panagenda

panagenda reached out to 750+ professionals to share their company’s Domino application strategy. Join this session to find out what was most important to your peers and what challenges they had to overcome to make their project a success. Find out about the critical questions everybody should ask and have answers to throughout their project. Franz Walder presents the exciting results of the survey and explains what role analytics can play when tackling these challenges.

Why Pay for Open Source Linux? Avoid the Hidden Cost of DIY

Enterprise Management Associates

Organizations can pick between numerous free community-supported distributions of the Linux operating system. In the data center and on AWS, Azure, GKE, CloudFlare, DigitalOcean, and other public clouds, these free versions are available as part of the default configuration. Why, then, would you pay for Linux? These slides, based on a webinar hosted by Red Hat and leading IT research firm EMA, provide insights into what has and has not worked related to the adoption of free versus subscription-based Linux distributions.

D1: The NMC Methodology

lisbk

Performance monitoring for remote locations

AppNeta

Onkurananda1Onkurananda Ganguly

Enase20.ppt

Yann-Gaël Guéhéneuc

HLayer / Cloud Native Best Practices

Aymen EL Amri

FASTEN presentation at OW2con'22

Fasten Project

FASTEN presentation at OW2con 2021

Fasten Project

FASTEN H2020 project presentation at Paris Open Source Summit, December 2019.

Fasten Project

Fasten and Quartermaster presentation at FOSSCOMM, October 2019 in Lamia, Gre...

Fasten Project

Software engineers reuse code to reduce development and maintenance costs but how safe is it to use open source software (OSS)? By using OSS and dependencies to external libraries they can introduce to projects significant operational and compliance risk as well as difficult to assess security implications. The aim of the FASTEN project (a European Union’s H2020 research and innovation programme led by TU Delft) is to address this situation, by developing an intelligent software package management system that will enhance robustness and security in software ecosystems. Our team in Endocode AG is part of the FASTEN project with our FOSS toolchain Quartermaster, which detects license compliance on softwares.

Fasten Industry Meeting with GitHub about Dependancy Management

Fasten Project

More from Fasten Project (17)

FASTEN presentation at OW2con'22

FASTEN presentation at OW2con 2021

FASTEN Introduction, at EclipseCon 2021

FASTEN user experience from a software vendor perspective : The future of ext...

Eclipse sw360 Web Application for managing software Bill-Of-Material, FASTEN ...

Demonstration of FASTEN Dependency Management tools on top of Maven, FASTEN v...

Highlight on FASTEN's Software Composition Analysis Market Background, Virtua...

Software Ecosystems as Networks - Advances on the FASTEN project, Paolo Boldi...

FASTEN presentation at SFScon, November 2020

FASTEN: Scaling static analyses to ecosystem, presented at FOSDEM 2020 in Bru...

FOSDEM 2020 Presentation - There's no sustainability problem in FOSS, Except ...

FOSDEM 2020 Presentation: Comparing dependency management issues across packa...

FOSDEM 2020 Presentation : Precise, cross-project code navigation at GitHub s...

Presentation of the FASTEN project, Conference SFScon, Bolzano, Italy

FASTEN H2020 project presentation at Paris Open Source Summit, December 2019.

Fasten and Quartermaster presentation at FOSSCOMM, October 2019 in Lamia, Gre...

Fasten Industry Meeting with GitHub about Dependancy Management

Recently uploaded

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024

Neo4j

UiPath Test Automation using UiPath Test Suite series, part 5

DianaGray10

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

Albert Hoitingh

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

Neo4j

Dr. Sean Tan, Head of Data Science, Changi Airport Group Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.

Monitoring Java Application Security with JDK Tools and JFR Events

Ana-Maria Mihalceanu

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

Alex Pruden

This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second). Paper: https://eprint.iacr.org/2023/1886

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

DevOps and Testing slides at DASA Connect

Kari Kakkonen

Elizabeth Buie - Older adults: Are we really designing for our future selves?

Nexer Digital

みなさんこんにちはこれ何文字まで入るの？40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの？えこ...

名前です男

By Design, not by Accident - Agile Venture Bolzano 2024

Pierluigi Pugliese

How to Get CNIC Information System with Paksim Ga.pptx

danishmna97

Free Complete Python - A step towards Data Science

RinaMondal9

Introduction to CHERI technology - Cybersecurity

mikeeftimakis1

20240607 QFM018 Elixir Reading List May 2024

Matthew Sinclair

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!

SOFTTECHHUB

As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.

Recently uploaded (20)

GraphRAG is All You need? LLM & Knowledge Graph

Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024

UiPath Test Automation using UiPath Test Suite series, part 5

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

Monitoring Java Application Security with JDK Tools and JFR Events

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DevOps and Testing slides at DASA Connect

Elizabeth Buie - Older adults: Are we really designing for our future selves?

By Design, not by Accident - Agile Venture Bolzano 2024

How to Get CNIC Information System with Paksim Ga.pptx

Free Complete Python - A step towards Data Science

Introduction to CHERI technology - Cybersecurity

20240607 QFM018 Elixir Reading List May 2024

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!

FASTEN presentation at FOSDEM 2022 : Making Dependency Management Intelligent

1. FASTEN Making Dependency Management Intelligent Amir M. Mir (TU Delft) @amir_mir93 s.a.m.mir@tudelft.nl

2. Open Source Software (OSS) ● Open-source libraries as a building block for creating new software

3. Dependencies

4. Package Dependency Networks (PDNs) ● Version constraints make these networks more complicated

5. Recent Failures with PDNs ● Leftpad in 2016 ● Equifax data breach in 2017 ● Log4j in 2021

6. Issues with PDNs ● The update problem ● The compliance problem ● The deprecation problem ● The lack of incentive problem

7. Existing Solutions to the Issues of PDNs ● Services like GitHub, Dependabot ● Problems: ○ No support for assessing updates ○ No help with impact assessment ○ False positives

8. The Root Cause Current Solutions Call Dependency Networks (CDNs)

9. The FASTEN Project ● Fine-Grained Analysis of Software Ecosystems as Network ● Aims at solving the issues of PDNs by making package management robust and intelligent ● A centralized service to host the graphs and serve the analyses

10. The FASTEN Solution ● More precise license compliance ○ Am I linking to GPL code? ● More precise risk profiling ○ Does this vulnerability affect my package? ● More precise change impact analysis ○ How many packages will I break if I change this function? ○ Can I safely update the dependencies of my package? ● Integration with package managers

11. Overview of the FASTEN Architecture Data streams Package repositories Vulnerability information FASTEN server Call graph generators Analysis layer Security Change impact Compliance Quality and Risk Storage layer REST API Web UI Continuous Integration servers

12. Examples of FASTEN Workflow Update with confidence Before FASTEN After FASTEN

13. Examples of FASTEN Workflow Deciding to use a library Before FASTEN After FASTEN

14. FASTEN Consortium

15. Questions? https://www.fasten-project.eu/ https://github.com/fasten-project https://twitter.com/fastenproject The FASTEN project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 825328.

FASTEN presentation at FOSDEM 2022 : Making Dependency Management Intelligent

Recommended

Recommended

More Related Content

Similar to FASTEN presentation at FOSDEM 2022 : Making Dependency Management Intelligent

Similar to FASTEN presentation at FOSDEM 2022 : Making Dependency Management Intelligent (20)

More from Fasten Project

More from Fasten Project (17)

Recently uploaded

Recently uploaded (20)

FASTEN presentation at FOSDEM 2022 : Making Dependency Management Intelligent