Managing the evolution of F/OSS with Model Driven Techniques
Upcoming SlideShare
Loading in...5
×
 

Managing the evolution of F/OSS with Model Driven Techniques

on

  • 431 views

Short Tutorial at GTTSE 2011 in Braga, Portugal

Short Tutorial at GTTSE 2011 in Braga, Portugal

Statistics

Views

Total Views
431
Views on SlideShare
431
Embed Views
0

Actions

Likes
0
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Managing the evolution of F/OSS with Model Driven Techniques Managing the evolution of F/OSS with Model Driven Techniques Presentation Transcript

  • Università degli Studi dell’AquilaAlfonso Pierantonio, Davide Di RuscioDipartimento di InformaticaUniversità degli Studi dell’Aquila (Italy)alfonso.pierantonio@univaq.it
  • 2 → Linux distributions are among the most complex software ecosystems → More than 29,000 interdependent packages → Keynote at SLE11 by Krzysztof ca. 10,000 features only in the kernel GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model- Driven Techniques
  • 3 explicit dependencies among linux packages GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 4 Free and Open Source Software → From tarballs to distributions → Package technology Problem: upgrade failures Abstracting the System → Modeling languages for describing the System → Maintainer Script Analysis → Injectors Analyzing the System → Upgrade simulator → Fault detector Conclusions GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 5 Today Alfonso Pierantonio → A gentle introduction → Models of Complex Systems July 6, 2011 – 14:45 Davide Di Ruscio → Concrete Artifacts → Demo GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 6 Free and Open Source Software FOSS distributions are among the most complex software systems known, being made of thousands components evolving rapidly without centralized coordination Large numbers (components/developers) → SourceForge.net contains more than 123,736 projects, 1,342,153 “users” → Debian GNU/Linux is distribuited with more than 29,000 pre-compiled packages GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 7 Free and Open Source Software Numbers like that have the potential to radically change the way we produce and study software, in particular the complex systems of the future complexity dependencies
  • 8 In the beginning it was the tarball Before the advent of distributions, the peculiar way to install free software on client machine was: server client side side bazaar proj 1 user proj 2 installations proj n GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 9 To answer these problems, GNU/Linux distributions have born as intermediaries between FOSS projects and their usersFOSS proj 1 server clientbazaar proj 2 side side user proj n installations package management meta- installer distribution editors package repository
  • 10 Free and Open Source Software FOSS distributions are among the most complex software systems known, being made of thousands components evolving rapidly without centralized coordination FOSS components are provided in packaged form by distribution editors Packages define the granularity at which components are managed using package manager applications GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 11 GNU/Linux distributions Central notion in distributions (to abstract over the complex underlying infrastructure): → package, together with package management software package management distribution editors package repository GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 12 GNU/Linux distributions Central notion in distributions (to abstract over the complex underlying infrastructure): → package, together with package management software package management distribution editors And yet, doing things right can be package repository extremely difficult! GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 13 Installation intricacies In state of the art distributions: → knowing if the installation process terminates without errors is undecidable (reason: configuration scripts) … today → there is no guarantee that distribution repositories contains only installable packages → a large number of errors faced by users are dependency resolution errors ─ more than 29,000 packages (and 10,000 sources) in Debian ─ more than 200.000 inter-package relationships GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 14 PROBLEM: UPGRADE FAILURES
  • 15 → each phase can fail – it actually happens quite often! → efforts should be made to identify errors as early as possible, if not predict
  • 16 Upgrade failures Essentially they can fail for the following classes of problems → Consistency breach due to administrator operations at a finer level of granularity than that of packages ─ eg. needed resources are manipulated or removed from the command-line → Implicit dependencies are not considered in the installation/removal script in the upgraded packages → Script erroneous behavior GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 17 Installing package libapache-mod-ssl Configuration 1 > apt-get install libapache-mod-ssl + Configuration 2
  • 18
  • 19
  • 20
  • 21 once mod_ssl is installed, it is enabled in apache GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 22 removing package libapache-mod-ssl Configuration n+1 - > apt-get remove libapache-mod-ssl Configuration n
  • 23 GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 24 It disables mod_ssl in apache. What happens if the maintainer does not write this statement ? GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 25 An “inconsistent” configuration is reached and it is detected only at run-time: the package dependency metadata are not enough ! GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 26 Problem: Upgrade failures Current tools are able to predict a limited set of upgrade failures before deployment When trying to predict upgrade failures, existing tools only consider static package metadata and the behaviour of the maintainer scripts is completely ignored → This leaves a wide range of failures unpredicted GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 27 Upgrade failures classification Static deploy-time occur when a static requirement is violated during the upgrade. → the low-level package manager fails at deploy-time, aborting the upgrade process Dynamic deploy-time occur when a maintainer script fails → such failures can originate from errors ranging from syntax to failures in external tools → they are not addressed by state of the art package managers Undetected failures remain undetected through upgrade deployment → the upgrade has been completed successfully, but the obtained system configuration contains undetected incoherences GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 28 IDEA : ABSTRACTING AND ANALYZING THE SYSTEM
  • 29 Idea Extracting relevant information from a linux box («snapshot») at a given moment of time Analysis and upgrade simulation on the «snapshot» injectors «snapshot» Linux box
  • 30 Ingredients 1 Modeling languages for describing the several aspects of a linux distribution → Packages ─ including maintainer scripts → System Configuration ─ Installed packages ─ Configuration files ─ MIME-type handlers ─ Alternatives ─ etc Injectors for harvesting the system and building the models → collection of injectors GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 31 Ingredients 2 Modeling language for the maintainer scripts → scripts are written in POSIX languages whose semantics is far from being simple, although → maintainer scripts does not harness the full expressivity of such languages (template-based “macro-language”) Maintainer scripts as models → which semantics ? GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 32 Ingredients 3 Transformational semantics for simulating the behavior of the maintainer scripts on the system «snapshot» → M2M transformations obtained by «compiling» the maintainer scripts into ATL Fault detector, a general mechanism for performing queries over the «snapshot» for digging the model and search for inconsistencies GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 33 Configuration 1 ? Configuration 2 package injection system injection M2M transformation M2M transformation «snapshot» 1 «snapshot» 2
  • 34 A model-based approach is introduced to support the package upgrades and enhance the failure detection possibilities: → the simulator is used to predict the effect of maintainer script executions (deploy-time failures) → the fault detector is used to deal with undetected failures
  • 35 EVOSS A model-based approach is introduced to support the package upgrades and enhance the failure detection possibilities: → the simulator is used to predict the effect of maintainer script executions (deploy-time failures) → the fault detector is used to deal with undetected failures
  • 36 MAINTAINER SCRIPT ANALYSIS
  • 37 Rigourous maintainer script analysis (1/2) Script analysis for Debian and RPM based distributions To support the analysis and represent the results a database of scripts has been developed → Already known templates have been identified ─ By taking into account annotations in the scripts (when available, eg. in case of Debian scripts) ─ Exact match of existing templates in the scripts → New templates have been identified ─ By using an “exact match” between scripts ─ By analyzing block statements (if, while, case, for, etc.) ─ by using a similarity based approach GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 38 Rigourous maintainer script analysis 2/2 http://evoss.di.univaq.it
  • 39 INJECTORS
  • 40 Used to represent in terms of models packages and system configurations GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 41 Configuration and package injectors By means of the model injection, given a linux system a corresponding model is obtained Injectors inspect software artifacts and inject relevant information into corresponding models The process is driven by the metamodels GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 42 Configuration injector Debian Mandriva Caixa Magica Ubuntu Model Injector Model Injector … Model Injector Model Injector Mancoosi Model Injection Infrastructure Mancoosi Model Management cfginject [-d <distribName>] [-full] → It creates a system configuration model starting from a real system. It is possible : ─ to specify the distribution name ─ to force the complete regeneration of the model. By default incremental changes of the model are performed in order to keep the system and the corresponding model synchronizedDEMO
  • 43 Package injector pkginject [--installed] [--package <package file name>] [--cachedir] → It creates package models. By default, a model for each installed package is created (--installed) → It is possible to inject only one package by specifying the package file name (--package) → It is also possible to inject all the packages which are in the cachedir (-- cachedir)DEMO
  • 44 UPGRADE SIMULATOR
  • 45 Upgrade Simulator Used to simulate system configuration upgrades GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 46 Input Output Upgrade plan: New {(p1,u1),(p2,u2)…,(pn,un)} Configuration Model Configuration Upgrade Model Simulator Error Model {p1, p2, …, pn} Package Model1 Package Package Model2 Injection … Package Modeln GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 47 Configuration Upgrade plan: Model {(p1,u1),(p2,u2)…,(pn,un) } For 1≤i ≤ n : Package Model1 retrieve the state of pi Package Model2 … ui pi state Package Modeln For each pi simulate upgrade scenario New Configuration valid simulation Model error found Error Upgrade Simulator Model GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 48 install Not installed Configuration Model Package Modeli preinst install “Files are unpacked” OK FAILED FAILED postinst configure postrm abort-install Half-configured OK OK Upgrade scenario FAILED Installed New Not installed Half installed Configuration “Reinst required” Model Error Model GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 49 Package Script Simulator Model Script Script Script Retrieval of Model transformations St1 St2 … Repository of Stn Model Model transformation transformations orchestration Configuration generation Model Model New transformation Configuration Model orchestration execution Error Model GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 50 Configuration Tst1 Model CM1 Script Tst2 st1 Orchestration st2 Error CM2 of model … Model transformations stn … CMn-1 New Configuration Tstn Model GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 51 GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 52 GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 53 swi-prolog 5.6.58 <?xml version="1.0" encoding="UTF-8"?> [Not-Installed] <selectionStates xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <selectionState type="install"> install <param name="package" value="swi-prolog"/> <param name="version" value="5.6.58"/> swi-prolog <param name="architecture" value="all"/> 5.6.58 </selectionState> [Installed] <selectionState type="remove"> <param name="package" value="swi-prolog"/> remove <param name="version" value="5.6.58"/> <param name="architecture" value="all"/> swi-prolog </selectionState> 5.6.58 <selectionState type="install"> [ConfigFiles] <param name="package" value="swi-prolog"/> <param name="version" value="5.6.58"/> install <param name="architecture" value="all"/> </selectionState> swi-prolog <selectionState type="install"> 5.6.58 <param name="package" value="swi-prolog"/> [Installed] <param name="version" value="5.7.59"/> <param name="architecture" value="all"/> upgrade </selectionState> </selectionStates> swi-prolog 5.7.59 [Installed] Sample upgrade planDEMO
  • 54 FAULT DETECTOR
  • 55 Used to check system configurations for faults that might give place to failures in the real system GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 56 Fault detector Discovering implicit dependencies among packages: we are able to discover dependencies that are not declared into the packages meta-information Discovering missing configuration files: according to the system configuration model, some configuration files are required but they are not available in the system Discovering Mime-type dangling handlers: according to the available information, the considered system should be able to manage a mime type, but the corresponding handler is missing in the system Discovering missing services: the init.d file contains services that should start at the system start-up; by querying the configuration model, it is possible to detect missing services … GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 57 Fault detector architecture MANCOOSI Server -Libraries (.jar files) and OCL queries User machine -Catalogue of faults -Catalogue of solutions FD-Server FD-Client Web Portal UserDEMO
  • 58 Conclusions 1 The prediction of the package upgrade impact has been enhanced: a larger number of cases which lead to faulty behaviors can be detected The core ingredients have been defined → Modeling languages for the different aspects → Injectors: system configuration and packages → Transformational Script Semantics ─ based on ATL → Fault detection: query-based global knowledge base ─ OCL and JAR queries to detect faults ─ Contribution from the community GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 59 Conclusions 2 The main difficulty is in building the model injectors → labor intensive, deep platform knowledge required, ad-hoc techniques → Interesting (academic) tools available ─ GRA2MOL – grammarware / modelware bridging ─ WIRES* – Model transformation orchestration Monolithic real scale metamodels → Maintenance of the developed metamodels has been an issue: interdependencies, ripple effects → Difficult stabilization of the support tools Model comparison for validating the simulation → EMF Compare GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 60 Conclusions 3 This work has been done within the → EU FP7 ICT STREP MANCOOSI http://www.mancoosi.org Industrial partners → Mandriva → Caixa Magica → IBM Ilog GTTSE 2011 Alfonso Pierantonio – Managing the evolution of F/OSS with Model-Driven Techniques
  • 61
  • 62 Questions? Università degli Studi dell’Aquila
  • 63
  • 64 Upgrade plan: {(p1,u1),…,(pn,un) Package Package Upgrade Package CM1 4.Simulator } Model Model Model 1.Upgrade Package System Planner 2. Injector 3.Injector CM2 (p,u) Real System Packages 5.Upgrade Upgraded Real System 6. Injector CM3 System Delta Configurations Model 7. compare
  • 65