This document discusses improving rollback capabilities in Linux via a domain specific language (DSL) approach and distributing data. It presents an overview of rollback, different types of rollback, and challenges. The authors propose using a DSL to model package maintainer scripts to better support rollback. The DSL would be logged and used to drive an automated rollback. Distributed data storage via peer-to-peer networks is also proposed to share rollback logs and package state information. The overall goal is to build a framework enabling transactionally protected package management including reliable rollback.

1. Energia Open Source
Improving Rollback in Linux
via DSL approach & distributing
Sunday, 6th February 2011
Presented by:
John Thomson: John.Thomson@caixamagica.pt
Researcher
Paulo Trezentos: Paulo.Trezentos@caixamagica.pt
http://twitter.com/PauloTrezentos
R&D Director

2. Energia Open Source
Overview
 An overview of what Caixa Mágica does.
– MANCOOSI project: European 7th Framework Programme
 Roll-back
– Definition
– Types of roll-back
– As part of a bigger system
 DSL
– Definition
– Example
 Approach
 Implementation
 Distributing data (kad/p2p)
 Conclusion
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 2

3. Energia Open Source
A brief summary of Caixa Mágica Software
and an overview of what we do:
 Caixa Mágica is a FOSS software and solutions
provider, based in Lisbon, Portugal
 Linux- Caixa Mágica 15 is the main Linux
Distribution available in Portugal
 Caixa Mágica work with national companies,
Government and with European partners to promote Open
Source in all aspects of work
 Over 800,000 Linux CM systems installed (single & dual boot)
 Interesting development → CM16 moving to Debian base
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 3

4. Energia Open Source
European based research projects
 Caixa Mágica works in many multi-national European based
research projects. After EDOS (EU FP6 STREP) based project
it is now actively working on MANCOOSI-FP7 Project
 Also working on ULOOP and Timbus projects
 Work with a multitude of top-tier Universities and research
institutions
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 4

5. Energia Open Source
About MANCOOSI
 MANCOOSI - Managing the complexity of open-
source software. www.mancoosi.org
Solving package management issues that have
been identified through EDOS project
Other talks
 Talks yesterday by Ralf and Jaap about solvers + tools
 André Guerreiro presents Gumby in room H.1302 at 15:30
– Visualise package meta-data
 Paulo Trezentos presents APT-PBO in room H.1302 at 16:00
– A better class of solver
 Aim is to provide better tools for package management
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 5

6. Energia Open Source
Roll-back as simple as traversing time?
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 6

7. Energia Open Source
What does roll-back mean, really?
 In terms of package configuration,
“roll-back is the process of inverting the changes to the
system made by package upgrades to get back to a
particular system state.”
 Many other mechanisms out there that work on using file
system snapshots/saving the state, (next slide)
S1 S2
PkgFoo v 1.00 Roll-back, PkgFoo v 2.00
Time: 10.00pm possible? Time: 10.20pm Installation Timeline
 The mechanism is one part of Transactionally Protected
Package Management as spoken about by Jeff Johnson last
year at FOSDEM 2010.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 7

8. Energia Open Source
Different types of roll-back
 Conary, a 2nd Generation Package Manager, aims to
improve current meta-installers. Used in Linux
distributions such as Foresight Linux.
 Augeas, is a configuration management tool that
makes manipulating config files much easier
 ZFS, used by Nexenta is an example of a file-system,
snapshot mechanism to snapshot several system states.
 NixOS, a revolutionary system that re-thinks how files
and resources are used to make them purely functional
 Other mechanisms e.g. etckeeper being developed by
Fedora also try and capture configuration files into a
VCS. BTRFS – another snapshot based mechanism.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 8

9. Energia Open Source
Drawbacks of the various approaches
 File-system Snapshot based mechanisms
– Capture state of the whole system → down to individual files
– Agnostic of packages
– Granularity is not well matched for packages
 Package Based
– Same as installing an older version
– Maintain configurations and settings
 System Call based
– Syscall trapping - records changes to the file-system
– Capturing files alone is not always enough
 Aim is to combine the complimentary parts of these systems
to cover a wide range of situations and events
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 9

10. Energia Open Source
Difficulties of Roll-back
 Package maintainers and developers tend to think in the
forward direction of upgrading packages
 Working in the reverse direction is a relatively unheard of
concept. Think of a toy slinky
 Downgrade is counter-intuitive
 Already have been done if it was necessary?
 Rolling-back changes is 'only' needed when a package fails
to work on the system
 There may be cases where roll-back is impossible using
the techniques that we have investigated, or possibly at all
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 10

11. Energia Open Source
Roll-back is one part of a bigger picture
 Ability to undo package upgrade/install is just one benefit of
having a system that can capture the package configuration
state and maintain a deterministic, state transition model of
the system.
 By examining the current maintainer scripts and templates
provided by deb/rpm-helper, we defined a language that
assists with problems not addressed now by meta-installers.
 Transactionaly Protected Package Management (TPPM) is
what we are aiming to move towards.
TPPM
roll-back roll-back
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 11

12. Energia Open Source
Advantages and benefits of Roll-back/TPPM
 By building a framework in which R/B + TPPM is possible we
can capture the functional elements of maintainer scripts
Error Detection
 Simulate/test the results of
an upgrade on a model of system
 Detect errors a-priori or
potential 'slow' errors after upgrades
 Detect/(correct) sequencing orders in package installations
Roll-back
 Framework → drive roll-back. Now provide more information
to a roll-back utility than available to current meta-installers
 Identify which packages can/cannot perform roll-back
 Allow individual packages to change versions and show the
dependencies for those changes
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 12

13. Energia Open Source
Domain Specific Language (DSL) -package upgrades
 Our DSL is a language used to abstract from the system and
model to solve particular, identified problems
 In our case, the DSL is focused on analysing package
maintainer scripts
 Designed not to be a Turing Complete Language like BASH
but rather something which can resolve problems
 It is an evolutionary language designed to capture the details
of the vast majority of common maintainer scripts
 Increase the coverage of DSL by examining new scripts
Many thanks to University of L'Aquila for their creation of the
language and for their support in using it
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 13

14. Energia Open Source
System Integration
 For roll-back we will have a log of
Apt the DSL commands executed in-
sequence
model_simulator (( ))
model_simulator
 Perform roll-back → run inverse
statements associated with those
DSL_rollback_pre (( ))
DSL_rollback_pre commands in 'reverse' order
 Have the simulator to pre-check if
run_transaction ()
run_transaction () the package configuration can be
rolled-back; whether it will leave it
DSL_rollback_post (( ))
in an erroneus state
DSL_rollback_post
 Performing LIFO style roll-back we
run post commands before we run
the pre statements
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 14

15. Energia Open Source
Domain Specific Language (DSL) Example
 The DSL has been created to assist with some of the
problems discovered and analysed by prior research
Using cups.spec %post example from CUPS-1.4.2
%post
 dslstart postinst_init(cups)
/sbin/chkconfig ­­add cups Matched DSL Pair
/sbin/chkconfig cups on
 dslstop postinst_init(cups)
# Restart cupsd if we are upgrading...
 dslstart post_init_restart(cups)
if test $1 ­gt 1; then
/sbin/service cups stop Matched DSL Pair
/sbin/service cups start
fi
 dslstop post_init_restart(cups)
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 15

16. Energia Open Source
Example continued, Log
rbHist dslID TID parentID DSL_CMD bhINVERSE
1 1 1 start postinst_init(cups) TRUE
2 1 1 stop postinst_init(cups) TRUE
3 1 1 start post_init_restart(cups) TRUE
4 1 1 stop post_init_restart(cups) TRUE
pkgHist id parent op pkgName pkgVer1 pkgVer2 dateTime
1 1 inst cups 0 1.4.2 2010-01-30
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 16

17. Energia Open Source
Transactions- what happens if a maintainer
script fails?
 If a maintainer script fails in the middle of one of the
operations will have a log like this
dslID TID parentID DSL_CMD bhINVERSE
1 1 1 start postinst_init(cups) TRUE
2 1 1 stop postinst_init(cups) TRUE
3 1 1 start post_init_restart(cups) TRUE
 Transaction has quite obviously failed. No matching end for a
DSL command reached. Odd number of elements etc.
 Perform a roll-back for all matching sub-transaction ID
elements, but in the reverse order with certain constraints.
 If a set of script elements cannot perform roll-back in the
middle of operating, then don't create a dsl tag.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 17

18. Energia Open Source
Performing the roll-back
 Identify the sub-transaction/package upgrade to roll-back.
 Check that all commands in the database have an inverse for
the particular transaction.
 Run them in reverse-operation order.
 dslstart postinst_init(cups) [1]
 dslstop postinst_init(cups) [2]
 dslstart post_init_restart(cups) [3] //Restart not needed postrm
becomes
 postrm_init_restart(cups) [3] //Remove non­needed element
 prerm_init_stop(cups) [4] //From lookup table.
 prerm_init(cups) [1+2]
 In other cases the order might be [3,2,1]. Identified by tags.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 18

19. Energia Open Source
Architecture
 Modified Package as input → DSL extracted → Log storage
and simulator → Mechanism for executing roll-backs →
Maintenance of scripts etc.
User I/P
Inject DSL
dslstart postinst_init(cups)
dslend postinst_init(cups) Logs
Roll-back
SQLite DB
.spec file Apt-rpm
%post
/sbin/chkconfig --add cups Modified System State
/sbin/chkconfig cups on WP2
...
Simulator
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 19

20. Energia Open Source
Overall Approach
Analyse maintainer scripts in standard systems
Identify common themes in maintainer scripts
Create first version of the DSL
Modify maintainer scripts to contain DSL
Log DSL into SQLite DB
Create roll-back mechanism
Logged Data + Stored Info → Perform Roll-back
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 20

21. Energia Open Source
A state/time perspective of roll-back
apt-get [Maintainer Scripts]
T0 install cups In terms of DSL
apt-get DSL cups pkg
rollback apt-rpm DSL stmts
Log
cups 1.2
apt-get
T0+n rollback
TID-028 [Maintainer Scripts]-1
apt-get
rollback
roll-back
2010-02-06
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 21

22. Energia Open Source
Distributing Rollback-Information
 Rollback data can be distributed
– Transceive data onto Kademlia network.
– Distribute rpms/rollback data/packages state to network
– Private data submission to network? Opt-in mechanism
Kad/P2P Network
Logs Config cups pkg
SQLite DB data
Normal upgrades
Initial System State Modified System State
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 22

23. Energia Open Source
Todo list:
 Completed:
– Single package rollback with DSL
– Supporting framework e.g. SQLite3 and script injectors.
 Now in the process of:
– Integrating Simulator/Failure Detector into package upgrade
– Generating tool suite for automatic detection/generation of DSL
commands for a given set of packages and their scripts
– Creating web interface for changing the auto-created DSL pkgs
– Completing rollback mechanism for multiple packages txns.
– Testing
– Documentation
– Release + revisions
– Integration into build systems
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 23

24. Energia Open Source
Conclusion
 The need for Roll-back of packages is growing
 Not a trivial problem
 Need to consider the problems of
complex upgrade paths
 Using the DSL we aim to remove
a lot of the failure cases in
package upgrades.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 24

25. Energia Open Source
Questions?
 Thank you for listening. I hope that it was interesting and
that you have some questions to ask.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 25