A package system for maintaining large model distributions in vle software

  • 162 views
Uploaded on

Presentation delivered at the 3rd IEEE Track on …

Presentation delivered at the 3rd IEEE Track on
Collaborative Modeling & Simulation - CoMetS'12.

Please see http://www.sel.uniroma2.it/comets12/ for further details.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
162
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A Package System for Maintaining Large Model Distributions in VLE Software WETICE/Comets 2012 Gauthier Quesnel, Ronan Tr´pos e INRA - French National Institute for Agricultural Research MIA - The Division of Applied Mathematics and Computer Sciences BIA - Biometrics and Artificial Intelligence UnitQuesnel et al. (INRA) Package system for VLE 1 / 32
  • 2. 1 Context2 Problem3 Package system4 Results5 Conclusions and perspectives Quesnel et al. (INRA) Package system for VLE 2 / 32
  • 3. VLEVirtual Laboratory EnvironmentA multi-modeling and simulation environment: A DEVS simulation engine A command line interface : VLE A modeling graphical user interface: GVLE A python package to develop web services: PyVLE A R package to analyze and visualize simulations results: RVLE VLE is developed with the support of: INRA, France Computer Science Laboratory, Calais, France CIRAD, Montpellier, France Computer Science Laboratory, University of Mons-Hainaut, BelgiumSince 2007, VLE has more and more users and developers Quesnel et al. (INRA) Package system for VLE 3 / 32
  • 4. RECORD, an INRA project An open platform for modeling and simulation of agro-ecosystems Project initiated in 2006 which relies on VLE software That makes multi-disciplinary work easier Capacity to take into account all the different elements of cropping systems Each element can be expressed in the most suited mathematical formalism Target: all researchers and engineers at INRA that use dynamic systems Quesnel et al. (INRA) Package system for VLE 4 / 32
  • 5. Need to improve the collaboration Needs for a collaboration system due to the heterogeneity of domains involved: agronomy, economists, statistician, computer scientists, . . . Needs in sharing models; eg. integration of models at the plot scale to little territories scale (thousands of plots) Needs in distributing existing models; eg. Stics (more than 25000 lines of fortran code) Needs in helping the collaboration between modelers. More and more people trained to the platform through 3 days practice sessions (2 per year, about 200 persons trained since 2007) and access to an E-learning web site Quesnel et al. (INRA) Package system for VLE 5 / 32
  • 6. 1 Context2 Problem3 Package system4 Results5 Conclusions and perspectives Quesnel et al. (INRA) Package system for VLE 6 / 32
  • 7. 1 Context2 Problem3 Package system4 Results5 Conclusions and perspectives Quesnel et al. (INRA) Package system for VLE 7 / 32
  • 8. How to develop models ?DEVS Initiated in 1976 by B. Zeigler to model and simulate dynamic systems A discrete event formalism: the events drive the simulation Modular and hierarchical approach: two entities atomic and coupled models Formal approach for model coupling: The closed under coupling property: a coupled model is equivalent to an atomic Abstract simulators (algorithms) to develop simulation tools: One simulator for driving one atomic model One coordinator for driving one coupled model Extensible: DS-DEVS: dynamic-structure DEVS, to add or remove, models or connections at run-time Cell-DEVS, Fuzzy-DEVS, RT-DEVS etc. Quesnel et al. (INRA) Package system for VLE 8 / 32
  • 9. Atomic model definition (DEVS) M = X , Y , S, δext , δint , δcon , ta, λWhere: δext , δint , δcon , ta, λ are functions, S the set of statesCoupled model definition (DEVS) N = X , Y , D, EIC , EOC , ICExecutive model definition (DS-DEVS) Mχ = Xχ , Yχ , Sχ , γ, Σ∗ , δintχ , δextχ , δconχ , λχ , τχWhere: γ and Σ∗ are structure change functions Quesnel et al. (INRA) Package system for VLE 9 / 32
  • 10. How to develop models ?VLE VLE provides C++ shared library which embeds the DEVS simulation kernel, thus: users have to develop C++ class to develop atomic (DEVS) or executive models (DS-DEVS), or sub-formalismsclass Dynamics{public : virtual Time timeAdvance () const ; virtual void i n t e r n a l T r a n s i t i o n ( const Time & time ); virtual void e x t e r n a l T r a n s i t i o n ( const Time & time , const E x t e r n a l E v e n t L i s t & lst ); virtual void output ( const Time & time , E x t e r n a l E v e n t L i s t & out ) const ; virtual void c o n f l u e n t T r a n s i t i o n ( const Time & time , const E x t e r n a l E v e n t L i s t & lst ); virtual Value * observation ( const O b s e r v a t i o n Ev e n t & event ) const ;} Quesnel et al. (INRA) Package system for VLE 10 / 32
  • 11. How to develop models ?Sub-formalismsDEVS allows to develop sub-formalism of DEVS. With VLE, we provide: dynamics which encapsulate mathematical formalisms such as differential equations, difference equations, Petri nets, etc.structurals which manipulates the structure of the model: cellular automaton, graph of communications, etc. Dynamic sub-formalisms are atomic models with a specified behavior: we provide transition functions Structural sub-formalisms are atomic models and executive models of DS-DEVS Quesnel et al. (INRA) Package system for VLE 11 / 32
  • 12. How to develop models ?For sub-formalisms We provide simplified API (object oriented development) We provide C++ generatorsFor example, with the difference equation sub-formalism (which inheritsthe previous Dynamics class), only the compute function is available:class MyModel : public vle :: extension :: D i f f e r e n c e E q u a t i o n{ [...] virtual double compute ( const Time & time ) { x = y ( -1) + z ( -1); y = z ( -1); }}; Quesnel et al. (INRA) Package system for VLE 12 / 32
  • 13. How to develop models ?GVLE: GUI of VLE, an IDETo develop source code, the structure of the model, to assign initialconditions, observations, to define experimental frames, etc.: Quesnel et al. (INRA) Package system for VLE 13 / 32
  • 14. The problemContextModelers develop models in C++ and can reuse models from othermodelersModels are stored into shared library. VLE loads and extracts atomicmodel from shared libraryProblemsWe clearly identify a problem of sharing and reusing code: VLE C++ API / ABI version Sub-formalism C++ API / ABI version Model change version (for example: the input or output DEVS ports)SolutionTo improve collaborative work under VLE we develop a package system Quesnel et al. (INRA) Package system for VLE 14 / 32
  • 15. 1 Context2 Problem3 Package system4 Results5 Conclusions and perspectives Quesnel et al. (INRA) Package system for VLE 15 / 32
  • 16. Package systemWe identify the following parts to develop Define the format of VLE’s package Develop and upgrade the tools and libraries of VLE (VLE, GVLE, PyVLE and RVLE) to take into account packages: Open, install, read, compile, check, remove packages Develop internet access from VLE to download and install packages from distant distributions of packages Add a checking consistency onto distributions and user’s installationThis work is inspired from the FOSS (Free Open Source Software)community Quesnel et al. (INRA) Package system for VLE 16 / 32
  • 17. What’s a package in VLEWe need to share with the package system: Data: eg. climate data files etc. External shared library: eg. scientific libraries Sub-formalism libraries: structurals and dynamics Models: classes inherit from atomic, executive or sub-formalisms classes GUI plug-ins: to improve GVLE with new features Output plug-ins: specific output simulation, database etc. DocumentationPackage must have a version number, an unique name and a list ofrun-time and build dependencies Quesnel et al. (INRA) Package system for VLE 17 / 32
  • 18. What’s a package in VLEFinite State automatonThe vle.extension.fsa package provides: Harel’s state-chart, Mealy, Moore and FDDevs sub-formalisms A graphical user interface to draw automaton an generate C++For the user of the package: Shared library which stores binary code of sub-formalisms C++ Headers of the sub-formalisms GUI plug-insThe vle.extension.fsa package, only depends of VLE 1.1 Quesnel et al. (INRA) Package system for VLE 18 / 32
  • 19. Example: FSA packageGUI to develop automaton Quesnel et al. (INRA) Package system for VLE 19 / 32
  • 20. Example: FSA packageGenerated C++ source code Quesnel et al. (INRA) Package system for VLE 20 / 32
  • 21. What’s a package in VLEThe content and the structure of a packagepackage / data / # data ( climate data file ) exp / # experiment file ( structure of DEVS model ) src / # source code test / # unit test lib / # shared sub - formalism libraries plugins / # simulators , GUI plug - ins shared libraries Authors . txt Description . txt # the most important file License . txt A source package does not have the lib and plugins directories A binary package does not have the src and test directories Quesnel et al. (INRA) Package system for VLE 21 / 32
  • 22. What’s a package in VLEThe description fileSource : weather - genVersion : 1.0.0 -0Section : agronomicMaintainer : quesnel@users . sourceforge . netHomepage : http :// www . vle - project . orgBuild - Depends : vle ( >= 1.1) , differential - equation ( >= 1.0)Depends : weather (= 1.2.3) , output - file ( >= 1.0)Conflicts :Description : A weather generator . The build-dependencies are required to build and compile the package The dependencies are required at run-time of the simulation The conflicts package must be removed before the compilation of the package or at run-time of the simulation Quesnel et al. (INRA) Package system for VLE 22 / 32
  • 23. What’s a package in VLEAn example of package dependencies with an agronomic model DifferenceEquation, Depends and build-depends of FSA, Agent, PRNG maize-optim package are sub-formalisms: indirect build-depends packages for maize-optim Optim is sub-formalism: direct build-depend package for maize-optim Other packages are run-time depends Quesnel et al. (INRA) Package system for VLE 23 / 32
  • 24. DistributionDefinition The model developers build models or build sub-formalisms and make upstream packages The package maintainers get, check, fix upstream packages and pull it into distribution The distribution stores all packages. A consistency check is automatically done before adding or removing a package. Users of the distribution user community can download and install distribution’s packages Users can be connected to several distributions Distributions are accessible via http or ftp protocols Quesnel et al. (INRA) Package system for VLE 24 / 32
  • 25. Distribution: Workflow Quesnel et al. (INRA) Package system for VLE 25 / 32
  • 26. Distribution: Workflow Quesnel et al. (INRA) Package system for VLE 25 / 32
  • 27. Distribution: Workflow Quesnel et al. (INRA) Package system for VLE 25 / 32
  • 28. Distribution: Workflow Quesnel et al. (INRA) Package system for VLE 25 / 32
  • 29. How to check package consistency ?A distribution allows users to install and uninstall packages from a distantrepository A distribution is a finite set of packages rules Each package rule is a tuple of the form (p, B, D, C ): where: p is a package, B is a set a dependency build clauses for p, D is a set of dependency clauses for p, C is a set of conflict clauses for pThe dependency build clauses and dependency clauses must be present tocompile and use the package pThe dependency build packages can be removed after the installation Quesnel et al. (INRA) Package system for VLE 26 / 32
  • 30. How to check package consistency ? Each dependency clause or dependency build clause is a disjunction of packages p1 |...|pk A dependency clause requires that some packages from the set {p1 , ..., pk } to be present Conflict clause requires that package p is not present in order to install the package p Thus, a valid installation is a subset of the packages in the distribution An installation profile is valid if dependency clauses and conflict clauses are satisfied A valid installation profile for a distribution should satisfy the dependency and conflict clauses of each package rule of the distribution Quesnel et al. (INRA) Package system for VLE 27 / 32
  • 31. How to check package consistency ?The package system has two critical problems: To install the new package p given a distribution R and an installation profile P, the package system has to check if there is a profile P containing p such that P ∪ P is a valid installation profile for R It may happen that a new package p cannot be installed because it is in conflict with some packages already installed on the machine. In this case, we must first uninstall some packages before attempting to install p. The task of the package system is then to identify a set of packages P such that p can be installed on P − PTo address these two problems, we can use a satisfiability problem solver(SAT) or a constraint satisfaction problem (CSP) Quesnel et al. (INRA) Package system for VLE 28 / 32
  • 32. 1 Context2 Problem3 Package system4 Results5 Conclusions and perspectives Quesnel et al. (INRA) Package system for VLE 29 / 32
  • 33. Results The major parts of the package system has been developed The major packages of VLE are converted into packages (near 20 packages) RECORD project is converting its packages in new VLE packages (near 100 packages) Package system is working to download and install packages from a server All tools of VLE are converted to package systems: VLE, GVLE, RVLE and PyVLE However, The package system does not embed a complete solver yet The distribution package does not provide consistency checkingThe package system of VLE is available. VLE and RECORD users use it ! Quesnel et al. (INRA) Package system for VLE 30 / 32
  • 34. 1 Context2 Problem3 Package system4 Results5 Conclusions and perspectives Quesnel et al. (INRA) Package system for VLE 31 / 32
  • 35. Conclusions and perspectivesA success: Users are happy to use the package system to share and reuse model, sub-formalism and data The use of package seems to simplify the construction of complex models by aggregating atomic models and data from other packages Users intensively use generic models in Weather or Glue packages: these packages are now robust Package system improves the global quality of the models and simplify the maintenance of modelsHowever,The package system needs to have computer engineers: To integrate and maintain packages of the community To perform release of the distribution Quesnel et al. (INRA) Package system for VLE 32 / 32