• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
The software management and engineering in the AI-oriented projects tutorial
 

The software management and engineering in the AI-oriented projects tutorial

on

  • 830 views

SDART Ltd presents a tutorial about Software Management Engineering in AI-based projects. The tutorial's aim is to familiarise participants with the problems concerning AI-based software project ...

SDART Ltd presents a tutorial about Software Management Engineering in AI-based projects. The tutorial's aim is to familiarise participants with the problems concerning AI-based software project management.

Statistics

Views

Total Views
830
Views on SlideShare
827
Embed Views
3

Actions

Likes
0
Downloads
7
Comments
0

1 Embed 3

http://www.techgig.com 3

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    The software management and engineering in the AI-oriented projects tutorial The software management and engineering in the AI-oriented projects tutorial Presentation Transcript

    • The software management and engineering in the AI-oriented projects Dr W Pietruszkiewicz SDART Ltd
    • Plan of presentation
      • Introduction
      • Software project management
      • Software design
      • AI implementation
      • Examples of AI software
    • Introduction
    • Artificial Intelligence
    • Project success
    • Research oriented software
      • Time and budget aren ’ t usually fixed – it is possible to extend the research
      • Resources are flexible
      • People mainly have the personal interests (fame, dissertations, publications …)
      • Quality (functionality) is the main concern
    • Product oriented software
      • Time and budget are usually fixed
      • Quality is adjusted to the available budget and schedule
      • People mainly have the business interests
      • Resources aren ’ t flexible
    • AI & software in projects   Proper management Struggle
    • This tutorial
      • Reasons to propose this tutorial:
      • There exist the software projects without AI but I ’ m not sure if there exist AI projects without the software (embedded count too!) …
      • The knowledge about the software management is unpopular among AI researchers
      • At SDART we deal with the AI-oriented software projects
    • This tutorial
      • Will:
        • present a selection of the popular techniques and methods for software development
        • evaluate them from AI perspective
        • propose what could be used
      • Will not:
        • say what have to be used
        • be a guide ready to be applied step-by-step
    • Software project management
    • AI & soft – PM nightmare
      • In the project management two types of projects are the symbols of unpredictability:
        • Research projects
        • … . software projects
      • What could we do dealing with an AI-oriented software projects? Professionally manage them!
    • Software development schema
    • DM management
      • Data mining management seems to be more organised than AI
      • There are different models of DM organisation:
      • CRISP-DM
      • SEMMA
      • DMAIC/DMADV
    • CRISP-DM
    • SEMMA
    • DMAIC&DMADV
    • Management model
      • Can ’ t we use any DM model to properly manage the AI&software projects? Yes we could …. only if the software does not constitute any significant part of the project. Meanwhile …
      • AI projects require software to research
      • These projects require software to use AI
    • Common risk sources
    • Risk strategies
      • Avoidance
      • Retention
      • Sharing
      • Reduction
    • Software quality
      • There are many software quality factors, among them:
    • Software quality
    • Software quality
      • If an application is a product users perceive only its „front ” parts:
      • GUI
      • Errors
      • Documentation
    • Software metrics
      • Tom DeMarco said „You can ’ t control what you can ’ t measure ”
      • Software metrics measure some properties of a software or its specification e.g. complexity, quality, efficiency or size
    • Software estimation
      • Estimation problem
      I think I ’ ve forgotten something, but what was it???
    • Methodology & Life cycle
      • Software methodology defines the structure, plan and management of developed software
      • Methodology contains development philosophy and tools, methods and models supporting the process
      • Life cycle model defines the phases and activities
    • M&LC– staged delivery Requirements Design Implementation Verification Maintenance
    • M&LC - spiral
      • Spiral model – Boehm 1988
    • M&LC – staged delivery Conception Analysis Desing Stage 1 Stage 2 Stage n
    • M&LC – design to schedule Conception Analysis Desing Stage 1 Stage 2 Stage n Software release Time? Costs?
    • M&LC - Agile
      • The Agile development is based on the iterative model
      Initial planning Planning Requirements Analysis and design Implementation Deployment Testing Evaluation
    • M&LC - XP
      • Extreme programming is an Agile implementation:
      • Programming in pairs
      • Extensive code review
      • Avoids the programming required in the future
      • Automated tasks
      • Nothing is fixed i.e. everything could be redone
    • M&LC - SCRUM
      • Project is being developed by Pigs & Chickens
      • Pig is committed to the project i.e. ScrumMaster, Team, ProductOwner
      • Chicken is involved i.e.: Stakeholder, Manager
    • M&LC - SCRUM
    • M&LC – Unified Process
      • Is an iterative software development framework
    • M&LC – UP versions
      • Rational Unifed Process
      • Agile Unified Process
      • Open Unified Process
      • Enterprise Unified Process
    • M&LC – selection
      • The factors that influences life cycle selection are:
      • Size of project
      • Understood requirements
      • Desired reliability
      • Allowed overheads
      • Progress visibility
      • Sophisticated management
      • Allowed corrections
    • M&LC - summary
      • Waterfall is still the most frequently used model by the biggest software companies
      • But they do it preparing „yet another version of X ”
      • AI projects are usually new for you as well as for everyone else, so do not expect it to be as easy as software mentioned above
    • Collaboration Homogeneous research team Heterogeneous software team
    • Project controlling
    • Software design
    • Architecture
      • Architecture is a high level planning for the software that does not involve any functional decisions
    • Views
      • The views are perspectives that are used to represent different aspect of the system:
      • Code/module view
      • Concurrency/process/thread view
      • Data view
      • Development/structural view
      • Functional/logic view
      • Physical/deployment view
      • User action/feedback view
    • Frameworks
      • Software architecture is defined by various frameworks defining its necessary elements:
      • 4+1
      • RM-ODP
      • DODAF
      • SOMF
    • Frameworks - 4+1
    • Design concepts
      • Abstraction & refinement – generalisation
      • Modularity
      • Hierarchy
      • Information & functionality enclosure
      • Structural partitioning – vertical & horizontal
    • UML
      • Unified Modeling Language:
      • Object oriented
      • Unified Booch, OOSE, OMT modelling methods
      • Currently in 2.2 version – the most of software supports 2.0
    • UML Diagrams
    • UML Advantages
      • UML became an industry standard due to:
      • Visualisation
      • Documenting
      • Specifying
      • Constructing
    • UML Tools
      • Papyrus – http://www.papyrusuml.org
      • Poseidon for UML - http://www.gentleware.com
      • Visual Paradigm for UML - http://www.visual-paradigm.com
      • Umbrello UML Modeller – http://uml.sourceforge.net
      • StarUML – http://staruml.sourceforge.net
    • Supporting tools
      • Let ’ s take a look at the supporting tools
      • If something could be done simpler and at least with the same quality
      • Use them and forget the time-consuming repetive tasks and focus on the difficult ones
    • Version controlling
      • How to avoid the mess dealing with the different versions of files in projects?
      • The source code is constantly changing …
      • The AI files too …
    • Version controlling
      • Version controlling allows the development team to manage different versions of software. They keep track of work and allow the team to look at the project ’ s past.
      • There are 3 kinds of version controlling systems:
      • Local
      • Distributed
      • Client-server
    • Version controlling – local systems
      • Revision Control System
      • Source Conde Control System
      • MKS Implementer
      • History Explorer
    • Version controlling - distributed
      • DCVS (Distributed CVS)
      • Monotone
      • GNU arch
      • Plastic SCM
      • BitKeeper
      • TeamWare
      PC Rep PC Rep PC Rep
    • Version controlling – client/server
      • CVS
      • SVN
      • Codendi
      • ClearCase
      • Visual SourceSafe
      • Autodesk Valut
      Client Client Client Server Rep
    • Supporting tools & technologies
    • Automated builds - advantages
      • Efficiency increase
      • Development process simplification
      • Increased software quality
      • Reduced costs
      • False version removal
      • Reduction of the development team or task relocation
      • Project archiving
    • Automated builds
      • The automated build tools include:
      • make
      • Maven
      • Ant
      • MSBuild
      • Cmake
      • and many others
    • Automated testing - make An example of makefile taken from QT library. PNGMAK=scripts/makefile.linux all: cd zlib; ./configure; make cd libpng; make -f $(PNGMAK) clean: -cd zlib; make clean -cd libpng; make -f $(PNGMAK) clean
    • Automated testing - Ant <project name=&quot;MyProject&quot; default=&quot;dist&quot; basedir=&quot;.&quot;> <description>simple example build file</description> <!-- set global properties for this build --> <property name=&quot;src&quot; location=&quot;src&quot;/> <property name=&quot;build&quot; location=&quot;build&quot;/> <property name=&quot;dist&quot; location=&quot;dist&quot;/> <target name=&quot;init&quot;> <!-- Create the time stamp --> <tstamp/> <!-- Create the build directory structure used by compile --> <mkdir dir=&quot;${build}&quot;/> </target> <target name=&quot;compile&quot; depends=&quot;init ” description=&quot;compile the source &quot; > <!-- Compile the java code from ${src} into ${build} --> <javac srcdir=&quot;${src}&quot; destdir=&quot;${build}&quot;/> </target> <target name=&quot;dist&quot; depends=&quot;compile ” description=&quot;generate the distribution&quot; > <!-- Create the distribution directory --> <mkdir dir=&quot;${dist}/lib&quot;/> <!-- Put everything in ${build} into the MyProject-${DSTAMP}.jar file --> <jar jarfile=&quot;${dist}/lib/MyProject-${DSTAMP}.jar&quot; basedir=&quot;${build}&quot;/> </target> <target name=&quot;clean ” description=&quot;clean up&quot; > <!-- Delete the ${build} and ${dist} directory trees --> <delete dir=&quot;${build}&quot;/> <delete dir=&quot;${dist}&quot;/> </target> </project>
    • Automated testing - Maven <project xmlns=&quot;http://maven.apache.org/POM/4.0.0&quot; mlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot; xsi:schemaLocation=&quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd&quot;> <modelVersion>4.0.0</modelVersion> <groupId>com.mycompany.app</groupId> <artifactId>my-app</artifactId> <packaging>jar</packaging> <version>1.0-SNAPSHOT</version> <name>Maven Quick Start Archetype</name> <url>http://maven.apache.org</url> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> </dependencies> </project>
    • Automated testing - unit The most popular unit testing tool is xUnit available in many versions for all popular languages Test A Test ABCD vs Test B Test C Test D Data Data Data Data Data
    • Automated testing - system
      • Automated GUI tests perceive the application as a black-box and test it from the user ’ s perspective.
    • Code documenting
      • Automatic API documentation tools:
      • JavaDoc
      • DoxyGen
      • RoboDoc
      Manual documentation Automatic documentation Code Doc Code Doc
    • Bug tracking
      • Issue/Bug Tracking Systems usage leads to:
      • Better quality control
      • Gaining users ’ feedback
      • Better resource allocation
      • Increased users ’ comfort
      • Better information flow
      • Feature request available
    • Bug tracking – bug categories
    • Bug tracking
      • Bugzilla
      • Flyspray
      • Mantis
      • Trac
      • Redmine
    • Bug tracking
    • AI implementation
    • Implementation
      • Transforms the design (plan) into the code
      • Problems:
        • Technology
        • Programming language
    • Levels of programming languages
    • 1st generation
      • The code lines are equal to the processor instructions – one to one
      • Machine code
    • 2nd generation
      • Symbolic representation of processor ’ s instructions
      • Assembler language
    • 3rd generation
      • Instructions represent the basic operations
      • Oriented on programmer
      • Most popular generation
    • 4th generation
      • Specialised languages oriented on advanced operations
      • RAD tools are 4GL build over 3GL
      Matlab disp('Hello, world!') SAS data _null_; put 'Hello, world!'; run; Postscript (Hello, world!n) print
    • 5th generation
      • Represent not algorithms but desired outcome
      • Logical programming
      • Programming for non-programmers
      Common Lisp (write-line &quot;Hello, world!&quot;) Prolog :- write('Hello, world!'),nl.
    • Hybrid programming – embedding
      • Hybrid programming (embedding languages) could be synergic but:
      • Increases the technological risk
      • Increases the demand on skilled developers
      • Blurs the project
    • Programming paradigms
      • Paradigm is a fundamental style of computer programming and defines the concept of programming language
      • The most of programming languages support various paradigms (different approaches to problem)
    • Programming paradigms
      • The most popular paradigms are:
      • Agent-oriented
      • Declarative
      • Event-driven
      • Imperative
      • Metaprogramming
    • Desktop applications
      • C/C++
      • Java
      • C#
      • Python
    • Web services – server side
      • Java EE
      • PHP
      • Python
      • Ruby
      • Perl
      • .NET
      • CGI (every language that could be run by server)
    • Web services - RIAs
      • Flash
      • AJAX
      • Java SE
      • JavaFX
      • FLEX
      • OpenLaszlo
      • Silverlight
    • Mobile applications
      • Java ME
      • C++
      • Objective-C
    • Languages popularity http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
    • Characteristics - domain
      • Specialised – good at something but below average on other things
      • General purpose – no specific domain and a compromise on all tasks
    • Characteristics - speed
      • Compiled faster
      • Interpreted slower
      • But:
      • Compiled start-up slower
      • Interpreted start-up faster (write-run-test cycle)
      • JIT compilers for interpreted languages
    • Characteristics - portability
      • Software could be portable on two levels:
      • Source code
      • Binary code
      • Portability does not relate only to the programming language but also to used libraries
    • Characteristics - prototyping
      • Ability to verify the assumption:
      • Dynamic language
      • Simple syntax
      • Efficiency isn ’ t a main concern
      • Checks if the idea works
    • Characteristics – target platform
      • Target platform:
      • PC
      • Mobile
      • Web
    • Computer languages Scientific perspective Programming perspective
    • Why use a specialised languages
      • Popular in the research community
      • Specialized languages
      • Many libraries/toolkits and examples are available
    • Why not use them
      • Specialised not universal
      • High price
      • Simple or no GUI
      • No networking
      • Research not production languges
    • Important programming features
      • Dynamic languages:
      • Do not care about types of data
      • Focus on the algorithms
      • But reject a full data control
    • Important programming features
      • Prototyping:
      • Checks algorithms
      • Easily visualise data or make GUI
      • Speed is measured as the time of coding not running
    • Important programming features
      • Scientific libraries:
      • How often do we construct new algorithms (Charlie Eppes Syndrome – NUMB3RS)?
      • The application is more like a large panel building than brick building – applies to the AI problems as well as to the „ordinary ” soft
    • Important programming features
      • Visual design (graphical programming):
      • GUI could be constructed using components
      • Link the components visually
      • Evaluate application faster
    • More important part of AI software Knowledge Models Rules Graphs
    • Hybrid software development Research Development Idea Product Replanning & redesign Technology X Technology X Technology Y ?? ??? ??
    • Programming libraries – C/C++
      • MLC++ http://www.sgi.com/tech/mlc/
      • FANN http://leenissen.dk/fann/
      • Waffles http://waffles.sourceforge.net/
      • OpenCV ML http://opencv.willowgarage.com/documentation/cpp/ml._machine_learning.html
      • dlib C++ http://dlib.net/
      • VFML http://www.cs.washington.edu/dm/vfml/
    • Programming libraries – Java
      • Java-ML http://java-ml.sourceforge.net/
      • Weka Engine http://www.cs.waikato.ac.nz/ml/weka/
      • MLJ http://sourceforge.net/projects/mldev/
      • MALLET http://mallet.cs.umass.edu/
      • LIBSVM http://www.csie.ntu.edu.tw/~cjlin/libsvm/
    • Programming libraries – C#
      • Infer.NET http://research.microsoft.com/en-us/um/cambridge/projects/infernet/
      • AForge.NET http://code.google.com/p/aforge/
      • C# Neural Network Library http://franck.fleurey.free.fr/NeuralNetwork/
      • NeuronDotNet http://neurondotnet.freehostia.com/
      • NeuroBox Neural Network Library http://www.cdrnet.net/projects/neuro/
    • Programming libraries – Python
      • Orange engine http://www.ailab.si/orange/
      • PyML http://pyml.sourceforge.net/
      • MDP http://mdp-toolkit.sourceforge.net/
      • PyBrain http://www.pybrain.org/
      • LIBSVM http://www.csie.ntu.edu.tw/~cjlin/libsvm/
    • Programming libraries – PHP
      • FANN PHP Extension http://fann.sourceforge.net/fann.html
      • Tremani Neural Network http://www.tremani.nl/open-source/neural-network/
      • ANN http://ann.thwien.de/index.php/Main_Page
    • Example I – Java
      • A simple data mining task
      • Uses Java-ML library
    • Example II – Python
      • Adaptive filtering algorithm
      • Use as a web service (server part)
      • Use mod_python on Apache
    • Example III – Flex
      • Particle Swarm Optimisation example
      • RIA application
      • Uses a standard Flex without any additional libraries
    • Thank you. Any questions? Get in touch: www.sdart.co.uk