Your SlideShare is downloading. ×
The software management and engineering in the AI-oriented projects tutorial
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

The software management and engineering in the AI-oriented projects tutorial

822
views

Published on

SDART Ltd presents a tutorial about Software Management Engineering in AI-based projects. The tutorial's aim is to familiarise participants with the problems concerning AI-based software project …

SDART Ltd presents a tutorial about Software Management Engineering in AI-based projects. The tutorial's aim is to familiarise participants with the problems concerning AI-based software project management.

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
822
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
60
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. The software management and engineering in the AI-oriented projects Dr W Pietruszkiewicz SDART Ltd
  • 2. Plan of presentation
    • Introduction
    • Software project management
    • Software design
    • AI implementation
    • Examples of AI software
  • 3. Introduction
  • 4. Artificial Intelligence
  • 5. Project success
  • 6. Research oriented software
    • Time and budget aren ’ t usually fixed – it is possible to extend the research
    • Resources are flexible
    • People mainly have the personal interests (fame, dissertations, publications …)
    • Quality (functionality) is the main concern
  • 7. Product oriented software
    • Time and budget are usually fixed
    • Quality is adjusted to the available budget and schedule
    • People mainly have the business interests
    • Resources aren ’ t flexible
  • 8. AI & software in projects   Proper management Struggle
  • 9. This tutorial
    • Reasons to propose this tutorial:
    • There exist the software projects without AI but I ’ m not sure if there exist AI projects without the software (embedded count too!) …
    • The knowledge about the software management is unpopular among AI researchers
    • At SDART we deal with the AI-oriented software projects
  • 10. This tutorial
    • Will:
      • present a selection of the popular techniques and methods for software development
      • evaluate them from AI perspective
      • propose what could be used
    • Will not:
      • say what have to be used
      • be a guide ready to be applied step-by-step
  • 11. Software project management
  • 12. AI & soft – PM nightmare
    • In the project management two types of projects are the symbols of unpredictability:
      • Research projects
      • … . software projects
    • What could we do dealing with an AI-oriented software projects? Professionally manage them!
  • 13. Software development schema
  • 14. DM management
    • Data mining management seems to be more organised than AI
    • There are different models of DM organisation:
    • CRISP-DM
    • SEMMA
    • DMAIC/DMADV
  • 15. CRISP-DM
  • 16. SEMMA
  • 17. DMAIC&DMADV
  • 18. Management model
    • Can ’ t we use any DM model to properly manage the AI&software projects? Yes we could …. only if the software does not constitute any significant part of the project. Meanwhile …
    • AI projects require software to research
    • These projects require software to use AI
  • 19. Common risk sources
  • 20. Risk strategies
    • Avoidance
    • Retention
    • Sharing
    • Reduction
  • 21. Software quality
    • There are many software quality factors, among them:
  • 22. Software quality
  • 23. Software quality
    • If an application is a product users perceive only its „front ” parts:
    • GUI
    • Errors
    • Documentation
  • 24. Software metrics
    • Tom DeMarco said „You can ’ t control what you can ’ t measure ”
    • Software metrics measure some properties of a software or its specification e.g. complexity, quality, efficiency or size
  • 25. Software estimation
    • Estimation problem
    I think I ’ ve forgotten something, but what was it???
  • 26. Methodology & Life cycle
    • Software methodology defines the structure, plan and management of developed software
    • Methodology contains development philosophy and tools, methods and models supporting the process
    • Life cycle model defines the phases and activities
  • 27. M&LC– staged delivery Requirements Design Implementation Verification Maintenance
  • 28. M&LC - spiral
    • Spiral model – Boehm 1988
  • 29. M&LC – staged delivery Conception Analysis Desing Stage 1 Stage 2 Stage n
  • 30. M&LC – design to schedule Conception Analysis Desing Stage 1 Stage 2 Stage n Software release Time? Costs?
  • 31. M&LC - Agile
    • The Agile development is based on the iterative model
    Initial planning Planning Requirements Analysis and design Implementation Deployment Testing Evaluation
  • 32. M&LC - XP
    • Extreme programming is an Agile implementation:
    • Programming in pairs
    • Extensive code review
    • Avoids the programming required in the future
    • Automated tasks
    • Nothing is fixed i.e. everything could be redone
  • 33. M&LC - SCRUM
    • Project is being developed by Pigs & Chickens
    • Pig is committed to the project i.e. ScrumMaster, Team, ProductOwner
    • Chicken is involved i.e.: Stakeholder, Manager
  • 34. M&LC - SCRUM
  • 35. M&LC – Unified Process
    • Is an iterative software development framework
  • 36. M&LC – UP versions
    • Rational Unifed Process
    • Agile Unified Process
    • Open Unified Process
    • Enterprise Unified Process
  • 37. M&LC – selection
    • The factors that influences life cycle selection are:
    • Size of project
    • Understood requirements
    • Desired reliability
    • Allowed overheads
    • Progress visibility
    • Sophisticated management
    • Allowed corrections
  • 38. M&LC - summary
    • Waterfall is still the most frequently used model by the biggest software companies
    • But they do it preparing „yet another version of X ”
    • AI projects are usually new for you as well as for everyone else, so do not expect it to be as easy as software mentioned above
  • 39. Collaboration Homogeneous research team Heterogeneous software team
  • 40. Project controlling
  • 41. Software design
  • 42. Architecture
    • Architecture is a high level planning for the software that does not involve any functional decisions
  • 43. Views
    • The views are perspectives that are used to represent different aspect of the system:
    • Code/module view
    • Concurrency/process/thread view
    • Data view
    • Development/structural view
    • Functional/logic view
    • Physical/deployment view
    • User action/feedback view
  • 44. Frameworks
    • Software architecture is defined by various frameworks defining its necessary elements:
    • 4+1
    • RM-ODP
    • DODAF
    • SOMF
  • 45. Frameworks - 4+1
  • 46. Design concepts
    • Abstraction & refinement – generalisation
    • Modularity
    • Hierarchy
    • Information & functionality enclosure
    • Structural partitioning – vertical & horizontal
  • 47. UML
    • Unified Modeling Language:
    • Object oriented
    • Unified Booch, OOSE, OMT modelling methods
    • Currently in 2.2 version – the most of software supports 2.0
  • 48. UML Diagrams
  • 49. UML Advantages
    • UML became an industry standard due to:
    • Visualisation
    • Documenting
    • Specifying
    • Constructing
  • 50. UML Tools
    • Papyrus – http://www.papyrusuml.org
    • Poseidon for UML - http://www.gentleware.com
    • Visual Paradigm for UML - http://www.visual-paradigm.com
    • Umbrello UML Modeller – http://uml.sourceforge.net
    • StarUML – http://staruml.sourceforge.net
  • 51. Supporting tools
    • Let ’ s take a look at the supporting tools
    • If something could be done simpler and at least with the same quality
    • Use them and forget the time-consuming repetive tasks and focus on the difficult ones
  • 52. Version controlling
    • How to avoid the mess dealing with the different versions of files in projects?
    • The source code is constantly changing …
    • The AI files too …
  • 53. Version controlling
    • Version controlling allows the development team to manage different versions of software. They keep track of work and allow the team to look at the project ’ s past.
    • There are 3 kinds of version controlling systems:
    • Local
    • Distributed
    • Client-server
  • 54. Version controlling – local systems
    • Revision Control System
    • Source Conde Control System
    • MKS Implementer
    • History Explorer
  • 55. Version controlling - distributed
    • DCVS (Distributed CVS)
    • Monotone
    • GNU arch
    • Plastic SCM
    • BitKeeper
    • TeamWare
    PC Rep PC Rep PC Rep
  • 56. Version controlling – client/server
    • CVS
    • SVN
    • Codendi
    • ClearCase
    • Visual SourceSafe
    • Autodesk Valut
    Client Client Client Server Rep
  • 57. Supporting tools & technologies
  • 58. Automated builds - advantages
    • Efficiency increase
    • Development process simplification
    • Increased software quality
    • Reduced costs
    • False version removal
    • Reduction of the development team or task relocation
    • Project archiving
  • 59. Automated builds
    • The automated build tools include:
    • make
    • Maven
    • Ant
    • MSBuild
    • Cmake
    • and many others
  • 60. Automated testing - make An example of makefile taken from QT library. PNGMAK=scripts/makefile.linux all: cd zlib; ./configure; make cd libpng; make -f $(PNGMAK) clean: -cd zlib; make clean -cd libpng; make -f $(PNGMAK) clean
  • 61. Automated testing - Ant <project name=&quot;MyProject&quot; default=&quot;dist&quot; basedir=&quot;.&quot;> <description>simple example build file</description> <!-- set global properties for this build --> <property name=&quot;src&quot; location=&quot;src&quot;/> <property name=&quot;build&quot; location=&quot;build&quot;/> <property name=&quot;dist&quot; location=&quot;dist&quot;/> <target name=&quot;init&quot;> <!-- Create the time stamp --> <tstamp/> <!-- Create the build directory structure used by compile --> <mkdir dir=&quot;${build}&quot;/> </target> <target name=&quot;compile&quot; depends=&quot;init ” description=&quot;compile the source &quot; > <!-- Compile the java code from ${src} into ${build} --> <javac srcdir=&quot;${src}&quot; destdir=&quot;${build}&quot;/> </target> <target name=&quot;dist&quot; depends=&quot;compile ” description=&quot;generate the distribution&quot; > <!-- Create the distribution directory --> <mkdir dir=&quot;${dist}/lib&quot;/> <!-- Put everything in ${build} into the MyProject-${DSTAMP}.jar file --> <jar jarfile=&quot;${dist}/lib/MyProject-${DSTAMP}.jar&quot; basedir=&quot;${build}&quot;/> </target> <target name=&quot;clean ” description=&quot;clean up&quot; > <!-- Delete the ${build} and ${dist} directory trees --> <delete dir=&quot;${build}&quot;/> <delete dir=&quot;${dist}&quot;/> </target> </project>
  • 62. Automated testing - Maven <project xmlns=&quot;http://maven.apache.org/POM/4.0.0&quot; mlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot; xsi:schemaLocation=&quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd&quot;> <modelVersion>4.0.0</modelVersion> <groupId>com.mycompany.app</groupId> <artifactId>my-app</artifactId> <packaging>jar</packaging> <version>1.0-SNAPSHOT</version> <name>Maven Quick Start Archetype</name> <url>http://maven.apache.org</url> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> </dependencies> </project>
  • 63. Automated testing - unit The most popular unit testing tool is xUnit available in many versions for all popular languages Test A Test ABCD vs Test B Test C Test D Data Data Data Data Data
  • 64. Automated testing - system
    • Automated GUI tests perceive the application as a black-box and test it from the user ’ s perspective.
  • 65. Code documenting
    • Automatic API documentation tools:
    • JavaDoc
    • DoxyGen
    • RoboDoc
    Manual documentation Automatic documentation Code Doc Code Doc
  • 66. Bug tracking
    • Issue/Bug Tracking Systems usage leads to:
    • Better quality control
    • Gaining users ’ feedback
    • Better resource allocation
    • Increased users ’ comfort
    • Better information flow
    • Feature request available
  • 67. Bug tracking – bug categories
  • 68. Bug tracking
    • Bugzilla
    • Flyspray
    • Mantis
    • Trac
    • Redmine
  • 69. Bug tracking
  • 70. AI implementation
  • 71. Implementation
    • Transforms the design (plan) into the code
    • Problems:
      • Technology
      • Programming language
  • 72. Levels of programming languages
  • 73. 1st generation
    • The code lines are equal to the processor instructions – one to one
    • Machine code
  • 74. 2nd generation
    • Symbolic representation of processor ’ s instructions
    • Assembler language
  • 75. 3rd generation
    • Instructions represent the basic operations
    • Oriented on programmer
    • Most popular generation
  • 76. 4th generation
    • Specialised languages oriented on advanced operations
    • RAD tools are 4GL build over 3GL
    Matlab disp('Hello, world!') SAS data _null_; put 'Hello, world!'; run; Postscript (Hello, world!n) print
  • 77. 5th generation
    • Represent not algorithms but desired outcome
    • Logical programming
    • Programming for non-programmers
    Common Lisp (write-line &quot;Hello, world!&quot;) Prolog :- write('Hello, world!'),nl.
  • 78. Hybrid programming – embedding
    • Hybrid programming (embedding languages) could be synergic but:
    • Increases the technological risk
    • Increases the demand on skilled developers
    • Blurs the project
  • 79. Programming paradigms
    • Paradigm is a fundamental style of computer programming and defines the concept of programming language
    • The most of programming languages support various paradigms (different approaches to problem)
  • 80. Programming paradigms
    • The most popular paradigms are:
    • Agent-oriented
    • Declarative
    • Event-driven
    • Imperative
    • Metaprogramming
  • 81. Desktop applications
    • C/C++
    • Java
    • C#
    • Python
  • 82. Web services – server side
    • Java EE
    • PHP
    • Python
    • Ruby
    • Perl
    • .NET
    • CGI (every language that could be run by server)
  • 83. Web services - RIAs
    • Flash
    • AJAX
    • Java SE
    • JavaFX
    • FLEX
    • OpenLaszlo
    • Silverlight
  • 84. Mobile applications
    • Java ME
    • C++
    • Objective-C
  • 85. Languages popularity http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html
  • 86. Characteristics - domain
    • Specialised – good at something but below average on other things
    • General purpose – no specific domain and a compromise on all tasks
  • 87. Characteristics - speed
    • Compiled faster
    • Interpreted slower
    • But:
    • Compiled start-up slower
    • Interpreted start-up faster (write-run-test cycle)
    • JIT compilers for interpreted languages
  • 88. Characteristics - portability
    • Software could be portable on two levels:
    • Source code
    • Binary code
    • Portability does not relate only to the programming language but also to used libraries
  • 89. Characteristics - prototyping
    • Ability to verify the assumption:
    • Dynamic language
    • Simple syntax
    • Efficiency isn ’ t a main concern
    • Checks if the idea works
  • 90. Characteristics – target platform
    • Target platform:
    • PC
    • Mobile
    • Web
  • 91. Computer languages Scientific perspective Programming perspective
  • 92. Why use a specialised languages
    • Popular in the research community
    • Specialized languages
    • Many libraries/toolkits and examples are available
  • 93. Why not use them
    • Specialised not universal
    • High price
    • Simple or no GUI
    • No networking
    • Research not production languges
  • 94. Important programming features
    • Dynamic languages:
    • Do not care about types of data
    • Focus on the algorithms
    • But reject a full data control
  • 95. Important programming features
    • Prototyping:
    • Checks algorithms
    • Easily visualise data or make GUI
    • Speed is measured as the time of coding not running
  • 96. Important programming features
    • Scientific libraries:
    • How often do we construct new algorithms (Charlie Eppes Syndrome – NUMB3RS)?
    • The application is more like a large panel building than brick building – applies to the AI problems as well as to the „ordinary ” soft
  • 97. Important programming features
    • Visual design (graphical programming):
    • GUI could be constructed using components
    • Link the components visually
    • Evaluate application faster
  • 98. More important part of AI software Knowledge Models Rules Graphs
  • 99. Hybrid software development Research Development Idea Product Replanning & redesign Technology X Technology X Technology Y ?? ??? ??
  • 100. Programming libraries – C/C++
    • MLC++ http://www.sgi.com/tech/mlc/
    • FANN http://leenissen.dk/fann/
    • Waffles http://waffles.sourceforge.net/
    • OpenCV ML http://opencv.willowgarage.com/documentation/cpp/ml._machine_learning.html
    • dlib C++ http://dlib.net/
    • VFML http://www.cs.washington.edu/dm/vfml/
  • 101. Programming libraries – Java
    • Java-ML http://java-ml.sourceforge.net/
    • Weka Engine http://www.cs.waikato.ac.nz/ml/weka/
    • MLJ http://sourceforge.net/projects/mldev/
    • MALLET http://mallet.cs.umass.edu/
    • LIBSVM http://www.csie.ntu.edu.tw/~cjlin/libsvm/
  • 102. Programming libraries – C#
    • Infer.NET http://research.microsoft.com/en-us/um/cambridge/projects/infernet/
    • AForge.NET http://code.google.com/p/aforge/
    • C# Neural Network Library http://franck.fleurey.free.fr/NeuralNetwork/
    • NeuronDotNet http://neurondotnet.freehostia.com/
    • NeuroBox Neural Network Library http://www.cdrnet.net/projects/neuro/
  • 103. Programming libraries – Python
    • Orange engine http://www.ailab.si/orange/
    • PyML http://pyml.sourceforge.net/
    • MDP http://mdp-toolkit.sourceforge.net/
    • PyBrain http://www.pybrain.org/
    • LIBSVM http://www.csie.ntu.edu.tw/~cjlin/libsvm/
  • 104. Programming libraries – PHP
    • FANN PHP Extension http://fann.sourceforge.net/fann.html
    • Tremani Neural Network http://www.tremani.nl/open-source/neural-network/
    • ANN http://ann.thwien.de/index.php/Main_Page
  • 105. Example I – Java
    • A simple data mining task
    • Uses Java-ML library
  • 106. Example II – Python
    • Adaptive filtering algorithm
    • Use as a web service (server part)
    • Use mod_python on Apache
  • 107. Example III – Flex
    • Particle Swarm Optimisation example
    • RIA application
    • Uses a standard Flex without any additional libraries
  • 108. Thank you. Any questions? Get in touch: www.sdart.co.uk