• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Data Migration In An Agile Open Source World

Data Migration In An Agile Open Source World



Data Migration In An Agile Open Source World presented by Craig Smith at the Open Source Developers Conference (OSDC) 2009 in Brisbane.

Data Migration In An Agile Open Source World presented by Craig Smith at the Open Source Developers Conference (OSDC) 2009 in Brisbane.



Total Views
Views on SlideShare
Embed Views



6 Embeds 102

http://craigsmith.id.au 44
http://cds43.wordpress.com 21
http://www.scoop.it 20
http://www.slideshare.net 15
http://webcache.googleusercontent.com 1
http://www.techgig.com 1


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Data Migration In An Agile Open Source World Data Migration In An Agile Open Source World Presentation Transcript

    • Data Migration in an Agile Open Source World Craig Smith Suncorp
    • Welcome… Image http://www.neopoleon.com/blog/images/excel/7.jpg
    • According to Wikipedia: Data migration is the process of transferring data between ... computer systems ... is usually performed programmatically to achieve an automated migration ... required when organizations ... change ... or upgrade to new systems, or when systems merge... Image: http://i.ehow.com/images/GlobalPhoto/Articles/2065416/dictionary_Full.jpg
    • “In” End Point Extract Extract Transform Load Transform (ETL) Load Pattern “Out” End Point
    • File transfer (Enterprise) Shared database Application Integration Remote Patterns Procedure Invocation Messaging Image http://www.eaipatterns.com/eaipatterns.html
    • Data Migration + Application Integration = Similar Approach Image http://kansolutions.net/yahoo_site_admin/assets/images/integration.302161703_std.jpg
    • Image http://blog.theworkinggroup.ca/wp-content/uploads/2009/04/agile-manifesto1.gif Agile Manifesto
    • Scrum Practices Image http://blogs.conchango.com/Admin/ImageGallery/blogs.conchango.com/Colin.Bird/Scrum%20Overview%20Diagram.png
    • Image Beck, Kent – Extreme Programming Explained: Embrace Change XP Practices
    • Development Practices Fixing a Fast (<3 Use It 100% Nulls broken build mins) Or Lose Code Are Evil is always the Builds It highest Coverage priority Test Changing The Code Is Following Things Prod If In Doubt, Code Is A Use Is A Team Code Spike It Liability Outlandish Out (On A Decision... Class Branch) Statics Names Are Evil Molecular No Code Over Ownership Atomic All Prod Create Use Tests Code Small Delegation Paired (Or Commit Classes And Over Reviewed) Often Methods Inheritance Source http://www.assembla.com/wiki/show/burro/Development_Principles
    • Image http://iqizone.com/images/paper%20pile.jpg Break It Down... Iteratively
    • Agile Builds Confidence Image http://static.soxfirst.com/soxfirst.com/imgname--derivatives_time_bomb_for_big_banks---50226711--bomb.jpg
    • Continuous (Inte)(Mi)gration
    • Throttle Data Image http://i.ehow.com/images/GlobalPhoto/Articles/2289906/P1120995RPM_Full.jpg
    • Image http://www.mikkeman.nl/blog/wp-content/uploads/2009/10/priority-300x137.jpg http://ralphlosey.files.wordpress.com/2009/10/playing-cards.jpg Prioritisation & Categorisation
    • Volatility Categorisation Image http://silverspaceship.com/static/shot_1.png
    • Image http://www.domainlanguage.com/_library/images_people_working/designing_at_whiteboard.jpg Analysis = BDUF NDUF SDUF
    • Implement “Big Bang” Image http://silverspaceship.com/static/shot_1.png
    • Image http://sapplanning.files.wordpress.com/2008/07/integrate-technology.jpg Implement “Master + Deltas”
    • Data in non-binary format > diff Image http://www.spycomponents.com/images/xml_at_work.gif
    • Image http://images.yogee.com.au/cat-dump-truck-609-2.jpg Master Database Dumps
    • Activity Migration Image http://www.life123.com/bm.pix/read-balance-sheet.s600x600.jpg
    • Image http://www.life123.com/bm.pix/read-balance-sheet.s600x600.jpg Reconciliation
    • Open Source Continuous Integration / Build Tools Continuous Integration Servers  Hudson: https://hudson.dev.java.net  Cruise Control: http://cruisecontrol.sourceforge.net Build Tools  Ant: http://ant.apache.org  Maven: http://maven.apache.org  Gradle: http://www.gradle.org  SCons: http://www.scons.org  Phing: http://phing.info  Rake: http://rake.rubyforge.org/
    • Open Source Developer Tools IDE’s (multiple language support)  Eclipse: http://www.eclipse.org  NetBeans: http://www.netbeans.org  RadRails: http://www.aptana.com/rails Database Clients  DBVisualizer: http://www.dbvis.com/  SQuirreL: http://squirrel-sql.sourceforge.net Web Service / SOAP Tools  soapUI: http://www.soapui.org Version Control Systems  Subversion: http://subversion.tigris.org  Git: http://git-scm.com Database Versioning  Liquibase: http://www.liquibase.org/
    • Open Source Testing Tools Test Specification – xUnit Tools  JUnit: http://www.junit.org  PHPUnit: http://www.phpunit.de/ Test Specification Tools – Expressive/BDD  FitNesse: http://fitnesse.org  Concordion: http://www.concordion.org  Cucumber: http://cukes.info  easyb: http://www.easyb.org Test Execution / Record / Log / UI  Selenium: http://seleniumhq.org  Watir: http://wtr.rubyforge.org Performance Test  JMeter: http://jakarta.apache.org/jmeter/  Grinder: http://grinder.sourceforge.net/
    • Open Source ETL/ESB Tools ETL  Donkey: http://www.assembla.com/wiki/show/burro  Pentaho (Kettle): http://kettle.pentaho.org/  Talend: http://www.talend.com  Clover: http://www.cloveretl.com/  Scriptella: http://scriptella.javaforge.com/ ESB  Mule: http://www.mulesoft.org  Apache ServiceMix: http://servicemix.apache.org  Apache Camel: http://camel.apache.org/
    • Questions ?? Craig Smith http://www.suncorp.com.au http://www.agileacademy.com.au craig.smith@suncorp.com.au @smithcdau Suncorp is one of Australia and New Zealand's largest diversified financial services providers, supplying banking, insurance and wealth management products to around 7 million customers through well-established and recognised brands such as AAMI, Australian Pensioners Insurance Agency, Shannons, Vero, Asteron and Tyndall, as well as Suncorp and GIO. Today, Suncorp is Australia's fifth largest bank and second largest domestic general insurance group, with over 16,000 staff. Suncorp has representation in 450 offices, branches and agencies throughout Australia and New Zealand.