• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Sunz2010   Development Version Control And Automated Deployment Of Source Code In A Data Warehouse Environment
 

Sunz2010 Development Version Control And Automated Deployment Of Source Code In A Data Warehouse Environment

on

  • 2,053 views

 

Statistics

Views

Total Views
2,053
Views on SlideShare
2,030
Embed Views
23

Actions

Likes
1
Downloads
18
Comments
0

3 Embeds 23

http://www.sunz.net.nz 12
http://www.slideshare.net 10
http://sunz.net.nz 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Sunz2010   Development Version Control And Automated Deployment Of Source Code In A Data Warehouse Environment Sunz2010 Development Version Control And Automated Deployment Of Source Code In A Data Warehouse Environment Presentation Transcript

    • SUNZ 2010 Version Control and Automated Deployment of Source Code in a Data Warehouse Environment Mike O’Neil MSD Data Warehouse Team [email_address]
    • Content
      • Background and Context
      • Motivation
      • Change Management
      • Automation of Deployment
      • Subversion and TortoiseSVN
      • Standards and Practices
      • SAS Features and Functionality
      • Where to from here?
    • Background and Context
      • IAP Data Warehouse (SAS based)
      • Established 1994
      • 30+ Source Systems
      • 10,000 Information Consumers
      • UNIX and Windows SAS platforms
      • 600+ Control-M scheduled jobs
      • Huge SAS Code Base
      • Frequency of Change
    • Motivation
      • Quality improvement
      • Reduction in Failure
      • Productivity and Efficiency
      • Cost
      • Physical Environment Separation
      • Maturing process
      • Removal of emotion
      • Increasing accountability and ownership
    •  
    • Change Management
      • Change Management
        • Policy and process
      • Change Control
        • Recording and approval
      • Version Control
        • Asset Management
    • Change Management
      • Survey Results
      • Not all changes logged 95%
      • Changes not thoroughly tested 90%
      • Lack of process enforcement 85%
      • Poor communication and dissemination 65%
      • Lack of centralized process ownership 60%
      • Lack of change approval policy 50%
      • Frequent change notification after the fact 40%
      Reference: Edward Stickel quoting Harris Kern in article at http://www.itsmwatch.com/itil/article.php/11700_3367151_1
    • Automation of Deployment
      • Common feature of commercial software development
      • Continuous integration
      • Regression testing
      • Configuration driven (what goes where)
      • Version Control
    • IAP Automation of Deployment
      • Subversion for Version Control
      • TortoiseSVN for Developer Interface
      • Configuration driven
      • Deployment Application written in SAS (and UNIX shell script)
    • Subversion & TortoiseSVN
      • Subversion – Database, Server, CLI
      • TortoiseSVN – Windows Explorer plug-in
      • Basic developer workflow:
        • Check Out to create a Working Copy
        • Branch to manage your own stuff
        • Commit changes
        • Update when new release hits trunk
        • Tag when ready for release
    • Subversion
    •  
    • IAP Automation of Deployment
      • Configuration “database”
        • Configuration_Objects – names all objects
        • Configuration_Paths – says where they go
      • Owned and maintained by developers
    •  
    • IAP Automation of Deployment
      • Reconciliation
      • Export from repository
      • Pre-deploy steps
      • Create target directory structure
      • Deploy software objects to target
      • Post-deploy steps
    • Standards and Practice
      • Typical IAP approach
      • Driver Shell
        • Invoked by Control-M
        • Creates environment variables
        • Calls Driver SAS program
      • Driver SAS
        • Retrieves environment variables (%sysget)
        • %includes SAS code to do work
    • Standards and Practice
      • Driver Shell Script
        • Export ENVAR=value
        • export SASLIB=/lev1_3/app/sasdata
        • export SASCODE=/r02/app/sasprogs
      • Driver SAS Program
        • %let saslib = %sysget(SASLIB);
        • libname applib “&saslib”;
        • %let sascode = %sysget(SASCODE);
        • filename appsrc “&sascode”;
        • %include appsrc(worker1.sas);
    • SAS Features and Functionality
      • Configuration “database” in CSV files
        • SAS Proc Import
      • SVN List command –xml
        • SAS xml libname engine and xml maps
      • Programs that write programs
        • Put “cp a.sh /r02/test/ControlM/shells”;
        • Put “chgrp develop /r02/test/ControlM/shells/a.sh”;
        • Put “chmod 2771 /r02/test/ControlM/shells/a.sh”;
      • Standards and practice (using %sysget)
    • Where to from here?
      • Acceptance
      • Training
      • Complete repository set up
      • Complete required refactoring
      • Integrate with JIRA and Remedy
      • Run under Control-M
    • Questions?
      • Thank you
      [email_address]