Your SlideShare is downloading. ×
0
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
24 hoursofpassppt jyotigupta_spring2011_fixed
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

24 hoursofpassppt jyotigupta_spring2011_fixed

387

Published on

expressor software presentation at the 24 Hour of PASS event on Mar 15 - 16, 2011.

expressor software presentation at the 24 Hour of PASS event on Mar 15 - 16, 2011.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
387
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Intelligent ETL with SQL Server<br />Jyoti Gupta, Senior Presales Engineer, expressor software<br />
  • 2. About me<br />10 years of ETL experience<br />Most of my career as an ETL Presales Engineer<br />Currently at expressor software<br />jgupta@expressor-software.com<br />
  • 3. Agenda<br />~40 minutes<br />Modularity<br />Abstraction <br />Modularity and abstraction in ETL<br />~10 minutes<br />Q/A<br />Wrap up<br />
  • 4. Topic 1: Modular Design<br />
  • 5. When you think of “modular” design, what do you think of?<br />Snap ins<br />Components<br />Assembly<br />Who remembers “units” knitwear?<br />
  • 6. Let’s Explore the Fast Food Example<br />Basic Ingredients<br />Basic toppings<br />Menu Choice<br />Hamburger<br />Cheeseburger<br />Cheese<br />Hamburger Patty<br />Double Cheeseburger<br />Chicken Sandwich<br />Hamburger Bun<br />Grilled Chicken Breast<br />Ranch Chicken Sandwich<br />Chicken Sandwich w/Cheese<br />Wrap<br />Ranch Chicken Sandwich w/Cheese<br />Fish Filet<br />Fish Sandwich<br />Ranch Dressing<br />Fish Wrap<br />
  • 7. Benefits of Modular Design<br />Combinability = meet the demands of more consumers<br />Adaptability = earliest responder to changes in market conditions<br />Efficiency = lower cost of product<br />Consistency in (good) quality = lowers perceived risk and lowers “barriers to entry”<br />
  • 8. Drawbacks of Modular Design<br />Change one/change all is a double edged sword<br />Module management<br />Storing<br />Organizing<br />Versioning<br />
  • 9. Topic 2: Abstraction<br />
  • 10. What is abstraction?<br />The thought process wherein ideas…<br /> are distanced….<br /> from objects….<br />
  • 11. Abstractions simplify and organize<br />Orange<br />Orange<br />Pumpkin<br />Basketball<br />
  • 12. The process by which data and programs are defined with a representation similar to its meaning (semantics)<br />Applies to<br />Data<br />Actions<br />Abstraction in computer science<br />
  • 13. Data Abstraction<br />
  • 14. Action abstraction<br />
  • 15. Modularity and Abstraction in ETL<br />
  • 16. Overview of ETL<br />What is ETL?<br />Where does SQL Server fit in?<br />
  • 17. Modularity in ETL<br />Which components of ETL apps can and should be modularized?<br />
  • 18. Data Abstraction in ETL<br />Data type wrangling… ever felt like a cowboy/cowgirl trying to herd the unherdable? <br />Git along li’lnvarchar<br />SQL Server: nvarchar<br />ODBC: SQL_VARCHAR<br />MySQL: <br />varchar<br />PostgreSQL: character varying<br />
  • 19. Function Abstraction in ETL<br />Think about using abstract functions. Does your transformation language support abstraction?<br />Examples<br />Lua anonymous functions and closures<br />function makeaddfunc(x)<br />return <br />function(y) <br /> return x + y<br /> end <br />end <br />plustwo= makeaddfunc(2) <br />C# anonymous function<br />Func<int,int> foo = x => x*x; <br />
  • 20. Put it all together<br />Modular, abstracted ETL programming environments:<br />Modular connectivity objects<br />Modular schema objects<br />Modular data types that can be abstracted<br />Modular functions that can be abstracted<br />Build your applications like a fast food hamburger<br />Mix n’ match applications are built efficiently<br />
  • 21. What else?<br />Think about how you are designing your ETL applications<br />How much of your code is dealing with data quality issues that have to be fixed manually?<br />Null values<br />Invalid values<br />Invalid formats<br />Missing values<br />Data out of range or not in the proscribed range<br />Find those errors first (and make fixing them someone else’s problem)<br />It’s hard to predict the future, so adaptable and modular designs have long-terms benefits.<br />
  • 22. Conclusions and Deep Thoughts<br />You’re doing something wonderful when you’re doing ETL!<br />ETL is applied fundamental computer science.<br />
  • 23. Professional Association for SQL Server<br />Thank you to our sponsor<br />
  • 24. Save 25%: Register by April 12th<br />www.sqlpass.org/sqlrally<br />May 11-13, Orlando, FL<br />Register by March31st: save 40% and have the chance to win a cruise to Alaska! <br />“24HR11” code gets you $100 off<br />www.sqlpass.org/summit<br />Oct 11-14, Seattle, WA<br />

×