SQL Server 2008 כפלטפורמה לניהול נתונים בעולם ה-Peta byte<br />
What is a VLDB?<br />Used to be a set size:<br /><ul><li>>100mb in the 1980’s
>100gb in the early 1990’s
>1tb in the early 2000’s
>10tb in the mid 2000’s
Now we are dealing with hundreds of TeraBytes</li></ul>Now, a more flexible definition for VLDB prevails:<br /><ul><li>It’...
Design Philosophy VLDB<br />Don’t think of it as Large complex database, think of it as smaller manageable components<br /...
Instance Partitioning
Database Partitioning
Upcoming SlideShare
Loading in …5
×

Vldb Yaron Moshe

630 views
561 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
630
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Vldb Yaron Moshe

  1. 1. SQL Server 2008 כפלטפורמה לניהול נתונים בעולם ה-Peta byte<br />
  2. 2. What is a VLDB?<br />Used to be a set size:<br /><ul><li>>100mb in the 1980’s
  3. 3. >100gb in the early 1990’s
  4. 4. >1tb in the early 2000’s
  5. 5. >10tb in the mid 2000’s
  6. 6. Now we are dealing with hundreds of TeraBytes</li></ul>Now, a more flexible definition for VLDB prevails:<br /><ul><li>It’s a VLDB when you can no longer use “common and standard” techniques for management and performance</li></li></ul><li>VLDB project Challenges<br />Plenty of data to handle<br />Massive data insertion<br />Massive query loads<br />Need for High Availability<br />Need for Scalability<br />Cost reduction<br />
  7. 7. Design Philosophy VLDB<br />Don’t think of it as Large complex database, think of it as smaller manageable components<br />Putting all the data in one single database doesn’t make you a hero.<br />Partitioning data into smaller manageable sizes<br />Methods for data partitioning:<br /><ul><li>Server Partitioning
  8. 8. Instance Partitioning
  9. 9. Database Partitioning
  10. 10. Tables Partitioning</li></li></ul><li>Data separation: Data Partitioning vs. Function Partitioning<br />Examples of data partitioning by data<br />Data broken up by months in a year: Jan, Feb, Mar……<br />Data broken up by States in the US: Alabama, Arkansas, Alaska…..<br />Examples of data partitioning by function<br />Data separated by time: sales data, consulting data, competitive data….<br />Data separated by business function: manufacturing data, design data, inventory data…….<br />
  11. 11. Table Partitioning: Common Scenario “Sliding Window”<br />New data is added continually (every minute, hourly, Daily, Weekly, Monthly)<br />Data must be scrubbed, indexed, etc.<br />As data ages, data is modified less frequently and then - not at all.<br />Eventually, it may be discarded <br />Sliding Window<br />Time<br />Obsolete<br />Hot<br />Read-Mostly<br />Read-Only<br />
  12. 12. Data Feeder Aggr<br />First tier Databases Aggr<br />Tiered Manager<br />ETL<br />Queries<br />Tiered Manager<br />Manager Server<br />Where can I find my Data?<br />Queries<br />Second tier Databases Aggr<br />Application Server<br />
  13. 13. FileGroup 2<br />FileGroup 1<br />FileGroup 1<br />Key (partitioning column)<br />LocatingARow<br />Partition Scheme<br />Partition<br />Function<br />
  14. 14. HA and scalability<br />N+1 Clustering<br />Smart management enables fast scalability by adding servers, databases, instances, partitions etc.<br />
  15. 15. Cost reduction<br />Page Level Data compression – saves up to 50% of disk storage (and reduces IO)<br />Smaller partitions may reduce indexes size by 30% (!) – critical in huge environments<br />Moving historical to cheaper storage/media <br />No need for Mega servers - Standard servers are doing the job<br />
  16. 16. Valinor’s people in this project<br />Shahar Bar<br />Meir Dudai<br />Tzahi Hakikat<br />…. And more….<br />
  17. 17. Thank you!<br />

×