More Related Content
Recently uploaded
Recently uploaded (20)
Featured
Featured (20)
EDW Database Backup and DR Strategy
- 1. Enterprise Data Warehouse Database Backup & Disaster Recovery Strategy By Tan Kian Hui https://www.linkedin.com/in/tankianhui
- 2. Problem Statement 1 Business Need – To access at least 7 years of data 2 Banking Compliance – Require to perform Database Backup and Disaster Recovery (DR) plan 3 Short maintenance window for database clean up, optimization, backup etc. 4 MPP database does not support incremental & cumulative backup 2 5 EDW database size >50 TB in compressed format 6 Critical system – Short RTO & MTD 7 Shorten backup time, restore time, RTO & MTD https://www.linkedin.com/in/tankianhui
- 3. Method 1 - Standard Backup & DR Strategy
- 4. Method 1 - Standard Backup & DR Strategy Task : DR Environment Table Partition ev ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 .. .. .. .. .. ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 PROD Environment 4 https://www.linkedin.com/in/tankianhui
- 5. Method 1 - Standard Backup & DR Strategy Task : DR Environment Table Partition ev ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 .. .. .. .. .. ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 PROD Environment 5 https://www.linkedin.com/in/tankianhui Weekly
- 6. Method 1 - Standard Backup & DR Strategy WeeklyTask : DR Environment Table Partition ev ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 .. .. .. .. .. ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 Full database/table backup 1 PROD Environment 6 https://www.linkedin.com/in/tankianhui
- 7. Method 1 - Standard Backup & DR Strategy WeeklyTask : DR Environment Table Partition ev ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 .. .. .. .. .. ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 Full database/table backup 1 PROD Environment 7 https://www.linkedin.com/in/tankianhui DR
- 8. Method 1 - Standard Backup & DR Strategy Weekly DRTask : DR Environment Table Partition ev ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 .. .. .. .. .. ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 .. .. .. .. .. ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 Full database/table backup 1 Full database/table restore 2 Catch-up run 7 days 3 PROD Environment 8 https://www.linkedin.com/in/tankianhui
- 9. Method 1 - Standard Backup & DR Strategy Weekly DRTask : DR Environment Table Partition ev ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 .. .. .. .. .. ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 .. .. .. .. .. ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 Full database/table backup 1 Full database/table restore 2 Catch-up run 7 days 3 PROD Environment Business continue 4 9 https://www.linkedin.com/in/tankianhui
- 10. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly)
- 11. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table 11 https://www.linkedin.com/in/tankianhui Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2013 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2013 Restored in DR on 3/4/2013 Backup Tape keep for 7 years
- 12. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 DailyTask : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily Example 12 https://www.linkedin.com/in/tankianhui Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2013 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2013 Restored in DR on 3/4/2013 Backup Tape keep for 7 years
- 13. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Backup Daily Table 1 DailyTask : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily Example 13 https://www.linkedin.com/in/tankianhui Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2013 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2013 Restored in DR on 3/4/2013 Backup Tape keep for 7 years
- 14. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Backup Daily Table 1 DailyTask : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily Example 14 https://www.linkedin.com/in/tankianhui Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2013 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2013 Restored in DR on 3/4/2013 Backup Tape keep for 7 years Monthly 2nd Aug 2019
- 15. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Exchange Partition >3 months Backup Daily Table 1 Daily Monthly Backup Monthly Table 3 Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily2nd Aug 2019 Example 15 https://www.linkedin.com/in/tankianhui Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2013 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2013 Restored in DR on 3/4/2013 Backup Tape keep for 7 years Housekeeping 4 2
- 16. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Exchange Partition >3 months Backup Daily Table 1 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Housekeeping Daily Monthly Yearly 4 Table Partition ev_h Backup Monthly Table 3 Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily2nd Aug 2019 2nd April 2020 Example 16 https://www.linkedin.com/in/tankianhui ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2013 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2013 Restored in DR on 3/4/2013 Backup Tape keep for 7 years 2
- 17. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Year MOD 7 2010 1 2011 2 2012 3 2013 4 2014 5 2015 6 2016 0 2017 1 2018 2 2019 3 2020 4 2021 5 Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Exchange Partition >3 months Backup Daily Table 1 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Housekeeping Daily Monthly Yearly 4 Exchange Partitions Table Partition ev_h Backup Monthly Table 3 Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily2nd Aug 2019 2nd April 2020 Example 17 https://www.linkedin.com/in/tankianhui ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Backup Yearly Table Backup Tape keep for 7 years 6 Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2013 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2020 Restored in DR on 3/4/2013 Backup Tape keep for 7 years 2 5
- 18. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Year MOD 7 2010 1 2011 2 2012 3 2013 4 2014 5 2015 6 2016 0 2017 1 2018 2 2019 3 2020 4 2021 5 Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Exchange Partition >3 months Backup Daily Table 1 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Housekeeping Daily Monthly Yearly 4 Exchange Partitions Table Partition ev_h Backup Monthly Table 3 Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily2nd Aug 2019 2nd April 2020 Example 18 https://www.linkedin.com/in/tankianhui ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Backup Yearly Table Backup Tape keep for 7 years 6 Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2013 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2020 Restored in DR on 3/4/2013 Backup Tape keep for 7 years 2 5 Disaster Preparedness Plan 3rd April 2020
- 19. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Year MOD 7 2010 1 2011 2 2012 3 2013 4 2014 5 2015 6 2016 0 2017 1 2018 2 2019 3 2020 4 2021 5 Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Exchange Partition >3 months Backup Daily Table 1 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Housekeeping Daily Monthly Yearly 4 Table Partition ev_h Backup Monthly Table 3 Disaster Preparedness Plan Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily2nd Aug 2019 2nd April 2020 3rd April 2020 Example 19 https://www.linkedin.com/in/tankianhui ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Backup Yearly Table Backup Tape keep for 7 years 6 Yearly Table Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2020 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2020 Restored in DR on 3/4/2020 Backup Tape keep for 7 years 2 Exchange Partitions Restore in DR 5 7
- 20. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Daily Monthly Yearly Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Disaster Preparedness Plan DRTask : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily2nd Aug 2019 2nd April 2020 3rd April 2020 15th April 2020 Example 20 https://www.linkedin.com/in/tankianhui Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2020 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2020 Restored in DR on 3/4/2020 Backup Tape keep for 7 years
- 21. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Daily Monthly Yearly Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Disaster Preparedness Plan DR Table ev Only require to restore Daily Tables to continue Daily Batch/Catch-up 1 day Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Daily2nd Aug 2019 2nd April 2020 3rd April 2020 15th April 2020 Example 21 https://www.linkedin.com/in/tankianhui Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2020 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2020 Restored in DR on 3/4/2020 Backup Tape keep for 7 years
- 22. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Daily Monthly Yearly Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Disaster Preparedness Plan DR Table ev Only require to restore Daily Tables to continue Daily Batch/Catch-up 1 day Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Business Continue Daily2nd Aug 2019 2nd April 2020 3rd April 2020 15th April 2020 Example 22 https://www.linkedin.com/in/tankianhui Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2020 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2020 Restored in DR on 3/4/2020 Backup Tape keep for 7 years
- 23. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Daily Monthly Yearly Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Disaster Preparedness Plan DR Table ev Only require to restore Daily Tables to continue Daily Batch/Catch-up 1 day Table ev_h Restore Monthly Tables stage-by-stage (later) May restore monthly for Disaster Preparedness Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Business Continue Daily2nd Aug 2019 2nd April 2020 3rd April 2020 15th April 2020 Example 23 https://www.linkedin.com/in/tankianhui Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Restored on 3/4/2020 Restored on 3/4/2014 Restored on 3/4/2019 Backup on 2/4/2020 Restored in DR on 3/4/2020 Backup Tape keep for 7 years
- 24. Method 2 - Backup & DR Strategy for Super Big Table - Store data in several tables (Daily, Monthly, Yearly) Table Partition ev ev_p201905 ev_p201906 ev_p201907 ev_p201908 Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 Table Partition ev ev_p202001 ev_p202002 ev_p202003 ev_p202004 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c4 ev_p201301 ev_p201302 ev_p201303 ev_p201304 ev_p201305 ev_p201306 ev_p201307 ev_p201308 ev_p201309 ev_p201310 ev_p201311 ev_p201312 Table Partition ev_c0 ev_p201601 ev_p201602 ev_p201603 ev_p201604 ev_p201605 ev_p201606 ev_p201607 ev_p201608 ev_p201609 ev_p201610 ev_p201611 ev_p201612 Table Partition ev_c1 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 Table Partition ev_c2 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Table Partition ev_c5 ev_p201401 ev_p201402 ev_p201403 ev_p201404 ev_p201405 ev_p201406 ev_p201407 ev_p201408 ev_p201409 ev_p201410 ev_p201411 ev_p201412 Table Partition ev_c6 ev_p201501 ev_p201502 ev_p201503 ev_p201504 ev_p201505 ev_p201506 ev_p201507 ev_p201508 ev_p201509 ev_p201510 ev_p201511 ev_p201512 Backup on 2/4/2017 Restored in DR on 3/4/2017 Backup Tape keep for 7 years Backup on 2/4/2018 Restored in DR on 3/4/2018 Backup Tape keep for 7 years Backup on 2/4/2019 Restored in DR on 3/4/2019 Backup Tape keep for 7 years Backup on 2/4/2014 Restored in DR on 3/4/2014 Backup Tape keep for 7 years Backup on 2/4/2015 Restored in DR on 3/4/2015 Backup Tape keep for 7 years Backup on 2/4/2016 Restored in DR on 3/4/2016 Backup Tape keep for 7 years DR Environment Table ev_c0 Table ev_c1 Table ev_c2 Table ev_c3 Table ev_c4 Table ev_c5 Table ev_c6 Restored on 3/4/2017 Restored on 3/4/2018 Restored on 4/4/2013 Restored on 3/4/2015 Restored on 3/4/2016 Table Partition ev_c3 ev_p201201 ev_p201202 ev_p201203 ev_p201204 ev_p201205 ev_p201206 ev_p201207 ev_p201208 ev_p201209 ev_p201210 ev_p201211 ev_p201212 Daily Monthly Yearly Table Partition ev_h ev_p201901 ev_p201902 ev_p201903 ev_p201904 ev_p201905 ev_p201906 ev_p201907 ev_p201908 ev_p201909 ev_p201910 ev_p201911 ev_p201912 Disaster Preparedness Plan DR Table ev Only require to restore Daily Tables to continue Daily Batch/Catch-up 1 day Table ev_h Restore Monthly Tables stage-by-stage (later) May restore monthly for Disaster Preparedness Task : Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Yearly Table Daily Table Monthly Table Restored on 3/4/2020 Restored on 3/4/2014 Restored on 3/4/2019 Business Continue Daily2nd Aug 2019 2nd April 2020 3rd April 2020 15th April 2020 Example 24 https://www.linkedin.com/in/tankianhui Summary • Backup Time • Backup Size • Backup Storage • Restore Time • RTO & MTD • Maintenance Window • And etc. • No backup on same historical data yearly/*monthly/*daily Backup on 2/4/2020 Restored in DR on 3/4/2020 Backup Tape keep for 7 years
- 25. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop)
- 26. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Task : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Example 26 https://www.linkedin.com/in/tankianhui
- 27. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 DailyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Example 27 https://www.linkedin.com/in/tankianhui
- 28. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 DailyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Example Backup Daily Table 1 28 https://www.linkedin.com/in/tankianhui
- 29. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 DailyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Example Backup Daily Table 1 29 https://www.linkedin.com/in/tankianhui Monthly 2nd June 2019
- 30. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p20170 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 Daily MonthlyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Exchange Partition >3 months 2 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Example 2nd June 2019 Backup Daily Table Backup Historical Table 1 3 30 https://www.linkedin.com/in/tankianhui ev_p201902
- 31. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p20170 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 Daily MonthlyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Exchange Partition >3 months 2 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Example 2nd June 2019 Backup Daily Table Backup Historical Table 1 3 31 https://www.linkedin.com/in/tankianhui ev_p201902 Disaster Preparedness Plan 3rd June 2019
- 32. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p20170 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 Daily MonthlyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Exchange Partition >3 months 2 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Example 2nd June 2019 Backup Daily Table Backup Historical Table 1 3 32 https://www.linkedin.com/in/tankianhui ev_p201902 Disaster Preparedness Plan Table ev_h Restore in DR 4 3rd June 2019
- 33. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Daily Monthly Half-YearlyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Exchange Partition >3 months 2 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Disaster Preparedness Plan Table ev_h Restore in DR 4 Example 2nd June 2019 3rd June 2019 2nd July 2019 Table Partition ev_h ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 ev_p201903 Table Partition ev ev_p201904 ev_p201905 ev_p201906 ev_p201907 Backup Daily Table Backup Historical Table 1 3 33 https://www.linkedin.com/in/tankianhui ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706
- 34. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Daily Monthly Half-YearlyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Exchange Partition >3 months 2 Exchange Partitions >2 years 5 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Backup Half-Yearly Table. Backup Tape keep for 7 years 6 Off load to Hadoop 7 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 Disaster Preparedness Plan Table ev_h Restore in DR 4 Example 2nd June 2019 3rd June 2019 2nd July 2019 Table Partition ev_h ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 ev_p201903 Table Partition ev ev_p201904 ev_p201905 ev_p201906 ev_p201907 Backup Daily Table Backup Historical Table 1 3 34 https://www.linkedin.com/in/tankianhui ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706
- 35. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Daily Monthly Half-YearlyTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Exchange Partition >3 months 2 Exchange Partitions >2 years 5 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Backup Half-Yearly Table. Backup Tape keep for 7 years 6 Off load to Hadoop 7 Disaster Preparedness Plan Table ev_h Restore in DR 4 Example 2nd June 2019 3rd June 2019 2nd July 2019 Table Partition ev_h ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 ev_p201903 Table Partition ev ev_p201904 ev_p201905 ev_p201906 ev_p201907 Backup Daily Table Backup Historical Table 1 3 35 https://www.linkedin.com/in/tankianhui ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 Disaster Preparedness Plan 4th July 2019
- 36. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Daily Monthly Half-Yearly Disaster Preparedness Plan Task : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Exchange Partition >3 months 2 Exchange Partitions >2 years 5 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Backup Half-Yearly Table. Backup Tape keep for 7 years 6 Off load to Hadoop 7 Table ev_x Table ev_t Restore in DR Disaster Preparedness Plan Table ev_h Restore in DR 4 Example 2nd June 2019 3rd June 2019 2nd July 2019 4th July 2019 Table Partition ev_h ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 ev_p201903 Table Partition ev ev_p201904 ev_p201905 ev_p201906 ev_p201907 Backup Daily Table Backup Historical Table 1 3 36 https://www.linkedin.com/in/tankianhui ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 8
- 37. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Daily Monthly Half-Yearly Disaster Preparedness Plan DRTask : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Table ev_x Table ev_t Disaster Preparedness Plan Table ev_h Example 2nd June 2019 3rd June 2019 2nd July 2019 4th July 2019 15th July 2019 Table Partition ev_h ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 ev_p201903 Table Partition ev ev_p201904 ev_p201905 ev_p201906 ev_p201907 37 https://www.linkedin.com/in/tankianhui ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706
- 38. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Daily Monthly Half-Yearly Disaster Preparedness Plan DR Table ev Only require to restore Daily Tables to continue Daily Batch/Catch-up 1 day Task : DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Table ev_x Table ev_t Disaster Preparedness Plan Table ev_h Example 2nd June 2019 3rd June 2019 2nd July 2019 4th July 2019 15th July 2019 Table Partition ev_h ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 ev_p201903 Table Partition ev ev_p201904 ev_p201905 ev_p201906 ev_p201907 38 https://www.linkedin.com/in/tankianhui ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706
- 39. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Daily Monthly Half-Yearly Disaster Preparedness Plan DR Table ev Only require to restore Daily Tables to continue Daily Batch/Catch-up 1 day Task : Business Continue DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Table ev_x Table ev_t Disaster Preparedness Plan Table ev_h Example 2nd June 2019 3rd June 2019 2nd July 2019 4th July 2019 15th July 2019 Table Partition ev_h ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 ev_p201903 Table Partition ev ev_p201904 ev_p201905 ev_p201906 ev_p201907 39 https://www.linkedin.com/in/tankianhui ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706
- 40. Method 3 - Backup & DR Strategy for Big Table - Store data in several tables (Daily, Historical + Offload to Hadoop) Daily Table Partition ev ev_p201903 ev_p201904 ev_p201905 ev_p201906 Table Partition ev_h ev_p201612 ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706 ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 Daily Monthly Half-Yearly Disaster Preparedness Plan DR Table ev Only require to restore Daily Tables to continue Daily Batch/Catch-up 1 day Task : Business Continue DR Environment Table Partition ev_t Table Partition ev_x ev_p201206 .. ev_p201301 .. ev_p201401 .. ev_p201501 .. ev_p201601 .. ev_p201611 ev_p201612 Daily Table Historical Table (Min 2 years) Temporary Table External Table (Data in HDFS) Table ev_x Table ev_t Disaster Preparedness Plan Table ev_h Example 2nd June 2019 3rd June 2019 2nd July 2019 4th July 2019 15th July 2019 Table Partition ev_h ev_p201707 ev_p201708 ev_p201709 ev_p201710 ev_p201711 ev_p201712 ev_p201801 ev_p201802 ev_p201803 ev_p201804 ev_p201805 ev_p201806 ev_p201807 ev_p201808 ev_p201809 ev_p201810 ev_p201811 ev_p201812 ev_p201901 ev_p201902 ev_p201903 Table Partition ev ev_p201904 ev_p201905 ev_p201906 ev_p201907 40 https://www.linkedin.com/in/tankianhui Summary • Backup Time • Backup Size • Backup Storage • Restore Time • RTO & MTD • Maintenance Window • And etc. • No backup on same historical data yearly/*monthly/*daily ev_p201701 ev_p201702 ev_p201703 ev_p201704 ev_p201705 ev_p201706
- 41. Comparison – Backup Strategy x Data Volume to Backup Data Volume to Backup (Number of Months) 1. Backup Time 2. Backup File Size 3. Backup Storage = Stacked Column Chart (1-year projection) Stacked Column Chart (5-months projection) 41 https://www.linkedin.com/in/tankianhui
- 42. Important Things to Consider 1 Complexity & Workload – Developer, Modeler, DBA, Support, Operation 2 EDW data in Daily Table for Daily Batch Run & Reporting – 3 months data? 3 Data retention in EDW → Hot & Cold Data → Off load to Hadoop HDFS/HIVE (SQL-on-Hadoop) 4 Performance - EDW (MPP DB) vs Hadoop (SQL-on-Hadoop) 42 5 Only applicable for transaction or append-only table 6 Backup Strategy + Incremental/Cumulative Backup – 2 in 1 https://www.linkedin.com/in/tankianhui