SlideShare a Scribd company logo
1 of 9
DWH Operations
Components of DWH Batch
Sources:
Files
Reporting DB(BCV)
DataStage Jobs:
IN_STG_DAILY_BATCH_8PM
IN_STG_DAILY_BATCH_1230AM
TDS_IAA_HISTCALLS_GSM_0430AM
HISTCALLS_TBL_AGGREGATED_DATA_0205AM
IN_CRM_STG_DAILY_BATCH_0215AM
IN_STG_SOR_DAILY_BATCH_0600AM
DWH Operations
Components of DWH batch
Post Batch Activities:
Batch completion message
 Message audience
Jhatpat message
Important KPI’s message
 Message Audience
Daily Recharge SMS
 Message Audience
Report Triggers
DWH Operations
Components of DWH batch
Batch Independent Jobs:
VLR
AKSA
Erlang
Hourly Recharge (every 4 hour)
UNICA Jobs
Other Batch Activities:
Daily
 AA Stats
 Previous File directory deletion
 DataStage EDWProd and important scripts backup
DWH Operations
Components of DWH batch
Other Batch Activities:
Weekly
 SOR Stats
 Product Arrangement DS Job
Monthly
 Data File Recon
 DS Lock Release Activity
 Monthly Recon
DWH Operations
Bottle Necks / Limitations
Other Batch Activities:
BCV completion time reduction
Manual update of Lookup tables (e.g. GEOGRAPHIC_POINT, TAC_LIST, etc.)
Number of session DataStage jobs create
File availability issue (SDP dump, VLR, etc. )
Proper intimation to DWH team in case of any new product or critical change
(e.g. CCN EMM file update)
UNICA maintenance issues
DWH Operations
DWH Batch Improvements
Other Batch Activities:
Failure alerts monitoring to be monitored by NOC
Predefined resource allocation (rotational) in case of batch failures
Introduction of new checkpoints in DWH so that batch waits if critical table(s) are
not properly loaded.
BCV completion time reduction
DWH purging criteria finalization for automated purging
DWH Operations
Documentation
Completed:
Warid Design Document (IBM)
Warid EDWBI - User Procedure Manual (IBM)
CR document
DataStage Jobs error Log
Netezza DB Stats
DataStage Project Import & Export
DataStage Lock Release
DataStage Job change
Incomplete:
DWH Table and Job mappings
DWH Operations
Documentation
Upcoming:
DataStage Production backup
DWH Operations
Concerns & Suggestions
Unless critical, we should avoid entertaining business users request after
6 pm
Transactional activities that involve direct interaction with subscriber should be
avoided in DWH

More Related Content

Similar to DWH Operations presentation

Catalogic DPX: Dashboard Reporting with Microsoft Power BI
Catalogic DPX: Dashboard Reporting with Microsoft Power BICatalogic DPX: Dashboard Reporting with Microsoft Power BI
Catalogic DPX: Dashboard Reporting with Microsoft Power BICatalogic Software
 
Dynamics 365 for Finance and Operations - Power BI
Dynamics 365 for Finance and Operations - Power BIDynamics 365 for Finance and Operations - Power BI
Dynamics 365 for Finance and Operations - Power BIJuan Fabian
 
Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0
Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0
Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0Denodo
 
Project Systems
Project SystemsProject Systems
Project Systemschalah
 
IBM Insight 2013 - Aetna's production experience using IBM DB2 Analytics Acce...
IBM Insight 2013 - Aetna's production experience using IBM DB2 Analytics Acce...IBM Insight 2013 - Aetna's production experience using IBM DB2 Analytics Acce...
IBM Insight 2013 - Aetna's production experience using IBM DB2 Analytics Acce...Daniel Martin
 
Agile Testing Days 2017 Introducing AgileBI Sustainably
Agile Testing Days 2017 Introducing AgileBI SustainablyAgile Testing Days 2017 Introducing AgileBI Sustainably
Agile Testing Days 2017 Introducing AgileBI SustainablyRaphael Branger
 
SAP BI/DW Training with BO Integration
SAP BI/DW Training with BO IntegrationSAP BI/DW Training with BO Integration
SAP BI/DW Training with BO Integrationmishra4927
 
KeyAchivementsMimecast
KeyAchivementsMimecastKeyAchivementsMimecast
KeyAchivementsMimecastVera Ekimenko
 
Order Review Solution Application (Version 2.0)
Order Review Solution Application (Version 2.0)Order Review Solution Application (Version 2.0)
Order Review Solution Application (Version 2.0)Nag Arvind Gudiseva
 
RPAS MFP Tech Support Teach-Out
RPAS MFP Tech Support Teach-OutRPAS MFP Tech Support Teach-Out
RPAS MFP Tech Support Teach-Outwellforddowdy
 
SharePoint 2013 DR solution overview
SharePoint 2013 DR solution overviewSharePoint 2013 DR solution overview
SharePoint 2013 DR solution overviewEmilio Gratton
 
5 Years of Progress in Active Data Warehousing
5 Years of Progress in Active Data Warehousing5 Years of Progress in Active Data Warehousing
5 Years of Progress in Active Data WarehousingTeradata
 
Presentation On Clean Up MRP records in Exact Globe
Presentation On Clean Up MRP records in Exact GlobePresentation On Clean Up MRP records in Exact Globe
Presentation On Clean Up MRP records in Exact Globeronald_voets
 
Kevin Bengtson Portfolio
Kevin Bengtson PortfolioKevin Bengtson Portfolio
Kevin Bengtson PortfolioKbengt521
 
Standardizing Naming Conventions and Document Templates for Virtual Data Ware...
Standardizing Naming Conventions and Document Templates for Virtual Data Ware...Standardizing Naming Conventions and Document Templates for Virtual Data Ware...
Standardizing Naming Conventions and Document Templates for Virtual Data Ware...HMO Research Network
 

Similar to DWH Operations presentation (20)

Catalogic DPX: Dashboard Reporting with Microsoft Power BI
Catalogic DPX: Dashboard Reporting with Microsoft Power BICatalogic DPX: Dashboard Reporting with Microsoft Power BI
Catalogic DPX: Dashboard Reporting with Microsoft Power BI
 
Sprint 92
Sprint 92Sprint 92
Sprint 92
 
Dynamics 365 for Finance and Operations - Power BI
Dynamics 365 for Finance and Operations - Power BIDynamics 365 for Finance and Operations - Power BI
Dynamics 365 for Finance and Operations - Power BI
 
Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0
Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0
Enabling Fast Data Strategy: What’s new in Denodo Platform 6.0
 
Best practices and trends in people soft
Best practices and trends in people softBest practices and trends in people soft
Best practices and trends in people soft
 
Project Systems
Project SystemsProject Systems
Project Systems
 
Tracking Active Directory Changes
Tracking Active Directory ChangesTracking Active Directory Changes
Tracking Active Directory Changes
 
IBM Insight 2013 - Aetna's production experience using IBM DB2 Analytics Acce...
IBM Insight 2013 - Aetna's production experience using IBM DB2 Analytics Acce...IBM Insight 2013 - Aetna's production experience using IBM DB2 Analytics Acce...
IBM Insight 2013 - Aetna's production experience using IBM DB2 Analytics Acce...
 
Agile Testing Days 2017 Introducing AgileBI Sustainably
Agile Testing Days 2017 Introducing AgileBI SustainablyAgile Testing Days 2017 Introducing AgileBI Sustainably
Agile Testing Days 2017 Introducing AgileBI Sustainably
 
SAP BI/DW Training with BO Integration
SAP BI/DW Training with BO IntegrationSAP BI/DW Training with BO Integration
SAP BI/DW Training with BO Integration
 
KeyAchivementsMimecast
KeyAchivementsMimecastKeyAchivementsMimecast
KeyAchivementsMimecast
 
Order Review Solution Application (Version 2.0)
Order Review Solution Application (Version 2.0)Order Review Solution Application (Version 2.0)
Order Review Solution Application (Version 2.0)
 
MFP Teach Out
MFP Teach OutMFP Teach Out
MFP Teach Out
 
RPAS MFP Tech Support Teach-Out
RPAS MFP Tech Support Teach-OutRPAS MFP Tech Support Teach-Out
RPAS MFP Tech Support Teach-Out
 
BW_intro
BW_introBW_intro
BW_intro
 
SharePoint 2013 DR solution overview
SharePoint 2013 DR solution overviewSharePoint 2013 DR solution overview
SharePoint 2013 DR solution overview
 
5 Years of Progress in Active Data Warehousing
5 Years of Progress in Active Data Warehousing5 Years of Progress in Active Data Warehousing
5 Years of Progress in Active Data Warehousing
 
Presentation On Clean Up MRP records in Exact Globe
Presentation On Clean Up MRP records in Exact GlobePresentation On Clean Up MRP records in Exact Globe
Presentation On Clean Up MRP records in Exact Globe
 
Kevin Bengtson Portfolio
Kevin Bengtson PortfolioKevin Bengtson Portfolio
Kevin Bengtson Portfolio
 
Standardizing Naming Conventions and Document Templates for Virtual Data Ware...
Standardizing Naming Conventions and Document Templates for Virtual Data Ware...Standardizing Naming Conventions and Document Templates for Virtual Data Ware...
Standardizing Naming Conventions and Document Templates for Virtual Data Ware...
 

DWH Operations presentation

  • 1. DWH Operations Components of DWH Batch Sources: Files Reporting DB(BCV) DataStage Jobs: IN_STG_DAILY_BATCH_8PM IN_STG_DAILY_BATCH_1230AM TDS_IAA_HISTCALLS_GSM_0430AM HISTCALLS_TBL_AGGREGATED_DATA_0205AM IN_CRM_STG_DAILY_BATCH_0215AM IN_STG_SOR_DAILY_BATCH_0600AM
  • 2. DWH Operations Components of DWH batch Post Batch Activities: Batch completion message  Message audience Jhatpat message Important KPI’s message  Message Audience Daily Recharge SMS  Message Audience Report Triggers
  • 3. DWH Operations Components of DWH batch Batch Independent Jobs: VLR AKSA Erlang Hourly Recharge (every 4 hour) UNICA Jobs Other Batch Activities: Daily  AA Stats  Previous File directory deletion  DataStage EDWProd and important scripts backup
  • 4. DWH Operations Components of DWH batch Other Batch Activities: Weekly  SOR Stats  Product Arrangement DS Job Monthly  Data File Recon  DS Lock Release Activity  Monthly Recon
  • 5. DWH Operations Bottle Necks / Limitations Other Batch Activities: BCV completion time reduction Manual update of Lookup tables (e.g. GEOGRAPHIC_POINT, TAC_LIST, etc.) Number of session DataStage jobs create File availability issue (SDP dump, VLR, etc. ) Proper intimation to DWH team in case of any new product or critical change (e.g. CCN EMM file update) UNICA maintenance issues
  • 6. DWH Operations DWH Batch Improvements Other Batch Activities: Failure alerts monitoring to be monitored by NOC Predefined resource allocation (rotational) in case of batch failures Introduction of new checkpoints in DWH so that batch waits if critical table(s) are not properly loaded. BCV completion time reduction DWH purging criteria finalization for automated purging
  • 7. DWH Operations Documentation Completed: Warid Design Document (IBM) Warid EDWBI - User Procedure Manual (IBM) CR document DataStage Jobs error Log Netezza DB Stats DataStage Project Import & Export DataStage Lock Release DataStage Job change Incomplete: DWH Table and Job mappings
  • 9. DWH Operations Concerns & Suggestions Unless critical, we should avoid entertaining business users request after 6 pm Transactional activities that involve direct interaction with subscriber should be avoided in DWH