2. Vensai Consultants is an IT Consulting
firm which specializes in providing
Strategic leadership, Architectural
direction, Resourcing ramp-up, IT
Portfolio Management &
Implementation of Data Warehouses.
WHO ARE WE?
3. We are a team of IT Architects specialized in implementing Data
Warehouses based on customer's needs. We specialize in building
Reporting systems all the way from understanding data sources to
building a presentation layer.
5. Data Warehousing Today : Facts
Information processing using Microsoft
Office Productivity software is long gone.
Data is being collected in various formats
- Tables, XML, Messages, Images, BLOBs,
CLOBs etc.
6. Data Warehouses are driving the
analytical and decision support
systems.
99% of corporations have Data
Warehouse shops attempting to
process the data into useful
information for the Company.
7. Vensai Consultants believe that building a Data
Warehouse shouldn't be a daunting exercise once the
roadmap is adopted. Most companies fail in their data
warehouse efforts because :
8. Reporting requirements are not understood at the beginning of
the project
Software Development life cycle is plagued with
process deficiencies
Lack of Strategic direction & Monetary resources
9. WHAT IF?
The Client has access to a team of IT
Architects who have versatile experience
across several tools related to Data
Warehousing, who have sound foundation
of Agile methodology in developing Data
Warehouses, who have sound strategy in
overcoming the challenges related to
requirements and who cost a fraction of
the typical IT development shops?
11. INTRODUCING VENSAI CONSULTANTS, LLC
A team of Data Warehouse consultants with
diverse experience in all facets of development
life cycle. Our consultants have extensive
experience in Healthcare, Retail, Banking and
Defense sectors. We have access to several IT
Vendors who can assist the project in
procuring and implementation the required
software/hardware for building a Data
Warehouse. Our team is also well-versed with
Big Data technologies.
21. Data Integration Services
DATA PROFILING
DATA MAPPING,
TRANSFORMATION AND
DERIVED DATA
Data Quality Measures
Event Logging & Auditing
Data Discovery
Data Validation
Identity Mapping (Master
Data Management)
Generate Core Facts
Data Mining
References Data Taxonomies
Generate Dimensions (Type 1-3)
Generate Fact Groups (Get
Surrogates, Generate Stars)
22. DATA PROFILING
Data Discovery
(Data Profiling, Metadata collection, statistics,
histograms, Alerts, Etc..)
Data Validation
(RI, Conditionals, Data Types, Valid
Values, Cleansing, Scrubbing)
Data
Profiling
Area
DATABASE
23. DATABASE
Integration &
Transformation
Area
DATA MAPPING, TRANSFORMATION AND
DERIVED DATA
Identity Mapping (Master Data Management)
Matching Algorithms, Dimensional Authority, Data Domains Setup
Generate Core Facts
Natural Keys only (Type1-3)
Data Mining
Grouper Generative Routines (MEG, PEG etc.,)
References Data Taxonomies
Develop Crosswalks, Taxonomies
Generate Dimensions (Type 1-3)
Generate Fact Groups (Get Surrogates, Generate Stars)
24. DQ & Exception
Technical Audit
Area
DATABASE DATABASE
Audit, Balance &
Control Area
Data Quality Measures
Event Logging & Auditing
Define Quality Measures, Measure,
Remediation, Load & Reload)
System Event Logging, Auditing,
Balance & Control
DATABASE
27. Core Data Area
Atomic (Core) Facts
Database
Conformed Dimensions
Database
ETL – Generate Pre-Stored Aggregates
28. Data Marts
Value Proposition KPIs
DATABASE
Aggregate Snapshots
Pre-Aggregated Measures/Metrics Divisional Star-Schemas
DATABASE
DATABASE
DATABASE
29. Operational Admin Data &
Operational Decision Support
DATABASE DATABASE
Data Quality and Audit Logging Metadata Repository
DATABASE
Process Dashboards, Event Monitoring, Warehouse Controls
38. DATA WAREHOUSE
Purpose: Provides framework for creating
and managing change requests on all aspect
of application development.
Change Management
Purpose: Maintain the IT Systems after the
end-of development life cycle. Workflow,
Scheduling, Process models, Process grouping,
Interactions etc.
IT Support
Purpose: Audit mechanism for tallying
the processed data across Data
Warehouse layers. Error Handling,
Consolidation and reporting.
Audit, Balance &
Control
Purpose: Systems, Projects, Vendors,
Network Hardware, Software etc.,
IT Portfolio
Managements
Purpose: Administration access,
configuration, release management,
deployment of application servers.
Application
Administration
Purpose: Creates application code
to move data from one layer to
other
Application
Development
Purpose: Provides, maintains and
supports the authoritative reference
data along with taxonomies
Reference Data
Management
Purpose: Creates, maintains and supports
the metadata artifacts like data models, data
packages, App Dev Repositories etc.,
Data Architecture
Purpose: Define security policies
and manages the data acccess to
individuals and processes
Data Security &
access Control
Purpose: Create and manages mapping
documents and data transformation
rules
Data Integration
Purpose: Defines and manages the Data
Quality KPIs to assure the quality of the
Data Warehouse
Data Quality
Purpose: Creates a frameworks for
storing & publishing all metadata
content related to a Data Warehouse
Metadata Management
40. DATA ACQUISITION LAYER
Houses only the changed records from
the source system
Optimized for faster loading (Ex:
No indexes or Constraints)
Truncate before each load
Only INSERTS into this layer
Access to ETL processes only
Apply business and project specific filter
on data before this layer
Replica Structures of the
Source System
NO User access at all
Exception: Data Retention period
is limited to 2-6 months
Cleansed Data only
Data Load Frequency is requirement specific
(Daily, Weekly, Monthly)
Minimize processing impact on
application source database.
41. CANONICAL LAYER
Combine data from Multiple Sources
Denormalized Atomic Transaction Tables
Metadata Conformance
Transaction History accumulated for
specific period of time (Ex: 7 years)
NO User access at all
Snapshots at specific changes to a
transaction may be maintained
Purge Criteria will be established
42. Preferably STAR Schema
Modeling of Data Structures
Lookup Reference data is
loaded into the dimensions
Optimized for faster access (EX:
Many indexes, Partitioning)
access to all downstream processes
Transaction History accumulation for
specific period of time (Ex: 7 years)
access to all privileged users
DATAMART LAYER
44. SUGGESTED TOOLS REQUIRED FOR DEVELOPING & MAINTAINING A
DATA WAREHOUSE
13
121110
987
654
321 Data Modeling Relational Database
Management Systems
Non-Relational
Database Systems
ETL
Master Data
Management
Analytics
Reporting Data Profiling Job Control
Performance
Monitoring
Unix/Mainframe OS
Access
Identify Management
Support Management
48. ETL
Purpose: Extract,
Transform & Load of
data.
Informatica Power
Center
SSIS
Ab Initio
Cognos Decision
Streams
SAS DI Studio
Oracle Data
Integrator
49. MASTER DATA MANAGEMENT
IBM Initiate
Purpose: Create, retrieve Master data.
Nextgate MatchMetrix
Informatica MDM
51. REPORTING
Oracle Business Intelligence EE
Business Objects
Cognos
SSRS
Pyramid Analytics
TIBCO Spotfire
Microstrategy
SAP InfoMaker
Microsoft access
Purpose: Writing Reports.
58. ABOVE ALL
Vensai Consultants, LLC is a woman owned small business based out of Maryland.
We specialize in tailoring the solutions based on the client needs & budget. We
are available as a team and as well as on a consultant basis. Given our technical
acumen, we are sure that Vensail Consultants, LLC would be a value proposition to
our clients.
