NUMBERATE 8.0 
Rhino 
Technology Overview 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Numberate Rapid Warehouse Solution 
The Numberate Rapid Warehouse Solution is a data integration 
solution that maximises the cost efficiency of data oriented business 
intelligence environments. 
It builds on over a 15 Years of experience delivering these solutions to 
a range of corporate and government clients who needed to leverage 
business growth from their organisations underlying data assets. 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Numberate Rapid Warehouse Solution 
The Numberate Rapid Warehouse Solution differs from other solutions 
because of its flexibility, its non-proprietary architecture and its speed of 
implementation. 
Based around the concept of business rules, it incorporates a high 
degree of reusability allowing customers to leverage and extend their 
existing investments in data warehousing and business intelligence. 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Numberate Rapid Warehouse Solution 
The Numberate Rapid Warehouse Solution is the right solution to meet the needs 
of your business for going forward. You need to give your IT professionals 
tools that; 
1. Allow them to focus on your business rather than our technology. 
2. Provide options and alternatives rather than mandating yet another upgrade or 
add on. 
3. Deliver in weeks rather than months. 
4. Are driven by your business rules and processes rather than driving your 
business broke. 
Data Warehousing & Business Intelligence Solutions 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Solution Requirements: Business 
THE CHALLENGE 
 Business users don’t 
understand their data 
 Information is power 
“Its distribution is 
threatening” 
 New information, or new 
views, changes 
requirements 
 Timeliness is everything 
“Yesterday’s issues are 
meaningless today.” 
THE NEEDS 
• Involve business to 
– Confirm solutions 
– Change their 
mind 
• Rapid development 
through automation 
• No penalties for 
rework 
• Return control of data 
and distribution to the 
business 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Solution Requirements: ETL 
THE CHALLENGE 
 Data from disparate systems doesn’t match 
 Cross systems business rules difficult to 
model 
 Dirty data blows out exception handling 
 Users change their mind 
 Upstream systems constantly change 
THE NEEDS 
 Capture business rules once 
 Allow users to prototype inside the ETL 
process 
 Automate the process 
• Maintaining the rules 
• Defining the transforms 
• Defining the target environment 
 Provide capacity for rework 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Solution Requirements: Project 
THE CHALLENGE 
 Development program midstream 
 Fixed price contract 
 Existing tool sets 
 Existing development partners 
 Cost constrained project 
 Project schedule pressure 
THE NEEDS 
 No upfront investment 
 Fixed price 
 Reduce development schedules 
 Reduce cost and time for work packages 
 Provide alternate approaches 
Data Warehousing & Numberate Data Warehous iBng u& Bsuisnineesss Inste llIignentcee Slloilugtioensnce SolDuecteimobner s2, 2014
A different approach 
Value Proposition 
The Numberate Rapid Warehouse Solution automates the process of building 
and maintaining a warehouse 
→ Open, off the shelf, commodity technologies. 
→Solution prototyped before decision. 
→ Performance benchmarked before full implementation 
→ Independent of target technology 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
A different approach to building 
a data warehouse 
Solution Space 
Warehouse Development Environment 
 Rapid transformations 
 Prototyping 
 Maintenance 
Warehouse Integration Environment 
 All major tools and RDBMS’s 
Warehouse Enhancements 
 Storage 
 Front end 
 Backup archive and recovery 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Numberate Architecture 
Front End 
Power Users 
Operational 
Systems 
Operational 
Systems 
Detailed Data 
Cache or Store 
ODBC 
Standard Reporting Tools 
Logical 
Views 
RDBMS API 
RDBMS 
Repository 
or Data 
Warehouse 
RDBMS ODBC 
ETL Environment 
Operational 
Systems 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Numberate Matrix 
Physical 
Data 
Elements 
Additional 
Load 
Code 
Raw 
Data 
Raw Data 
Load 
Execute 
Data 
Code 
Generation 
Engine 
User Data Request 
Result Set 
End User 
Logical 
Database 
Definition 
Request 
User Data 
User 
Query 
Builder 
Result 
Metadata 
Manager 
User 
Element 
Security 
Intersection 
Numberate Matrix 
Numberate Matrix Manager 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Technology Features 
 Platform Transparency 
 Uses inexpensive, standard components: 
 hardware, 
 software 
 Disk 
 Uses any storage devices orthogonally 
 disk, tape, CD ROM, 
 Independent of front end tools and back end data stores 
Horizontally scalable on any platform 
 Does not require identical hardware 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Value Proposition 
Single metadata 
repository 
 Extract and transform 
data 
 Define warehouse 
environment 
 Define third party 
environments 
 Maintain, schedule and 
operate warehouse 
Scalable storage 
solution 
 Terabyte enabled 
 Uses standard server / 
disk environments 
 Proprietary partitioning 
technology 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Metadata- Magic or Myth 
 Multiple standards 
 Multiple repositories 
 Residing in multiple separate systems 
 Addresses only a partial view of the environment 
 Not integrated with operational environments 
 Usually managed as a task additional to the 
implementation 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Metadata 
Business 
Rules 
Numberate 
Metadata 
User 
Regts 
Source 
database 
Existing 
Metadata 
SOURCES 
Numberate metadata 
generates efficient, 
error free code for 
the entire warehouse 
environment 
Manual 
Inputs 
Existing 
Datamart warehouse 
USES 
Extraction 
Routines Define 
Numberate 
Metadata 
Staging 
area 
Transformation 
programs 
Define 
warehouse 
tables 
Generate 
tests 
Define 
table 
loads 
Load 
frontend 
metadata 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Numberate Metadata – It’s Alive.. 
 Active 
 Automated 
 Accurate 
 Essential 
 Self documenting 
 End to end 
 Reusable 
 Extensible 
 Pervasive 
 One and done approach to 
build a warehouse 
 The tool manages the 
project work 
 Reduces effort, time and 
cost 
 Maintain the warehouse 
through the metadata 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Numberate Kernel 
Numberate Unplugged 
Code Repository 
RWS 
Admin 
Interface 
RWS 
User 
Interface 
Metadata activates 
program objects 
Metadata 
Repository 
3rd Party 
Metadata Interfaces 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Staging & Test area 
 Source transactional data is partitioned for parallel processing. 
 Partitioned at table level. 
 Can be repartitioned for performance tuning with minimal 
rework. 
 Parallel operations come from standard systems multi 
threading. 
Summaries 
And Derived 
Data 
Standard Operating System 
Java Environment 
Numberate Environment 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Unstrained Integration 
 By architecture 
 Hub and spoke 
 Enterprise Focused 
 Data Marts 
 Normalised / Schemas 
 ETL 
 Direct database load 
 Parallel ETL processes 
 Auditable 
 Low maintenance costs 
 Prototyping/ Front end 
 Derived from metadata 
 Low cost 
 Rapid development times 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Integration Services 
Front End Environments 
New Datamarts 
Emerging Technologies 
Numberate Integration Layer 
DW 
Legacy Repositories 
Data & Event Bus 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Data quality makes 
inferred relationships 
difficult to code 
Transformation 
Engine(s) 
Transformation 
Engine(s) 
Typical Transformation Architecture 
Transformation Engines are 
single stream, linear processes 
Target logical 
models cannot be 
validated by users 
without the data 
Disparate systems 
cannot sustain 
desired relationships 
Data Architectures 
and models may need 
to be radically altered: 
Performance 
Data Quality 
Business needs 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Traditional Transformation Process 
Iterate the Stream 
Oracle 
Source 
System 
Oracle 
Source 
System 
Teradata 
Target 
System 
Teradata 
Target 
System 
Cobol 
data 
extraction 
process 
Cobol 
data 
extraction 
process 
Fastload 
or 
Multiload 
Fastload 
or 
Multiload 
Teradata 
SQL 
Programs 
Teradata 
SQL 
Programs 
Transformation processes occur at all 
points 
• Difficult to plan 
• Hard to code manually 
• Very hard to change or rework 
Single stream approach 
• Limits transformation models 
• Requires many iterations. 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Traditional Transformation Process 
1. Use the warehouse for ETL 
2. Co-ordinate many environments 
3. Co-ordinate multiple teams 
4. High project management load 
5. High documentation load 
6. Target environment is not independent of ETL stream 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Non Proprietary Architecture 
The Numberate Rapid Warehouse Solution leverages the power 
of modern multi- tasking operating systems and advanced 
storage solutions to provide scalable data environments that can 
operate in near real time and provide rapid access to complex 
queries in business time. 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
ETL Strengths 
 ETL has full units of parallelism 
 Multi staged ETL developed as one process 
 Implemented as different streams 
SAS 
Utilities 
SQL 
C / Cobol 
 Sophisticated high speed pattern 
matching 
 Matrix Input Loader 
 Generic Data Conversion Module 
 Saves tens of thousands of lines of code 
 Generated code is error free 
 Metadata driven 
 End to end – including warehouse definition 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Elements of Compatibility 
 STORAGE 
 Disk / tape / CD-rom 
 Appear as RDBMS views 
 Low cost storage 
 Suitable for transactional histories 
 DATA ARCHITECTURE 
 Normalised data models 
 Star Schemas 
 Snowflake Schemas 
 DATA STORAGE 
 Mix and match capability 
 Oracle / DB2 / Teradata/SAS 
 Native file systems 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Meeting Your Needs 
End to end automation reduces time and cost 
-Provide faster, cheaper work packages 
Capture existing sources of metadata 
-Warehouse work already completed 
-Definitions in old repositories and data marts 
Provide you with more options 
-Transformation can be done on 
• Mainframe 
• Data Warehouse 
• Unix 
-Move between platforms as run time option 
Solution includes a RAD / prototyping environment 
-Model and confirm business requirements 
-More accurate reporting specifications 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
Numberate Rapid Warehouse Solution 
If your organisation is ready to move to a 
new level of robust solutions that actually 
reduce your IT spend, then you are 
probably ready for the 
Numberate Rapid Warehouse Solution 
Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014

Numberate master presentation

  • 1.
    NUMBERATE 8.0 Rhino Technology Overview Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 2.
    Numberate Rapid WarehouseSolution The Numberate Rapid Warehouse Solution is a data integration solution that maximises the cost efficiency of data oriented business intelligence environments. It builds on over a 15 Years of experience delivering these solutions to a range of corporate and government clients who needed to leverage business growth from their organisations underlying data assets. Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 3.
    Numberate Rapid WarehouseSolution The Numberate Rapid Warehouse Solution differs from other solutions because of its flexibility, its non-proprietary architecture and its speed of implementation. Based around the concept of business rules, it incorporates a high degree of reusability allowing customers to leverage and extend their existing investments in data warehousing and business intelligence. Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 4.
    Numberate Rapid WarehouseSolution The Numberate Rapid Warehouse Solution is the right solution to meet the needs of your business for going forward. You need to give your IT professionals tools that; 1. Allow them to focus on your business rather than our technology. 2. Provide options and alternatives rather than mandating yet another upgrade or add on. 3. Deliver in weeks rather than months. 4. Are driven by your business rules and processes rather than driving your business broke. Data Warehousing & Business Intelligence Solutions Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 5.
    Solution Requirements: Business THE CHALLENGE  Business users don’t understand their data  Information is power “Its distribution is threatening”  New information, or new views, changes requirements  Timeliness is everything “Yesterday’s issues are meaningless today.” THE NEEDS • Involve business to – Confirm solutions – Change their mind • Rapid development through automation • No penalties for rework • Return control of data and distribution to the business Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 6.
    Solution Requirements: ETL THE CHALLENGE  Data from disparate systems doesn’t match  Cross systems business rules difficult to model  Dirty data blows out exception handling  Users change their mind  Upstream systems constantly change THE NEEDS  Capture business rules once  Allow users to prototype inside the ETL process  Automate the process • Maintaining the rules • Defining the transforms • Defining the target environment  Provide capacity for rework Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 7.
    Solution Requirements: Project THE CHALLENGE  Development program midstream  Fixed price contract  Existing tool sets  Existing development partners  Cost constrained project  Project schedule pressure THE NEEDS  No upfront investment  Fixed price  Reduce development schedules  Reduce cost and time for work packages  Provide alternate approaches Data Warehousing & Numberate Data Warehous iBng u& Bsuisnineesss Inste llIignentcee Slloilugtioensnce SolDuecteimobner s2, 2014
  • 8.
    A different approach Value Proposition The Numberate Rapid Warehouse Solution automates the process of building and maintaining a warehouse → Open, off the shelf, commodity technologies. →Solution prototyped before decision. → Performance benchmarked before full implementation → Independent of target technology Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 9.
    A different approachto building a data warehouse Solution Space Warehouse Development Environment  Rapid transformations  Prototyping  Maintenance Warehouse Integration Environment  All major tools and RDBMS’s Warehouse Enhancements  Storage  Front end  Backup archive and recovery Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 10.
    Numberate Architecture FrontEnd Power Users Operational Systems Operational Systems Detailed Data Cache or Store ODBC Standard Reporting Tools Logical Views RDBMS API RDBMS Repository or Data Warehouse RDBMS ODBC ETL Environment Operational Systems Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 11.
    Numberate Matrix Physical Data Elements Additional Load Code Raw Data Raw Data Load Execute Data Code Generation Engine User Data Request Result Set End User Logical Database Definition Request User Data User Query Builder Result Metadata Manager User Element Security Intersection Numberate Matrix Numberate Matrix Manager Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 12.
    Technology Features Platform Transparency  Uses inexpensive, standard components:  hardware,  software  Disk  Uses any storage devices orthogonally  disk, tape, CD ROM,  Independent of front end tools and back end data stores Horizontally scalable on any platform  Does not require identical hardware Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 13.
    Value Proposition Singlemetadata repository  Extract and transform data  Define warehouse environment  Define third party environments  Maintain, schedule and operate warehouse Scalable storage solution  Terabyte enabled  Uses standard server / disk environments  Proprietary partitioning technology Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 14.
    Metadata- Magic orMyth  Multiple standards  Multiple repositories  Residing in multiple separate systems  Addresses only a partial view of the environment  Not integrated with operational environments  Usually managed as a task additional to the implementation Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 15.
    Metadata Business Rules Numberate Metadata User Regts Source database Existing Metadata SOURCES Numberate metadata generates efficient, error free code for the entire warehouse environment Manual Inputs Existing Datamart warehouse USES Extraction Routines Define Numberate Metadata Staging area Transformation programs Define warehouse tables Generate tests Define table loads Load frontend metadata Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 16.
    Numberate Metadata –It’s Alive..  Active  Automated  Accurate  Essential  Self documenting  End to end  Reusable  Extensible  Pervasive  One and done approach to build a warehouse  The tool manages the project work  Reduces effort, time and cost  Maintain the warehouse through the metadata Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 17.
    Numberate Kernel NumberateUnplugged Code Repository RWS Admin Interface RWS User Interface Metadata activates program objects Metadata Repository 3rd Party Metadata Interfaces Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 18.
    Staging & Testarea  Source transactional data is partitioned for parallel processing.  Partitioned at table level.  Can be repartitioned for performance tuning with minimal rework.  Parallel operations come from standard systems multi threading. Summaries And Derived Data Standard Operating System Java Environment Numberate Environment Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 19.
    Unstrained Integration By architecture  Hub and spoke  Enterprise Focused  Data Marts  Normalised / Schemas  ETL  Direct database load  Parallel ETL processes  Auditable  Low maintenance costs  Prototyping/ Front end  Derived from metadata  Low cost  Rapid development times Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 20.
    Integration Services FrontEnd Environments New Datamarts Emerging Technologies Numberate Integration Layer DW Legacy Repositories Data & Event Bus Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 21.
    Data quality makes inferred relationships difficult to code Transformation Engine(s) Transformation Engine(s) Typical Transformation Architecture Transformation Engines are single stream, linear processes Target logical models cannot be validated by users without the data Disparate systems cannot sustain desired relationships Data Architectures and models may need to be radically altered: Performance Data Quality Business needs Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 22.
    Traditional Transformation Process Iterate the Stream Oracle Source System Oracle Source System Teradata Target System Teradata Target System Cobol data extraction process Cobol data extraction process Fastload or Multiload Fastload or Multiload Teradata SQL Programs Teradata SQL Programs Transformation processes occur at all points • Difficult to plan • Hard to code manually • Very hard to change or rework Single stream approach • Limits transformation models • Requires many iterations. Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 23.
    Traditional Transformation Process 1. Use the warehouse for ETL 2. Co-ordinate many environments 3. Co-ordinate multiple teams 4. High project management load 5. High documentation load 6. Target environment is not independent of ETL stream Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 24.
    Non Proprietary Architecture The Numberate Rapid Warehouse Solution leverages the power of modern multi- tasking operating systems and advanced storage solutions to provide scalable data environments that can operate in near real time and provide rapid access to complex queries in business time. Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 25.
    ETL Strengths ETL has full units of parallelism  Multi staged ETL developed as one process  Implemented as different streams SAS Utilities SQL C / Cobol  Sophisticated high speed pattern matching  Matrix Input Loader  Generic Data Conversion Module  Saves tens of thousands of lines of code  Generated code is error free  Metadata driven  End to end – including warehouse definition Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 26.
    Elements of Compatibility  STORAGE  Disk / tape / CD-rom  Appear as RDBMS views  Low cost storage  Suitable for transactional histories  DATA ARCHITECTURE  Normalised data models  Star Schemas  Snowflake Schemas  DATA STORAGE  Mix and match capability  Oracle / DB2 / Teradata/SAS  Native file systems Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 27.
    Meeting Your Needs End to end automation reduces time and cost -Provide faster, cheaper work packages Capture existing sources of metadata -Warehouse work already completed -Definitions in old repositories and data marts Provide you with more options -Transformation can be done on • Mainframe • Data Warehouse • Unix -Move between platforms as run time option Solution includes a RAD / prototyping environment -Model and confirm business requirements -More accurate reporting specifications Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014
  • 28.
    Numberate Rapid WarehouseSolution If your organisation is ready to move to a new level of robust solutions that actually reduce your IT spend, then you are probably ready for the Numberate Rapid Warehouse Solution Numberate Data Warehousing & Business Intelligence Solutions December 2, 2014