The document discusses an operational data store (ODS) that was implemented to integrate data from two banks, Velká česká banka and Nová česká banka, after a transaction integration, using APIs, ETL workflows, and data transformations to populate the ODS with consolidated customer, account, and transaction data from both banks for operational reporting. It also provides details on the types of data domains integrated into the ODS and growth in API usage over time as more systems accessed the shared ODS.
this is the ppt this contains definition of data ware house , data , ware house, data modeling , data warehouse architecture and its type , data warehouse types, single tire, two tire, three tire .
Data Warehousing is a data architecture that separates reporting and analytics needs from operational transaction systems. This presentation is an introduction into traditional data warehousing architectures and how to determine if your environment requires a data warehouse.
In computing, a data warehouse (DW or DWH), also known as an enterprise data warehouse (EDW), is a system used for reporting and data analysis, and is considered a core component of business intelligence.[1] DWs are central repositories of integrated data from one or more disparate sources. They store current and historical data in one single place that are used for creating analytical reports for knowledge workers throughout the enterprise.
Designing Fast Data Architecture for Big Data using Logical Data Warehouse a...Denodo
Companies such as Autodesk are fast replacing the once-true- and-tried physical data warehouses with logical data warehouses/ data lakes. Why? Because they are able to accomplish the same results in 1/6 th of the time and with 1/4 th of the resources.
In this webinar, Autodesk’s Platform Lead, Kurt Jackson,, will describe how they designed a modern fast data architecture as a single unified logical data warehouse/ data lake using data virtualization and contemporary big data analytics like Spark.
Logical data warehouse / data lake is a virtual abstraction layer over the physical data warehouse, big data repositories, cloud, and other enterprise applications. It unifies both structured and unstructured data in real-time to power analytical and operational use cases.
Designing high performance datawarehouseUday Kothari
Just when the world of “Data 1.0” showed some signs of maturing; the “Outside In” driven demands seem to have already initiated some the disruptive changes to the data landscape. Parallel growth in volume, velocity and variety of data coupled with incessant war on finding newer insights and value from data has posed a Big Question: Is Your Data Warehouse Relevant?
In short, the surrounding changes happening real time is the new “Data 2.0”. It is characterized by feeding the ever hungry minds with sharper insights whether it is related to regulation, finance, corporate action, risk management or purely aimed at improving operational efficiencies. The source in this new “Data 2.0” has to be commensurate to the outside in demands from customers, regulators, stakeholders and business users; and hence, you would need a high relformance (relevance + performance) data warehouse which will be relevant to your business eco-system and will have the power to scale exponentially.
We starts this webinar by giving the audiences a sneak preview of what happened in the Data 1.0 world & which characteristics are shaping the new Data 2.0 world. It then delves deep on the challenges that growing data volumes have posed to the Data warehouse teams. It also presents the audiences some of the practical and proven methodologies to address these performance challenges. Finally, in the end it will highlight some of the thought provoking ways to turbo charge your data warehouse related initiatives by leveraging some of the newer technologies like Hadoop. Overall, the webinar will educate audiences with building high performance and relevant data warehouses which is capable of meeting the newer demands while significantly driving down the total cost of ownership.
These are the slides from my talk at Data Day Texas 2016 (#ddtx16).
The world of data warehousing has changed! With the advent of Big Data, Streaming Data, IoT, and The Cloud, what is a modern data management professional to do? It may seem to be a very different world with different concepts, terms, and techniques. Or is it? Lots of people still talk about having a data warehouse or several data marts across their organization. But what does that really mean today in 2016? How about the Corporate Information Factory (CIF), the Data Vault, an Operational Data Store (ODS), or just star schemas? Where do they fit now (or do they)? And now we have the Extended Data Warehouse (XDW) as well. How do all these things help us bring value and data-based decisions to our organizations? Where do Big Data and the Cloud fit? Is there a coherent architecture we can define? This talk will endeavor to cut through the hype and the buzzword bingo to help you figure out what part of this is helpful. I will discuss what I have seen in the real world (working and not working!) and a bit of where I think we are going and need to go in 2016 and beyond.
Are You Killing the Benefits of Your Data Lake?Denodo
Watch the full webinar on-demand here: https://goo.gl/RL1ZSa
Data lakes are centralized data repositories. Data needed by data scientists is physically copied to a data lake which serves as a one storage environment. This way, data scientists can access all the data from only one entry point – a one-stop shop to get the right data. However, such an approach is not always feasible for all the data and limits it’s use to solely data scientists, making it a single-purpose system.
So, what’s the solution?
A multi-purpose data lake allows a broader and deeper use of the data lake without minimizing the potential value for data science and without making it an inflexible environment
Attend this session to learn:
• Disadvantages and limitations that are weakening or even killing the potential benefits of a data lake.
• Why a multi-purpose data lake is essential in building a universal data delivery system.
• How to build a logical multi-purpose data lake using data virtualization.
Do not miss this opportunity to make your data lake project successful and beneficial.
Know different types of tips about Importance of dataware housing, Data Cleansing and Extracting etc . For more details visit: http://www.skylinecollege.com/business-analytics-course
Partner Enablement: Key Differentiators of Denodo Platform 6.0 for the FieldDenodo
If you’re a Denodo Partner, this presentation is for you. Learn how to gain a competitive edge in the marketplace with Denodo Platform 6.0, and leverage off the new features and functionality.
This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/Qh8MeX.
Traditional BI vs. Business Data Lake – A ComparisonCapgemini
Traditional Business Intelligence (BI) systems provide various levels and kinds of analyses on structured data but they are not designed to handle unstructured data.
For these systems Big Data brings big problems because the data that flows in may be either structured or unstructured. That makes them hugely limited when it comes to delivering Big Data benefits.
The way forward is a complete rethink of the way we use BI - in terms of how the data is ingested, stored and analyzed.
More information: http://www.capgemini.com/big-data-analytics/pivotal
this is the ppt this contains definition of data ware house , data , ware house, data modeling , data warehouse architecture and its type , data warehouse types, single tire, two tire, three tire .
Data Warehousing is a data architecture that separates reporting and analytics needs from operational transaction systems. This presentation is an introduction into traditional data warehousing architectures and how to determine if your environment requires a data warehouse.
In computing, a data warehouse (DW or DWH), also known as an enterprise data warehouse (EDW), is a system used for reporting and data analysis, and is considered a core component of business intelligence.[1] DWs are central repositories of integrated data from one or more disparate sources. They store current and historical data in one single place that are used for creating analytical reports for knowledge workers throughout the enterprise.
Designing Fast Data Architecture for Big Data using Logical Data Warehouse a...Denodo
Companies such as Autodesk are fast replacing the once-true- and-tried physical data warehouses with logical data warehouses/ data lakes. Why? Because they are able to accomplish the same results in 1/6 th of the time and with 1/4 th of the resources.
In this webinar, Autodesk’s Platform Lead, Kurt Jackson,, will describe how they designed a modern fast data architecture as a single unified logical data warehouse/ data lake using data virtualization and contemporary big data analytics like Spark.
Logical data warehouse / data lake is a virtual abstraction layer over the physical data warehouse, big data repositories, cloud, and other enterprise applications. It unifies both structured and unstructured data in real-time to power analytical and operational use cases.
Designing high performance datawarehouseUday Kothari
Just when the world of “Data 1.0” showed some signs of maturing; the “Outside In” driven demands seem to have already initiated some the disruptive changes to the data landscape. Parallel growth in volume, velocity and variety of data coupled with incessant war on finding newer insights and value from data has posed a Big Question: Is Your Data Warehouse Relevant?
In short, the surrounding changes happening real time is the new “Data 2.0”. It is characterized by feeding the ever hungry minds with sharper insights whether it is related to regulation, finance, corporate action, risk management or purely aimed at improving operational efficiencies. The source in this new “Data 2.0” has to be commensurate to the outside in demands from customers, regulators, stakeholders and business users; and hence, you would need a high relformance (relevance + performance) data warehouse which will be relevant to your business eco-system and will have the power to scale exponentially.
We starts this webinar by giving the audiences a sneak preview of what happened in the Data 1.0 world & which characteristics are shaping the new Data 2.0 world. It then delves deep on the challenges that growing data volumes have posed to the Data warehouse teams. It also presents the audiences some of the practical and proven methodologies to address these performance challenges. Finally, in the end it will highlight some of the thought provoking ways to turbo charge your data warehouse related initiatives by leveraging some of the newer technologies like Hadoop. Overall, the webinar will educate audiences with building high performance and relevant data warehouses which is capable of meeting the newer demands while significantly driving down the total cost of ownership.
These are the slides from my talk at Data Day Texas 2016 (#ddtx16).
The world of data warehousing has changed! With the advent of Big Data, Streaming Data, IoT, and The Cloud, what is a modern data management professional to do? It may seem to be a very different world with different concepts, terms, and techniques. Or is it? Lots of people still talk about having a data warehouse or several data marts across their organization. But what does that really mean today in 2016? How about the Corporate Information Factory (CIF), the Data Vault, an Operational Data Store (ODS), or just star schemas? Where do they fit now (or do they)? And now we have the Extended Data Warehouse (XDW) as well. How do all these things help us bring value and data-based decisions to our organizations? Where do Big Data and the Cloud fit? Is there a coherent architecture we can define? This talk will endeavor to cut through the hype and the buzzword bingo to help you figure out what part of this is helpful. I will discuss what I have seen in the real world (working and not working!) and a bit of where I think we are going and need to go in 2016 and beyond.
Are You Killing the Benefits of Your Data Lake?Denodo
Watch the full webinar on-demand here: https://goo.gl/RL1ZSa
Data lakes are centralized data repositories. Data needed by data scientists is physically copied to a data lake which serves as a one storage environment. This way, data scientists can access all the data from only one entry point – a one-stop shop to get the right data. However, such an approach is not always feasible for all the data and limits it’s use to solely data scientists, making it a single-purpose system.
So, what’s the solution?
A multi-purpose data lake allows a broader and deeper use of the data lake without minimizing the potential value for data science and without making it an inflexible environment
Attend this session to learn:
• Disadvantages and limitations that are weakening or even killing the potential benefits of a data lake.
• Why a multi-purpose data lake is essential in building a universal data delivery system.
• How to build a logical multi-purpose data lake using data virtualization.
Do not miss this opportunity to make your data lake project successful and beneficial.
Know different types of tips about Importance of dataware housing, Data Cleansing and Extracting etc . For more details visit: http://www.skylinecollege.com/business-analytics-course
Partner Enablement: Key Differentiators of Denodo Platform 6.0 for the FieldDenodo
If you’re a Denodo Partner, this presentation is for you. Learn how to gain a competitive edge in the marketplace with Denodo Platform 6.0, and leverage off the new features and functionality.
This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/Qh8MeX.
Traditional BI vs. Business Data Lake – A ComparisonCapgemini
Traditional Business Intelligence (BI) systems provide various levels and kinds of analyses on structured data but they are not designed to handle unstructured data.
For these systems Big Data brings big problems because the data that flows in may be either structured or unstructured. That makes them hugely limited when it comes to delivering Big Data benefits.
The way forward is a complete rethink of the way we use BI - in terms of how the data is ingested, stored and analyzed.
More information: http://www.capgemini.com/big-data-analytics/pivotal
The Pivotal Business Data Lake provides a flexible blueprint to meet your business's future information and analytics needs while avoiding the pitfalls of typical EDW implementations. Pivotal’s products will help you overcome challenges like reconciling corporate and local needs, providing real-time access to all types of data, integrating data from multiple sources and in multiple formats, and supporting ad hoc analysis.
How to Place Data at the Center of Digital Transformation in BFSIDenodo
Watch full webinar here: https://bit.ly/3j7E9Jo
Consumers are increasingly using digital banking tools and insurance models, and these numbers will only continue to grow. Financial and insurance organizations have to adapt to the new and always changing situation while complying with new regulations, such as IFRS17, and embracing ESG criteria.
At the heart of any digital transformation is data. Therefore, it is not a stretch to say that data management and analytics strategies differentiate many of the leaders from the laggards in the banking, financial services and insurance (BFSI) industry. BFSI organizations still relying on slow, traditional systems and data management processes will find themselves falling behind their competition. In addition, as many adopt cloud strategies, these traditional approaches fill the cloud modernization process with downtime and end user frustration. In fact, according to a McKinsey article, cloud combined with distributed data infrastructure will define how consumers and providers adopt digital insurance models for the next decade.
Hear how the BFSI industry is leveraging data virtualization to deploy data fabric or data mesh architectures for enterprise-wide digital transformation.
Join this webinar to learn:
- The latest trends in BFSI for 2023 and how data and analytics is reshaping the industry
- How a logical data architecture can help you capitalize on your data
- How Denodo customers digitally transformed themselves using the Denodo Platform
Building an Effective Data Warehouse ArchitectureJames Serra
Why use a data warehouse? What is the best methodology to use when creating a data warehouse? Should I use a normalized or dimensional approach? What is the difference between the Kimball and Inmon methodologies? Does the new Tabular model in SQL Server 2012 change things? What is the difference between a data warehouse and a data mart? Is there hardware that is optimized for a data warehouse? What if I have a ton of data? During this session James will help you to answer these questions.
An Introduction to Data Virtualization in 2018Denodo
Watch full webinar on demand here: https://goo.gl/Rdrc1w
"Through 2020, 50% of enterprises will implement some form of data virtualization as one enterprise production option for data integration" according to Gartner. It is clear that data virtualization has become a driving force for companies to implement an agile, real-time and flexible enterprise data architecture.
Attend this session to learn:
• What data virtualization actually means and how it differs from traditional data integration approaches
• The all important use cases and key patterns of data virtualization
• What to expect in the upcoming sessions in the Packed Lunch Webinar Series, which will take a deeper dive into various challenges solved by data virtualization in big data analytics, cloud migration and various other scenarios
Agenda:
• Introduction & benefits of DV
• Summary & next steps
• Q&A
The seminar is about Data warehousing, in here we are gonna discuss about what is data warehousing, comparison b/w database and data warehouse, different data warehouse models.about Data mart, and disadvantages of data warehousing.
Watch Paul's session from Fast Data Strategy on-demand here: https://goo.gl/3veKqw
"Through 2020, 50% of enterprises will implement some form of data virtualization as one enterprise production option for data integration" according to Gartner. It is clear that data virtualization has become a driving force for companies to implement an agile, real-time and flexible enterprise data architecture.
Attend this session to learn:
• What data virtualization actually means and how it differs from traditional data integration approaches
• The most important use cases and key patterns of data virtualization
• The benefits of data virtualization
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
3. Prague Data Management Meetup
Data Management
Získávaní dat
Ukládání dat
Zpracování dat
Interpretace dat
Použití dat
• Otevřená profesionální zájmová
skupina
• Každý je vítán (ať už v pasivní
nebo aktivní roli)
• Témat není nikdy dost
• Snaha o pravidelné měsíční
setkávání
• Fungujeme od září 2015
4. Historie
Datum Téma
10. 9. 2015 Data Management
14. 10. 2015 Data Lake
23. 11. 2015 Dark Data (without Dark Energy and Dark Force)
12. 1. 2016 Data Lake (znova)
7. 3. 2016 Sad Stories About DW Modeling (sad stories only)
23. 3. 2016 Self-service BI Street Battle
27. 4. 2016 Let's explore the new Microsoft PowerBI!
22. 9. 2016 Data Management pro začátečníky
17. 10. 2016 Small Big Data
22. 11. 2016 Základy modelování DW
23.1.2017 Komponenty datových skladů
28.2.2017 Operational Data Store
8. Operational Database vs. Data Warehouse
Characteristic Operational Database Data Warehouse
Time focus Current Historical
Details level Individual Individual and summary
Orientation Process Subject
Records per request Few Thousands
Normalization level Mostly normalized Normalization relaxed
Update level Highly volatile Mostly refreshed (non volatile)
Data model Relational (3NF) Relational (star schemas, hybrid, 3NF) and multidimensional
(data cubes)
Source: CourseraOperational Data Store
9. Inmon, Imhoff & Battas ODS Definition
• Features:
• Subject-oriented (like a data warehouse)
• Made up of integrated data (standard, consistent data formats)
• Volatile (changes as often as the source system)
• Current (low-latency data capture; no historical detail)
• Defined in the mid-1990s
• Later Adopted by Gartner, Inc.
• When limited in scope to customer or product data, the canonical
ODS is similar to master data management (MDM).
9
10. Adastra Business Intelligence Reference Architecture
10
ODS
Operational
reporting
Enterprise DWH Big Data
Platform
Data Lake
Event
Processing
Semantic
Models
Advanced Analytics
Perceptual / cognitive intelligence
Information Management
Relational / Structured data Unstructured data Streaming
Data Workflow
Orchestration
Data Transformation /
Processing
Data
Management
Event Ingestion
Complex Event
Processing
Notifications
BI / Application
Integration
Machine Learning
In-database Data Mining, R
Recognition of human
interaction and intent
SMP and MPP
In-memory technologies
In-memory Columnar
In-memory technologies Hadoop, NoSQL
Business Intelligence / Data Delivery
Real-time DashboardsDashboards and visualizationsReports Self-service BIMobile BI
IoT Network
Field Gateway
Big data
OLAP
11. Architecture Reasons for ODS
• Copy vs. Reference - why copy data into ODS?
• Performance issues
• Faster local data access
• Load distribution (Operational and Reporting)
• Time issues
• Less granularity of secondary system
• History
• Availability issues
• e.g. primary 10x5, secondary 24x7
• Consolidation issues
• e.g. Consolidated client, product
• Security issues
11
12. ODS Possible Roles in Architecture
• ODS as data store for operational processes (PDI/CDI)
• ODS as DWH stage
• ODS as operational reporting data source
• ODS as data exchange component
• ODS as data cache for other systems
• ODS as MDM solution
• ODS as replacement of legacy system
• ODS as DWH data load type (near-real time DWH)
12
13. Truth in data
13
Primary data
Primary data
(another system)
Secondary data
Consolidated data
…Noise generator
Truth
• Independent truth in data does not exist
• Truth depends on Business and Data architect definition
14. Inmon ODS Classes
• Class I. (Real-Time ODS)
• Transactions were moved to th e ODS in an immediate manner from applications in a range
of one to two seconds from the moment the transaction was executed in the operational
environment until the transaction arrived at the ODS. In this case, the end user could hardly
tell the difference between an activity that had occurred in the operational environment and
the same activity as it was transmitted in the ODS environment.
• Class II. (Near Real-Time ODS)
• Activities that occurred in the operational environment were stored and forwarded to the
ODS every four hours or so. In this case, there was a noticeable lag between the original
execution of the transaction and the reflection of that transaction in the ODS environment.
However, this class of ODS was much easier to build and to operate than a Class I ODS.
• Class III. (Daily ODS)
• The time lag between execution in the operational environment and reflection in the ODS is
overnight. In a Class III ODS there is a noticeable time lag between the execution of the
transaction in the operational environment and the reflection of the transaction in the ODS
environment. This type of ODS is relatively easy to build.
• Class IV. (Datawarehouse ODS)
• A Class IV ODS is one that is fed from the data warehouse from analysis created by the DSS
analyst in the data warehouse environment and condensed down to a point where the
results of the analytical processing fit comfortably in the ODS. The input to the ODS can be
either regular or irregular. This class of ODS is very easy to build as long as the data
warehouse has already been constructed.
• (Class V.)
• Highly integrated and aggregated data source for reporting
14
15. Alternative ODS Typology (Execution MiH)
• TYPE I (Data Cache)
• Online data store, used for transaction execution and system interface purpose
• These data stores have source system data replicated in the central data store. The source system exchange data with other systems through this data store,
instead of exchanging point to point interface files
• Other applications of this kind of data store architectures is to provide a common database for source systems to directly refer to. For example, you can
have the source systems updating and referring to the sanitized master tables existing in the ODS (we will refer to this in our Master Data Management
Section, which is still under authoring). There are situations where the source system is directly referring to or updating a table in an ODS.
• TYPE II (CDI/PDI)
• Online data stores, used for Servicing and Relationship
• This is a similar application as mentioned above, however the focus is limited to getting single customer, process and master data view for the sake of
stakeholder servicing (like customer, employee and Vendor servicing). The examples are customer relationship single view, or customer touch point single
view. You can retrieve this single view during your in-bound or out-bound interactions with the customers. This online operational access, gives you the
benefit of risk management, cross-sell, up-sell etc.
• TYPE III (Operational Reporting)
• For reporting
• Technically it is not an ODS, but people use the term for this application as well. You can have a reporting data to churn out your operational reporting. It
has replica of select data from the source systems. It generally has low-intervention transformation.
15
16. Microsoft: DWH vs. ODS
• The purpose of the Data Warehouse (DWH) in the overall Business Intelligence Architecture is to integrate corporate data from different
heterogeneous data sources in order to facilitate historical and trend analysis reporting. It acts as a central repository and contains the
"single version of truth" for the organization that has been carefully constructed from data stored in disparate internal and external
operational databasessystems.
• The purpose of the Operation Data Store (ODS) is to integrate corporate data from different heterogeneous data sources in order to
facilitate real time or near real time operational reporting. Often data in the ODS will be in structured similar to the source systems,
although during integration it can involve data cleansing, de-duplication and can apply business rules to ensure data integrity. An ODS is
mainly intended to integrate data quite frequently at the lowest granular level for operational reporting in a close to real time data
integration scenario. Normally, an ODS will not be optimized for historical and trend analysis on huge set of data.
• Let's summarize the differences between an ODS and DW:
• An ODS is meant for operational reporting and supports current or near real-time reporting requirements whereas a
DW is meant for historical and trend analysis reporting on a large volume of data
• An ODS is targeted for low granular queries whereas a DW is used for complex queries against summary-level or on
aggregated data
• An ODS provides information for operational, tactical decisions about current or near real-time data acquisition
whereas a DW delivers feedback for strategic decisions leading to overall system improvements
• In an ODS the frequency of data load could be hourly or daily whereas in an DW the frequency of data loads could be
daily, weekly, monthly or quarterly
16
18. Adastra ODS Principles
Integrated and
consolidated
data
Subject
oriented data
Master data
focus (business
entities)
Changing data
(actual data)
Limited history
data
(transactions)
Low level data
granularity (no
aggregations)
Mix between
OLTP and DWH
„The best from
both worlds“
18
19. ODS Features
• One version of truth (with different processes presentation)
• Single customer view across all systems / businesses
• Customer Data Integration
• Product Data Integration
• Data cleansing and consolidation (MDM platform)
• Integrated data for other systems or applications (data cache)
• Online access (read and write)
• Quick access to actual data (operational reporting)
• One of component for SOA Architecture (not only)
• Efficient common information exchange among businesses or systems
• One platform for all businesses and IT systems (online and offline processes)
• Data sets from many sources
• Support or replacement for legacy systems
19
20. ODS Benefits
Business Benefits
• Real-time consolidated and integrated data for any purpose
• More reliable mission critical processes
• Reduce costs on IT solutions
• Single customer view
• Integrated product data
• Enabling multichannel and efficient campaign management
• Data for credit risk management
• Integrated communication across all channels
• Economical network analysis
• Faster collection processes
• Online fraud detection
• Near-real time operational reporting
• Data monetization
Technical Benefits
• One version of truth (with different process presentation)
• Single customer view across all systems / businesses
• Customer Data Integration (CDI)
• Product Data Integration (PDI)
• Data cleansing and consolidation (MDM platform)
• Integrated data for other systems or applications (data
cache)
• Online access (read and write)
• Quick access to actual data (operational reporting)
• One of central component of SOA Architecture
• Efficient common information exchange among businesses or
systems
• One platform for all businesses and IT systems (online and
offline processes)
• Data sets from many sources
• Support or replacement for legacy systems
20
21. ODS
ADS
(DWH or EDW)
DATA
ONLINE WORLD OFFLINE WORLD
1. Focus on operational processes
2. Online read and write 24/7
3. For other IT systems / prorcesses
4. Limited data set
5. Very limited history
6. Focus on current data
7. Low data granularity
8. Integration with ADS
1. Focus on analytic tasks
2. Offline batch processing
3. For end-users
4. Large data Set
5. Long history
6. Focus on all data
7. Many levelds of data granularity
8. Data marts and data aggregates
21
22. ODS Data Refresh Time Period
Real-time
Near-real
time
Many times
per day
Daily
Monthly Ad-hoc Hybrid
23. ODS Data Transformations
• Batch Processing
• ETLs
• Extract, Transform, Load
• Transform data from source table / tables to one target table
• Transformation ETLs, Synchronization ETLs
• Advanced data processing
• Batch data cleansing and unification
• Advanced calculations
• Online Processing
• APIs
• Read APIs
• Write APIs
• Change Data Capture (CDC)
24. Database provider’s
competency
Consumer’s competencyConsumer’s competency
System independency – Reason for API
24
Database
External Data Consumer
Database
External Data Consumer
Interface layer
Concentrated transformation logics
Enterprise level impact analysis required
External workload consumers
25. Service layer agreement (SLA)
• A definition of services
• Availability (99.99%)
• Open hours (24x7, 10x5)
• Performance
• Problem management
• Security
• Disaster recovery
• Termination of agreement
25
Availability % Downtime per year
98% 7.30 days
99% 3.65 days
99.5% 1.83 days
99.9% 8.76 hours
99.99% 52.6 min
99.999% 5.26 min
99.9999% 31.5 s
30. Datové domény
Produkty 3.
stran
Oddlužnění ETM Nabídky Žádosti Souhlasy
Klasifikace
Ekonomické
skupiny
Kampaně Produkty Segmentace
Behaviorální
data
Externí data
Identifikace
klienta
Podpisová
oprávnění
Kontaktní
údaje
Unifikace Ostatní
30
33. Instance Party
Unified PartyLocated Address
Instance Address Instance Phone
Unified Phone
Account
Product Instance
Product Instance Party Role
Application
Account Balance Fact
Account Role
Product Instance Relationship
Loan Instance
Facility Instance
Business Product Type
ODS Core Tables
(ABDM)
Card Instance
... Instance
Instance Email
Instance ID Card
Application Detail
34. Benefits
Business Benefits
• Real-time consolidated and integrated data for any purpose
• More reliable mission critical processes
• Reduce costs on IT solutions
• Single customer view
• Integrated product data
• Enabling multichannel and efficient campaign management
• Data for credit risk management
• Integrated communication across all channels
• Economical network analysis
• Faster collection processes
• Online fraud detection
• Near-real time operational reporting
• Data monetization
Technical Benefits
• One BI version of truth (with different process
presentation)
• Single customer view across all systems / businesses
• Customer Data Integration (CDI)
• Product Data Integration (PDI)
• Data cleansing and consolidation (MDM platform)
• Integrated data for other systems or applications (data
cache)
• Online access (read and write)
• Quick access to actual data (operational reporting)
• One of central component of SOA Architecture
• Efficient common information exchange among businesses or
systems
• One platform for all businesses and IT systems (online and
offline processes)
• Data sets from many sources
• Support or replacement for legacy systems 34
38. Datové domény
Běžné účty /
Deposita
Úvěry Karty Pojištění Služby
Produkty třetích
stran (Energie,
Telco,..)
Transakce Rezervace/Blokace Klienti Žádosti o produkty
Žádosti o procesní
zpracování
Kontakty Zajištění Eventy
38
39. Přínosy
Konsolidace dat z
mnoha BE
Odlehčení
middleware
Zrychlení odezvy
front end
aplikacím.
Zajištění vysoké
dostupnosti služeb.
Online interface
pro DWH.
Detekce událostí
Datový rozcestník
do BE
Kratší čas a méně
úsilí pro dodávku
požadavků.
Bez složité procesní
integrace
Propis dat je mimo
účetní uzávěrky
opravdu rychlý.
41. 41
WEB Services
WEB Services
CRM
Vrstva L0
eShop
Vrstva L1
Navision
Rozhraní pro návazné systémy
CRM eShop
Metadata
Adresář
MS SQL Server 2012
OLE DB
OLE DB
Navision
ODS
Navision
Diskový svazek
pro NAV
Snapshot
svazku
SQL Server 2012 SQL Server 2012
ODS
Agent diskového pole
Diskové pole
Připojení svazku k
serveru
Metadata
Konec ETL
SQL Server Agent
Odpojení
svazku
Start ETL
42. Přínosy
Uvolnění zátěže
primárního systému
Integrace e-shopů
Podpora pro věrnostní
program
Snadnější integrace
nových systémů
Zpřehlednění datových
toků
Jedna verze pravdy pro
návazné systémy i
zákazníky na webu
Přímý přístup k datům
prostřednictvím
databázových
snapshotů
Webové služby
•metody s online přístupem
•metody pro synchronizaci
dat
42