Traditional BI systems have limitations in handling big data as they are not designed for unstructured data and have data latency issues. A business data lake provides a new approach by storing all raw structured and unstructured data in a single environment at low cost. This allows for near real-time analysis on any data from any source to gain insights.
Informatica Becomes Part of the Business Data Lake EcosystemCapgemini
Informatica is now part of the Business Data Lake ecosystem developed by Capgemini and Pivotal. Customers worldwide will now be able to leverage Informatica’s data integration software in addition to Pivotal’s advanced big data, analytics and application software, and Capgemini’s industry and implementation expertise. Informatica will deliver certified technologies for Data Integration, Data Quality and Master Data Management (MDM) to help enterprises distill raw data into actionable insights.
http://www.capgemini.com/resources/the-business-data-lake-delivering-the-speed-and-accuracy-to-solve-your-big-data-problems
The Business Data Lake is a new approach to information management, analytics and reporting that better matches the culture of business and better enables organizations to truly leverage the value of their information.
EMC World 2014 Breakout: Move to the Business Data Lake – Not as Hard as It S...Capgemini
Rip and replace isn't a good approach to IT change. When looking at Hadoop, MPP, in-memory and predictive analytics the challenge is making them co-exist with current solutions.
Learn how Capgemini’s Pivotal CoE utilizes Cloud Foundry and PivotalOne to help businesses adopt new technologies without losing the value of current investments.
Presented by Michael Wood of Pivotal and Steve Jones, Global Director, Strategy, Big Data and Analytics, Capgemini, at EMC World 2014.
Modern Integrated Data Environment - Whitepaper | QuboleVasu S
A whit-paper is about building a modern data platform for data driven organisations with using cloud data warehouse with modern data platform architecture
https://www.qubole.com/resources/white-papers/modern-integrated-data-environment
Oracle OpenWorld London - session for Stream Analysis, time series analytics, streaming ETL, streaming pipelines, big data, kafka, apache spark, complex event processing
The Pivotal Business Data Lake provides a flexible blueprint to meet your business's future information and analytics needs while avoiding the pitfalls of typical EDW implementations. Pivotal’s products will help you overcome challenges like reconciling corporate and local needs, providing real-time access to all types of data, integrating data from multiple sources and in multiple formats, and supporting ad hoc analysis.
Accelerating Fast Data Strategy with Data VirtualizationDenodo
"Information from the past won't support the insights of the future - businesses need real-time data," said Forrester Analyst Noel Yuhanna. In this presentation, he explains the challenges of latent data faced by business users, the need to accelerate fast data strategy using data virtualization, and the implications of such strategy.
This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/a2xNyZ.
Informatica Becomes Part of the Business Data Lake EcosystemCapgemini
Informatica is now part of the Business Data Lake ecosystem developed by Capgemini and Pivotal. Customers worldwide will now be able to leverage Informatica’s data integration software in addition to Pivotal’s advanced big data, analytics and application software, and Capgemini’s industry and implementation expertise. Informatica will deliver certified technologies for Data Integration, Data Quality and Master Data Management (MDM) to help enterprises distill raw data into actionable insights.
http://www.capgemini.com/resources/the-business-data-lake-delivering-the-speed-and-accuracy-to-solve-your-big-data-problems
The Business Data Lake is a new approach to information management, analytics and reporting that better matches the culture of business and better enables organizations to truly leverage the value of their information.
EMC World 2014 Breakout: Move to the Business Data Lake – Not as Hard as It S...Capgemini
Rip and replace isn't a good approach to IT change. When looking at Hadoop, MPP, in-memory and predictive analytics the challenge is making them co-exist with current solutions.
Learn how Capgemini’s Pivotal CoE utilizes Cloud Foundry and PivotalOne to help businesses adopt new technologies without losing the value of current investments.
Presented by Michael Wood of Pivotal and Steve Jones, Global Director, Strategy, Big Data and Analytics, Capgemini, at EMC World 2014.
Modern Integrated Data Environment - Whitepaper | QuboleVasu S
A whit-paper is about building a modern data platform for data driven organisations with using cloud data warehouse with modern data platform architecture
https://www.qubole.com/resources/white-papers/modern-integrated-data-environment
Oracle OpenWorld London - session for Stream Analysis, time series analytics, streaming ETL, streaming pipelines, big data, kafka, apache spark, complex event processing
The Pivotal Business Data Lake provides a flexible blueprint to meet your business's future information and analytics needs while avoiding the pitfalls of typical EDW implementations. Pivotal’s products will help you overcome challenges like reconciling corporate and local needs, providing real-time access to all types of data, integrating data from multiple sources and in multiple formats, and supporting ad hoc analysis.
Accelerating Fast Data Strategy with Data VirtualizationDenodo
"Information from the past won't support the insights of the future - businesses need real-time data," said Forrester Analyst Noel Yuhanna. In this presentation, he explains the challenges of latent data faced by business users, the need to accelerate fast data strategy using data virtualization, and the implications of such strategy.
This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/a2xNyZ.
Data-Ed Online Presents: Data Warehouse StrategiesDATAVERSITY
Integrating data across systems has been a perpetual challenge. Unfortunately, the current technology-focused solutions have not helped IT to improve its dismal project success statistics. Data warehouses, BI implementations, and general analytical efforts achieve the same levels of success as other IT projects – approximately 1/3rd are considered successes when measured against price, schedule, or functionality objectives. The first step is determining the appropriate analysis approach to the data system integration challenge. The second step is understanding the strengths and weaknesses of various approaches. Turns out that proper analysis at this stage makes actual technology selection far more accurate. Only when these are accomplished can proper matching between problem and capabilities be achieved as the third step and true business value be delivered. This webinar will illustrate that good systems development more often depends on at least three data management disciplines in order to provide a solid foundation.
Takeaways:
Data system integration challenge analysis
Understanding of a range of data system-integration technologies including
Problem space (BI, Analytics, Big Data), Data (Warehousing, Vault, Cube) and alternative approaches (Virtualization, Linked Data, Portals, Meta-models)
Understanding foundational data warehousing & BI concepts based on the Data Management Body of Knowledge (DMBOK)
How to utilize data warehousing & BI in support of business strategy
Bringing Strategy to Life: Using an Intelligent Data Platform to Become Data ...DLT Solutions
Anil Chakravarthy, Executive Vice President and Chief Product Officer at Informatica, shares how to use an intelligent data platform to become data ready from the 2015 Informatica Government Summit.
Intuit's Data Mesh - Data Mesh Leaning Community meetup 5.13.2021Tristan Baker
Past, present and future of data mesh at Intuit. This deck describes a vision and strategy for improving data worker productivity through a Data Mesh approach to organizing data and holding data producers accountable. Delivered at the inaugural Data Mesh Leaning meetup on 5/13/2021.
Making the Case for Hadoop in a Large Enterprise-British AirwaysDataWorks Summit
Making the Case for Hadoop in a Large Enterprise
British Airways
Alan Spanos
Data Exploitation Manager
British Airways
Jay Aubby
Architect
British Airways
IBM Governed data lake is a value-driven big data platform journey. The journey starts by ingesting wide variety of data, governing it, applying data science and machine learning on it to produce actionable insights.
Claudia Imhoff of the Boulder BI Brain Trust gives the lowdown on integrating real-time data to leverage modern BI practices for your business in this Information Builders Innovation Session presentation.
Slides: Accelerating Queries on Cloud Data LakesDATAVERSITY
Using “zero-copy” hybrid bursting on remote data to solve data lake analytics capacity and performance problems.
Data scientists want answers on demand. But in today’s enterprise architectures, the reality is that most data remains on-prem, despite the promise of cloud-based analytics. Moving all that data to the cloud has typically not been possible for many reasons including cost, latency, and technical difficulty. So, what if there was a technology that would connect these on-prem environments to any major cloud platform, enabling high-powered computing without the need to move massive amounts of data?
Join us for this webinar where Alex Ma of Alluxio, an open-source data orchestration platform, will discuss how a data orchestration approach offers a solution for connecting traditional on-prem data centers and cloud data lakes with other clouds and data centers. With Alluxio’s “zero-copy” burst solution, companies can bridge remote data centers and data lakes with computing frameworks in other locations, enabling them to offload, compute, and leverage the flexibility, scalability, and power of the cloud for their remote data.
Enterprise Search: Addressing the First Problem of Big Data & Analytics - Sta...StampedeCon
This session addresses the first problems of Big Data & Analytics–Identifying, Indexing, Connecting and Gaining Insight of Existing Data to Drive Value. HPE’s Chief Field Technologist will give her perspectives on Enterprise Search as a Fundamental Cornerstone of Building a Data Driven Enterprise.
Data Governance, Compliance and Security in Hadoop with ClouderaCaserta
In our recent Big Data Warehousing Meetup, we discussed Data Governance, Compliance and Security in Hadoop.
As the Big Data paradigm becomes more commonplace, we must apply enterprise-grade governance capabilities for critical data that is highly regulated and adhere to stringent compliance requirements. Caserta and Cloudera shared techniques and tools that enables data governance, compliance and security on Big Data.
For more information, visit www.casertaconcepts.com
Active Governance Across the Delta Lake with AlationDatabricks
Alation provides a single interface to provide users and stewards to provide active and agile data governance across Databricks Delta Lake and Databricks SQL Analytics Service. Understand how Alation can expand adoption in the data lake while providing safe and responsible data consumption.
Analyst Webinar: Best Practices In Enabling Data-Driven Decision MakingDenodo
Watch full webinar here: https://bit.ly/37YkgN4
This presentation looks at the trends that are emerging from companies on their journeys to becoming data-driven enterprises.
These trends are taken from a survey of 500 companies and highlight critical success factors, what companies are doing, their progress so far and their plans going forward. It also looks at the role that data virtualization has within the data driven enterprise.
During the session we'll address:
- What is a data-driven enterprise?
- What are the critical success factors?
- What are companies doing to create a data-driven enterprise and why?
- What progress are they making?
- What are the plans on people, process and technologies?
- Why is data virtualization central to provisioning and accessing data in a data-driven enterprise?
- How should you get started?
Logical Data Warehouse and Data Lakes can play a role in many different type of projects and, in this presentation, we will look at some of the most common patterns and use cases. Learn about analytical and big data patterns as well as performance considerations. Example implementations will be discussed for each pattern.
- Architectural patterns for logical data warehouse and data lakes.
- Performance considerations.
- Customer use cases and demo.
This presentation is part of the Denodo Educational Seminar, and you can watch the video here goo.gl/vycYmZ.
Caserta Concepts, Datameer and Microsoft shared their combined knowledge and a use case on big data, the cloud and deep analytics. Attendes learned how a global leader in the test, measurement and control systems market reduced their big data implementations from 18 months to just a few.
Speakers shared how to provide a business user-friendly, self-service environment for data discovery and analytics, and focus on how to extend and optimize Hadoop based analytics, highlighting the advantages and practical applications of deploying on the cloud for enhanced performance, scalability and lower TCO.
Agenda included:
- Pizza and Networking
- Joe Caserta, President, Caserta Concepts - Why are we here?
- Nikhil Kumar, Sr. Solutions Engineer, Datameer - Solution use cases and technical demonstration
- Stefan Groschupf, CEO & Chairman, Datameer - The evolving Hadoop-based analytics trends and the role of cloud computing
- James Serra, Data Platform Solution Architect, Microsoft, Benefits of the Azure Cloud Service
- Q&A, Networking
For more information on Caserta Concepts, visit our website: http://casertaconcepts.com/
Unlocking data science in the enterprise - with Oracle and ClouderaCloudera, Inc.
Today, leading organizations struggle to make their data scientists productive in their modern data platforms. Data scientists find it difficult to use their existing open source languages (e.g. Python, R) and libraries with Hadoop, especially when the clusters are secured with Kerberos. At the same time, IT doesn't want to give special access to these users, who require very diverse and specific environment configurations to run their experiments. As a result, most data science teams work away from the big data cluster, often on their laptops or in other data silos. The negative business impacts are a lack of insight and agility for the most advanced users, and the security, governance, and cost issues that arise from data silos.
Data Lakehouse, Data Mesh, and Data Fabric (r2)James Serra
So many buzzwords of late: Data Lakehouse, Data Mesh, and Data Fabric. What do all these terms mean and how do they compare to a modern data warehouse? In this session I’ll cover all of them in detail and compare the pros and cons of each. They all may sound great in theory, but I'll dig into the concerns you need to be aware of before taking the plunge. I’ll also include use cases so you can see what approach will work best for your big data needs. And I'll discuss Microsoft version of the data mesh.
The Importance of DataOps in a Multi-Cloud WorldDATAVERSITY
There’s no denying that Cloud has evolved from being an outlying market disruptor to a mainstream method for delivering IT applications and services. In fact, it’s not uncommon to find that Enterprises use the services of more than one cloud at the same time. However, while a multi-cloud strategy offers many benefits, it also increases data management complexity and consequently reduces data availability. This webinar defines the meaning of DataOps and why it’s a crucial component for every multi-cloud approach.
Data-Ed Online Presents: Data Warehouse StrategiesDATAVERSITY
Integrating data across systems has been a perpetual challenge. Unfortunately, the current technology-focused solutions have not helped IT to improve its dismal project success statistics. Data warehouses, BI implementations, and general analytical efforts achieve the same levels of success as other IT projects – approximately 1/3rd are considered successes when measured against price, schedule, or functionality objectives. The first step is determining the appropriate analysis approach to the data system integration challenge. The second step is understanding the strengths and weaknesses of various approaches. Turns out that proper analysis at this stage makes actual technology selection far more accurate. Only when these are accomplished can proper matching between problem and capabilities be achieved as the third step and true business value be delivered. This webinar will illustrate that good systems development more often depends on at least three data management disciplines in order to provide a solid foundation.
Takeaways:
Data system integration challenge analysis
Understanding of a range of data system-integration technologies including
Problem space (BI, Analytics, Big Data), Data (Warehousing, Vault, Cube) and alternative approaches (Virtualization, Linked Data, Portals, Meta-models)
Understanding foundational data warehousing & BI concepts based on the Data Management Body of Knowledge (DMBOK)
How to utilize data warehousing & BI in support of business strategy
Bringing Strategy to Life: Using an Intelligent Data Platform to Become Data ...DLT Solutions
Anil Chakravarthy, Executive Vice President and Chief Product Officer at Informatica, shares how to use an intelligent data platform to become data ready from the 2015 Informatica Government Summit.
Intuit's Data Mesh - Data Mesh Leaning Community meetup 5.13.2021Tristan Baker
Past, present and future of data mesh at Intuit. This deck describes a vision and strategy for improving data worker productivity through a Data Mesh approach to organizing data and holding data producers accountable. Delivered at the inaugural Data Mesh Leaning meetup on 5/13/2021.
Making the Case for Hadoop in a Large Enterprise-British AirwaysDataWorks Summit
Making the Case for Hadoop in a Large Enterprise
British Airways
Alan Spanos
Data Exploitation Manager
British Airways
Jay Aubby
Architect
British Airways
IBM Governed data lake is a value-driven big data platform journey. The journey starts by ingesting wide variety of data, governing it, applying data science and machine learning on it to produce actionable insights.
Claudia Imhoff of the Boulder BI Brain Trust gives the lowdown on integrating real-time data to leverage modern BI practices for your business in this Information Builders Innovation Session presentation.
Slides: Accelerating Queries on Cloud Data LakesDATAVERSITY
Using “zero-copy” hybrid bursting on remote data to solve data lake analytics capacity and performance problems.
Data scientists want answers on demand. But in today’s enterprise architectures, the reality is that most data remains on-prem, despite the promise of cloud-based analytics. Moving all that data to the cloud has typically not been possible for many reasons including cost, latency, and technical difficulty. So, what if there was a technology that would connect these on-prem environments to any major cloud platform, enabling high-powered computing without the need to move massive amounts of data?
Join us for this webinar where Alex Ma of Alluxio, an open-source data orchestration platform, will discuss how a data orchestration approach offers a solution for connecting traditional on-prem data centers and cloud data lakes with other clouds and data centers. With Alluxio’s “zero-copy” burst solution, companies can bridge remote data centers and data lakes with computing frameworks in other locations, enabling them to offload, compute, and leverage the flexibility, scalability, and power of the cloud for their remote data.
Enterprise Search: Addressing the First Problem of Big Data & Analytics - Sta...StampedeCon
This session addresses the first problems of Big Data & Analytics–Identifying, Indexing, Connecting and Gaining Insight of Existing Data to Drive Value. HPE’s Chief Field Technologist will give her perspectives on Enterprise Search as a Fundamental Cornerstone of Building a Data Driven Enterprise.
Data Governance, Compliance and Security in Hadoop with ClouderaCaserta
In our recent Big Data Warehousing Meetup, we discussed Data Governance, Compliance and Security in Hadoop.
As the Big Data paradigm becomes more commonplace, we must apply enterprise-grade governance capabilities for critical data that is highly regulated and adhere to stringent compliance requirements. Caserta and Cloudera shared techniques and tools that enables data governance, compliance and security on Big Data.
For more information, visit www.casertaconcepts.com
Active Governance Across the Delta Lake with AlationDatabricks
Alation provides a single interface to provide users and stewards to provide active and agile data governance across Databricks Delta Lake and Databricks SQL Analytics Service. Understand how Alation can expand adoption in the data lake while providing safe and responsible data consumption.
Analyst Webinar: Best Practices In Enabling Data-Driven Decision MakingDenodo
Watch full webinar here: https://bit.ly/37YkgN4
This presentation looks at the trends that are emerging from companies on their journeys to becoming data-driven enterprises.
These trends are taken from a survey of 500 companies and highlight critical success factors, what companies are doing, their progress so far and their plans going forward. It also looks at the role that data virtualization has within the data driven enterprise.
During the session we'll address:
- What is a data-driven enterprise?
- What are the critical success factors?
- What are companies doing to create a data-driven enterprise and why?
- What progress are they making?
- What are the plans on people, process and technologies?
- Why is data virtualization central to provisioning and accessing data in a data-driven enterprise?
- How should you get started?
Logical Data Warehouse and Data Lakes can play a role in many different type of projects and, in this presentation, we will look at some of the most common patterns and use cases. Learn about analytical and big data patterns as well as performance considerations. Example implementations will be discussed for each pattern.
- Architectural patterns for logical data warehouse and data lakes.
- Performance considerations.
- Customer use cases and demo.
This presentation is part of the Denodo Educational Seminar, and you can watch the video here goo.gl/vycYmZ.
Caserta Concepts, Datameer and Microsoft shared their combined knowledge and a use case on big data, the cloud and deep analytics. Attendes learned how a global leader in the test, measurement and control systems market reduced their big data implementations from 18 months to just a few.
Speakers shared how to provide a business user-friendly, self-service environment for data discovery and analytics, and focus on how to extend and optimize Hadoop based analytics, highlighting the advantages and practical applications of deploying on the cloud for enhanced performance, scalability and lower TCO.
Agenda included:
- Pizza and Networking
- Joe Caserta, President, Caserta Concepts - Why are we here?
- Nikhil Kumar, Sr. Solutions Engineer, Datameer - Solution use cases and technical demonstration
- Stefan Groschupf, CEO & Chairman, Datameer - The evolving Hadoop-based analytics trends and the role of cloud computing
- James Serra, Data Platform Solution Architect, Microsoft, Benefits of the Azure Cloud Service
- Q&A, Networking
For more information on Caserta Concepts, visit our website: http://casertaconcepts.com/
Unlocking data science in the enterprise - with Oracle and ClouderaCloudera, Inc.
Today, leading organizations struggle to make their data scientists productive in their modern data platforms. Data scientists find it difficult to use their existing open source languages (e.g. Python, R) and libraries with Hadoop, especially when the clusters are secured with Kerberos. At the same time, IT doesn't want to give special access to these users, who require very diverse and specific environment configurations to run their experiments. As a result, most data science teams work away from the big data cluster, often on their laptops or in other data silos. The negative business impacts are a lack of insight and agility for the most advanced users, and the security, governance, and cost issues that arise from data silos.
Data Lakehouse, Data Mesh, and Data Fabric (r2)James Serra
So many buzzwords of late: Data Lakehouse, Data Mesh, and Data Fabric. What do all these terms mean and how do they compare to a modern data warehouse? In this session I’ll cover all of them in detail and compare the pros and cons of each. They all may sound great in theory, but I'll dig into the concerns you need to be aware of before taking the plunge. I’ll also include use cases so you can see what approach will work best for your big data needs. And I'll discuss Microsoft version of the data mesh.
The Importance of DataOps in a Multi-Cloud WorldDATAVERSITY
There’s no denying that Cloud has evolved from being an outlying market disruptor to a mainstream method for delivering IT applications and services. In fact, it’s not uncommon to find that Enterprises use the services of more than one cloud at the same time. However, while a multi-cloud strategy offers many benefits, it also increases data management complexity and consequently reduces data availability. This webinar defines the meaning of DataOps and why it’s a crucial component for every multi-cloud approach.
Data lakes are central repositories that store large volumes of structured, unstructured, and semi-structured data. They are ideal for machine learning use cases and support SQL-based access and programmatic distributed data processing frameworks. Data lakes can store data in the same format as its source systems or transform it before storing it. They support native streaming and are best suited for storing raw data without an intended use case. Data quality and governance practices are crucial to avoid a data swamp. Data lakes enable end-users to leverage insights for improved business performance and enable advanced analytics.
For Impetus’ White Papers archive, visit- http://www.impetus.com/whitepaper
In this paper, Impetus focuses at why organizations need to design an Enterprise Data Warehouse (EDW) to support the business analytics derived from the Big Data.
Which Change Data Capture Strategy is Right for You?Precisely
Change Data Capture or CDC is the practice of moving the changes made in an important transactional system to other systems, so that data is kept current and consistent across the enterprise. CDC keeps reporting and analytic systems working on the latest, most accurate data.
Many different CDC strategies exist. Each strategy has advantages and disadvantages. Some put an undue burden on the source database. They can cause queries or applications to become slow or even fail. Some bog down network bandwidth, or have big delays between change and replication.
Each business process has different requirements, as well. For some business needs, a replication delay of more than a second is too long. For others, a delay of less than 24 hours is excellent.
Which CDC strategy will match your business needs? How do you choose?
View this webcast on-demand to learn:
• Advantages and disadvantages of different CDC methods
• The replication latency your project requires
• How to keep data current in Big Data technologies like Hadoop
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization Denodo
Watch here: https://bit.ly/2NGQD7R
In an era increasingly dominated by advancements in cloud computing, AI and advanced analytics it may come as a shock that many organizations still rely on data architectures built before the turn of the century. But that scenario is rapidly changing with the increasing adoption of real-time data virtualization - a paradigm shift in the approach that organizations take towards accessing, integrating, and provisioning data required to meet business goals.
As data analytics and data-driven intelligence takes centre stage in today’s digital economy, logical data integration across the widest variety of data sources, with proper security and governance structure in place has become mission-critical.
Attend this session to learn:
- Learn how you can meet cloud and data science challenges with data virtualization.
- Why data virtualization is increasingly finding enterprise-wide adoption
- Discover how customers are reducing costs and improving ROI with data virtualization
COVID-19 heightened chronic challenges within the global healthcare industry. It became a catalyst amid fierce competition and tight regulations for health providers and payers to focus on digital health, cybersecurity, patient data transparency, and a variety of customer-centric and operational enhancements. As a result, we found the 2022 trendline pointing to improvements in access and quality of care.
Healthcare challenges such as optimizing the cost of care while simultaneously enabling personalized interventions and consumer-friendly shoppable services are long-standing − but, historically, the industry has been slow to react.
Read our Top Trends 2022 report to examine the lingering ramifications of the pandemic, responses from medical and insurance organizations, and the worldwide impact of ever-changing regulatory standards and mandates.
A combination of factors − the pandemic, catastrophic weather events, evolving policyholder expectations, and insurers’ drive for operational efficiency and future relevance − are sparking P&C industry changes.
In a post-COVID, new-normal environment, the most strategic insurers are building resilient, crisis-proof enterprises poised to take advantage of emerging and future business opportunities. They are leveraging advanced data analytics and novel technologies to assure agility and achieve positive revenue and customer satisfaction outcomes. Competitive advantage will hinge on accelerated digitalization and faster go-to-market. Therefore, win-win partnerships and embedded services with InsurTechs and other ecosystem players are critical.
Read Capgemini’s Top P&C Insurance Trends 2022 for a glimpse at the tactical and strategic initiatives carriers are undertaking to boost customer-centricity, product agility, intelligent processes, and an open ecosystem to ensure profitable growth and future-readiness.
This analysis provides an overview of the top trends in the commercial banking sector as they shift to technology high gear to boost client efficiency and battle a volatile, uncertain, competitive, and evolving landscape.
First, it was retail banking. Now, advanced technology is shifting to – and disrupting − the commercial banking space. Many commercial banks, known for paperwork, red tape, and branch dependency, were unprepared to support clients during their post-COVID-19 ramp-up. But now, the digital pivot to new mindsets, partnerships, and processes is in overdrive.
As commercial banks grapple with competition from FinTechs, BigTechs, and alternative lenders, their inability
to fulfill SME demands and pandemic after-shocks necessitates transformative process changes and a move
to experiential, sustainable, and inclusive banking models. We expect banks to strive to meet the demands
of corporate clients and SMEs by digitally transforming critical workflows and improving client experience.
Additionally, incremental process improvements in the middle and back-office that leverage intelligent
automation will keep the competition at bay because engaged clients are loyal.
Adopting newer methods to mine data and moving to as-a-Service models will prepare commercial banks
to flexibly respond to newcomers and find ways to co-exist through effective collaboration. The time has come for commercial banks to put transformation on the fast track as lending losses in wallet and market share could spill over to other functions!
How incumbents react and respond to 2022 trends could determine their relevancy and resiliency in the years ahead.
The Covid-19 pandemic necessitated the payments industry undergo a facelift, sparked by novel approaches from new-age players, fostered by industry consolidation, and customers’ demand for end-to-end experience. Crossing the threshold, the industry is entering a new era – Payments 4.X, where payments are embedded and invisible, and an enabling function to provide frictionless customer experience. As customers make a permanent shift to next-gen payment methods, Digital IDs are critical for a seamless payment experience. The B2B payments segment is witnessing rapid digitization. BigTechs, PayTechs, and industry newcomers are ready to jump in with newfangled solutions to help underserved small to medium-sized businesses (SMBs).
As incumbents struggle with profits, new-age firms are forging ahead to take the lead in the Payments 4.X era by riding the success of non-card products and services. The new era demands collaboration, platformification, and firms can unleash full market potential only by embracing API-based business models and open ecosystems. Data prowess and enhanced payment processing capabilities are inevitable to thrive ahead. The clock is ticking for banks and traditional payments firms because the competitive advantage is not guaranteed forever. As industry players seek economies of scale, consolidations loom, and non-banks explore new territories to threaten incumbents’ market share. While all these 2022 trends are at play, central bank digital currency (CBDC) is emerging globally and might open a new chapter in the current payments landscape.
As we slowly move out of the pandemic, financial services firms have learned the criticality of virtual engagement to business resilience. Wealth management firms will need capabilities to cater to new-age clients and deliver new-age services. This report aims to understand and analyze the top trends in the Wealth Management industry this year and beyond.
A year ago, our Top Trends in Wealth Management report emphasized how the pandemic sparked disruption and digital transformation and changing investor attitudes around Environmental, Social, and Corporate Governance (ESG) products. As we begin 2022, many of those trends continue to hold as COVID-19’s wide-reaching effects continue to influence the wealth management industry.
As wealth management (WM) firms supercharge their digital transformation journeys, investments in cybersecurity and human-centered design are becoming critical to building superior digital client experience (CX). Another holdover trend − sustainable investing – is gaining mainstream attention and generating increasingly sophisticated client demands. Data and analytics capabilities will become ever more essential for ESG scoring and personalized customer engagement. As large financial services firms refocus on their wealth management business while new digital players make industry strides, competition is becoming historically intense. Not surprisingly, client experience is the new battleground.
This analysis provides an overview of the top trends in the retail banking sector driven by the competition, digital transformation, and innovation led by retail banks exploring novel ways to create and retain value in evolving landscape.
COVID-19 caught banks off guard and shook legacy mindsets to the core. With 20/20 (2020) hindsight, firms are more aware, digitally resilient, and financially stable as they head into 2022. The trials of the past 18 months forced firms to shore up existing business and consider new models and revenue streams.
Customer-centricity remains at the top of most FS agendas and is a 2022 focal point. Banks will focus on achieving operational excellence as diligently as delivering superior CX. In 2022 and beyond, it will be paramount for FIs to explore and invest in new technologies to remain relevant and resilient.
Banking 4.X will arrive in full force in 2022 with platform-supported firms monetizing diverse ecosystem capabilities and aggressively harvesting data to create experiential customer journeys through intelligent and personalized engagements. The new era will compel future-focused banks to finally abandon legacy infrastructure and collaborate with third-party specialists to solidify their best-fit, long-term roles. Increasingly, open platforms will make banks invisible as banking becomes embedded into customer lifestyles. At the same time, banks will shed asset-heavy models and shift to the cloud for greater agility, speed to market, and faster innovation. The shift will act as a precursor to adopting new technologies on the horizon – 5G and Decentralized Finance.
The recent past was filled will extraordinary lessons for financial institutions. Now is the time to act on those learnings and move forward profitably.
While COVID-19 has sparked the demand for life insurance, it has also exposed the operating model vulnerabilities in distribution, servicing, and customer retention. In a post-COVID, new-normal environment, insurers need to enhance their capabilities around advanced data management and focus on seamless and secure data sharing to provide superior CX and hyper-personalized offerings. Accelerated digitalization and faster go-to-market are vital to remaining competitive, and win-win partnerships with ecosystems are critical in the journey.
Read our Top Life Insurance Trends 2022 to explore the tactical and strategic initiatives carriers undertake to acquire competencies around customer centricity, product agility, intelligent processes, and an open ecosystem to ensure profitable growth and future readiness.
Property & Casualty Insurance Top Trends 2021Capgemini
The Property & Casualty insurance landscape is evolving quickly with the changing risk landscape, entry of new players, and changing customer expectations. The ripple effects of COVID-19 on the P&C insurance industry and natural disasters such as forest fires have adversely impacted insurance firm books.
In this scenario, to ensure growth and future-readiness, the most strategic insurers strive to be ‘Inventive Insurers’ – assuming a customer-centric approach, deploying intelligent processes, practicing business resilience and go-to-market agility, and embracing an open ecosystem.
Read our Property & Casualty Insurance Top Trends 2021 report to explore the strategies insurers are adapting to remain competitive amidst the evolving business landscape and how they can explore new ways to enhance their profitability.
A combination of factors such as demographic changes, evolving consumer preferences, and desire to become operationally efficient were already spurring changes in the life insurance industry. Enter 2020 – the COVID-19 pandemic is having a significant impact on the industry.
At the peak of disruption, the focus was on ensuring business continuity, but new initiatives are cropping up to tackle the challenges as the industry is adapting to the new normal.
Furthermore, COVID-19 has acted as a catalyst, pushing life insurers to prioritize their efforts on improving customer centricity, developing go-to-market agility, making processes intelligent, building business resilience, and embracing the open ecosystem.
Read our Life Insurance Top Trends 2021 report to explore the strategies insurers are adopting to manage the changing market dynamics.
The uncertainty of 2020 is setting the global tone for the immediate future in the financial services industry. So it is no surprise banks are laser-focused on business resilience, emphasizing both financial and operational risks. The need to adapt quickly to new normal conditions through virtual customer engagement is clear.
Customer centricity continues to drive commercial banks’ solution designs. And, the pandemic compelled products that deliver immediate client value ‒ quick digital onboarding, seamless lending, and support for small and medium-sized enterprises (SMEs). The onus is now on banks to go to market more quickly, which requires the implementation of intelligent processes and integrating corporates’ enterprise resource planning (ERP) systems with banking workflows.
To achieve go-to-market agility, banks across the globe are investing in and collaborating with FinTechs. Many of these partnerships are focused on boosting digital lending and providing seamless support to anxious small-business clients in need of assurance.
With newfound impetus for FinTech collaboration, commercial banks have picked up their step on the path toward OpenX. COVID-19 made it evident that survival during turbulence is manageable through collaboration with ecosystem players.
Read our Top Trends in Commercial Banking 2021 report to explore the strategies banks are adapting to transform their businesses from a product-led, siloed model to an experiential and agile plan.
When we published the Top Trends in Wealth Management 2020, little did we foresee the pandemic that would sweep through the world and disrupt life as we knew it. Yet, when we reviewed last year’s trends, we found that many still hold and some have taken on even greater relevance. One such trend is sustainable investing, which had begun to gain prominence as investors became more aware of ESG considerations, and firms rolled out more sustainable investing offerings. Another trend that has accelerated in the post-COVID world is the importance of investing in omnichannel capabilities and technologies such as artificial intelligence (AI) to enhance personalization and advisor effectiveness. The pandemic has driven wealth management firms to accelerate their digital transformation journey, with some immediate focus areas being interactive client communications and digital advisor tools.
There is no denying that time is of the essence. Yes, budgets are tight, but the Open X ecosystem offers wealth management firms opportunities to reimagine their operating models and deliver excellent customer experience cost-effectively.
Top trends in Payments: 2020 highlighted the payments industry’s flux driven by new trends in technology adoption, innovative solutions, and changing consumer behavior. The pandemic has tested the digital mastery of players, who are already grappling with transition. Non-cash transactions are on a robust growth path, accelerated by increased adoption during COVID-19. Regulators are working to instill trust and address non-cash payments risk amid unparalleled growth as players collaborate to quell uncertainty. Regional initiatives, such as the P27 (Nordics real-time payments system) and the EPI (European Payments Initiative), are gaining traction in response to country-level fragmentation and competition.
Investment in emerging technologies is looked upon as an elixir to mitigate fraud, data-driven offerings are being considered for providing value-added propositions, and distributed ledger technology is in focus for digital currency solutions, efficiency enhancement, and cost gains. New players, such as retailers/merchants, are integrating payments into their value chains while technology giants are upscaling their financial services game by weaving offerings around payments as a center stage. Constrained by budgets, firms consider business models such as Platform-as-a-Service (PaaS) to provide cost-effective and superior customer experience.
A combination of factors, including demographic changes, evolving consumer preferences, and regulatory and compliance mandates, were already spurring change in the health insurance industry. Enter 2020 and the COVID-19 pandemic, which is having sweeping implications for the industry.
At the peak of disruption, the focus was on ensuring business continuity, but new initiatives are cropping up to tackle the challenges as the industry adapts to the new normal.
Furthermore, some changes are here to stay, and it will be prudent for the industry players to be resilient to the market shifts by being agile, improving member centricity, making processes intelligent, and embracing the open ecosystem.
Read our Health Insurance Top Trends 2021 report to explore the strategies insurers are adopting to manage the external pressures.
The banking industry’s resilience is being tested as banks navigate through a remarkable 2020 filled with uncertainties. The impact of COVID-19 has been about setting the tone for future operational models. Retail banks have shifted focus towards integrated risk management with a more holistic view of operational risks. Adapting to the new normal, banks have prioritized cost transformation while engaging customers virtually. Incumbents sought to be more responsible within fast-changing environmental conditions and ESG remained a critical focus.
To provide more experiential services, banks are leveraging techniques such as segment-of-one to hyper-personalize offerings while aiming to humanize digital channels for increased engagement. Banks are also revamping middle and back offices, going beyond the front end leveraging intelligent processes. Open X is enabling banks to play on their strengths and use the expertise of ecosystem players. Going forward, banks are poised to become an enhanced one-stop shop by providing consumers value-adding FS and non-FS experiences.
To acquire customers in cost-effective manner, retail banks are tapping value-based propositions ‒ such as POS financing and mortgage refinancing. Further, Banking-as-Service provides incumbents a way to provide their high-value offerings to other players. In preparation for the future, banks will be looking to improve their go-to-market agility by leveraging the benefits of cloud. This analysis outlines the top 10 trends in retail banking for 2021.
Explore how Capgemini’s Connected autonomous planning fine-tunes Consumer Products Company’s operations for manufacturing, transport, procurement, and virtually every other aspect of the supply-value network in a touchless, autonomous way.
Financial services is undergoing a paradigm shift that is forcing incumbent retail banks to rethink growth strategies as they struggle to remain relevant. Growing competition from BigTechs, FinTech firms, and challenger banks has added to the complexity created by increasingly stringent regulatory and compliance requirements. Customers now expect a seamless customer journey and personalized offerings because they have become accustomed to top-notch individualized service from GAFA giants Google, Apple, Facebook, and Amazon. The changing ecosystem offers established banks new, unexplored opportunities and encourages a transition beyond traditional products to meet the exacting requirements of today’s customers. Bank collaboration with FinTech and RegTech partners is becoming commonplace. Incumbents are exploring point-of-sale financing and unsecured consumer lending, while they also boost their digital channel competencies to reach a broader customer base. Banks are beginning to accept open APIs and are working with third-party specialists to create an open shared marketplace. Technological advancements such as AI are fueling efforts to evolve customer onboarding and touchpoint processes. Increasingly, banks are turning to design thinking methodology to understand the customer journey, extract deep insights, and develop a more refined user experience across the customer lifecycle.
Our analysis of the top retail banking trends for 2020 offers a glimpse into the fast-changing banking ecosystem and explores the tools and solutions being used to face new-age challenges.
Aspects of the life insurance industry have remained constant for years – and so have premiums. Traditional savings products have taken a huge hit in terms of attractiveness because low interest-rates prevail. Meanwhile, the risk landscape is shifting, and insurers need to align better with the emerging business environment, manage changing customer preferences, and improve operational efficiencies. Within today’s scenario, industry players are undertaking tactical and strategic shifts in attempts to manage unpredictable market dynamics. Insurers must develop alternative products to breathe new life into policies and leverage emerging technologies (artificial intelligence (AI), analytics, and blockchain) to improve efficiency, agility, flexibility, and customer-centricity.
Read Top Trends in Life Insurance: 2020 for a look at the innovative steps future-focused insurers are considering to meet industry challenges and opportunities.
The health insurance industry is evolving and undergoing significant changes. As the risk landscape shifts, insurers are working to improve operational efficiencies, meet evolving customer preferences, and align better with the changing business environment. Accordingly, payers must adapt and align business models and offerings. An incisive tactical approach is required to accommodate members’ needs and related emerging risks — medical, health, and environmental. Advanced technologies such as artificial intelligence, analytics, automation, and connected devices are enabling insurers to manage these changes proactively, partner with members, and help to prevent risks, all the while continuing to fulfill payer responsibilities.
Read Top Trends in Health Insurance: 2020 to learn which strategies insurers are adopting to navigate and align with today’s challenges.
Similar to other financial services domains, payments is evolving into an open ecosystem. The EU’s Payment Services Directive (PSD2) pioneered open banking by encouraging banks and established payments players to securely open the systems to foster competition, innovation, and more customer choices. In tandem with non-cash transaction growth, regulations are driving banks and payments firms to expand their array of payment methods and channels. Governments are encouraging financial inclusion by also promoting the adoption of non-cash payments. Increasingly, merchants and corporates seek to offer alternative payment systems because of widespread popularity among consumers. Alternative payments also enable merchants to provide real-time and cross-border payments to boost business efficiency.
Banks, payment firms, card firms, BigTechs, FinTechs, and other players are continuously developing new technology to cash in on market changes. However, data breaches and fraud continue to hinder innovation as firms devote countless resources each year to address security issues. Many governments are also designing new regulations to reduce ecosystem threats. All these measures are expected to make the current ecosystem much more secure and simple for players as well as customers.
Top Trends in Payments: 2020 explores and analyzes payments ecosystem initiatives and solutions for this year and beyond
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Enhancing Performance with Globus and the Science DMZGlobus
ESnet has led the way in helping national facilities—and many other institutions in the research community—configure Science DMZs and troubleshoot network issues to maximize data transfer performance. In this talk we will present a summary of approaches and tips for getting the most out of your network infrastructure using Globus Connect Server.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
2. The need for new thinking around data
storage and analysis
Traditional Business Intelligence (BI) systems
provide various levels and kinds of analyses on
structured data but they are not designed to handle
unstructured data. For these systems Big Data brings
big problems because the data that flows in may be
either structured or unstructured. That makes them
hugely limited when it comes to delivering Big Data
benefits. The way forward is a complete rethink of the
way we use BI - in terms of how the data is ingested,
stored and analyzed.
2 Traditional BI vs. Business Data Lake – A comparison
3. Pivotal the way we see it
Further problems come with the need for near real-time analysis. This requires the
ability to handle and process high velocity data in near-real time - a major challenge
for the traditional BI implementation methods, which have data latency built into their
architecture.
Solutions have been developed to circumvent these issues and bring in as much data
as feasible in near real-time, but these create their own problems - not least the issue
of high storage volumes and costs.
The emergence of Big Data calls for a radically new approach to data management.
Organizations now need near real-time analysis on structured and unstructured data.
Traditional BI approaches that call for building EDWs and data marts are unable to
keep up.
The answer is the Business Data Lake (BDL).
A Business Data Lake is a data repository that can store and handle massive
amounts of structured, semi-structured and unstructured data in its raw form in
low cost commodity storage as it arrives. It provides the ability to perform Line of
Business-specific business analyses yet present a global enterprise view of the
business. Metadata information is maintained for traceability, history and future data
refinement needs.
The Business Data Lake, particularly when combined with Big Data, enables
business users to perform near real-time analysis on practically any data from any
source. It does this by:
• Storing all data in a single environment (a cluster of data stores),
• Setting the stage to perform analysis (standard or self-service) on data whose
structures and relationships are either already known or yet to be determined
• Providing analytical outputs for specific business needs across business functions
• Providing the capability to utilize the data for business benefits in near-real-time,
with the ability to showcase data agility and enable agile BI.
3
4. Traditional approaches and their pitfalls
Most traditional DW and BI implementations follow either a Top-Down or a
Bottom-Up approach to set up the EDW and Data Marts.
Top-Down Approach
The traditional Top-Down approach suggests bringing in the data from all the
data sources, storing it in the EDW in an atomic format in a relational model and
then building data marts with facts and dimensions on top of the EDW for analysis
and reporting.
Advantages
• Retains the authenticity and the sanctity of the data by keeping it close to the
source form
• Provides a single view of the enterprise as all the data is present in the EDW.
Disadvantages
A top-down approach is excellent as a solution to the “single source of the truth”
problem, but it can fail in practice due to the long implementation cycle and a
relational structure that is not friendly for business analysis on the fly.
• A two-step process of loading and maintaining the EDW and the data marts
• Making data available from across the enterprise is difficult
• Time consuming as implementation is slow and the first output is usually several
months away - businesses cannot wait that long to see the results.
• The business requirements may change by the time the implementation
is complete.
4 Traditional BI vs. Business Data Lake – A comparison
5. Pivotal the way we see it
Bottom-Up Approach
This approach suggests bringing in the data from all the data sources, transforming
and restructuring the data and loading them into the data marts in a dimensional
model. The data marts will be subject-area-specific containing conformed
dimensions across subject areas. The integration of the subject area specific data
marts will lead to the EDW.
Advantages
• Since this approach starts small and grows big, it is faster to implement than the
top-down approach. This gives more comfort to the customer as the users are
able to view the results more quickly.
• There is a one-step ETL effort of transforming/restructuring the data into the data
marts, which may be high, but it brings down the 2-step process of data load of
the top down approach.
Disadvantages
The bottom-up approach, while very flexible for business analysis, struggles to
maintain a “single source of truth” because data redundancy is possible across data
marts. Eventually, the model just becomes an integration of fragmented data marts.
• The process of data restructuring (or transformation) while loading into the data
marts involves complex ETL transformations.
• The process begins with small subject area specific data marts, which means
gaining an enterprise level view takes longer. It does not, therefore, immediately
address enterprise requirements.
• This approach is typically a collection of fragmented islands of information and may
not really translate into an EDW.
The choice between implementing a top-down or bottom-up approach is usually
based purely on the business need. In fact most organizations adopt a compromised,
hybrid model which accommodates ideas from both approaches.
5
6. An answer to the problem - the Business
Data Lake
Big Data is all the rage. But what is it, ultimately? A collection of really large volumes
/ massive amounts of data that may be structured or unstructured, generated almost
continuously and that is difficult to process or even handle using traditional data
processing systems.
The Business Data Lake has been designed to solve these challenges around Big
Data.
The data - both structured and unstructured - flows into the lake, and it stored there
until it is needed - when it flows back out again.
Traditional BI systems leveraged the concept of a staging area. Here data from
multiple data sources were Staged. This reduced the dependency on the source
systems to pull the data. Data can be pulled at specific times into the Staging Area.
A Business Data Lake is very similar to the staging area, the key difference being that
the Lake stores both structured and unstructured data. It also provides the ability to
analyze data as required by the business. Any kind of data from any source can be
loaded into the Lake. There is no need to define structures or relationships around the
data that is staged. In addition, storage appliances like Hadoop can bring in all the
enterprise data, without worrying about disk space or judging whether a piece of data
is required or not.
In a traditional DW implementation, the data in the staging area is transient. There is
no persistence of data. It has not been possible to stage such large volumes of data
for extended periods of time due to hardware costs and storage limitations. In the
Lake, the limitation on the storage is eliminated by using commodity hardware that is
much cheaper. Thus, the data that is staged is persistent over time and non-volatile.
Traditional DW approaches require long processes of data ingestion. It can take
months to even review results from the data. The Business Data Lake enables agile
BI, thereby providing the capability to turn around the business outcome of the data
consumed, in near-real time. Data agility enables provisioning of the right outcomes to
the right users at the right time.
Governance can be implemented on the data residing in the Business Data Lake as
required. Master data definitions and management can happen on the data that is
required for analysis, thereby eliminating an over-engineered metadata layer, providing
for data refinement/enrichment only for relevant data.
The Business Data
Lake enables agile
BI, thereby providing
the capability to turn
around the business
outcome of the data
consumed, in near-real
time. Data agility
enables provisioning
of the right outcomes
to the right users at
the right time.
6 Traditional BI vs. Business Data Lake – A comparison
7. Sources
Real-time
ingestion
Micro batch
ingestion
Batch
ingestion
Action tier
Real-time
insights
Interactive
insights
Batch
insights
Ingestion
tier
Real-time
Micro batch
Mega batch
Insights
tier
SQL
NoSQL
SQL
SQL
MapReduce
Query
interfaces
Unified operations tier
System monitoring System management
Unified data management tier
Data mgmt.
services
MDM
RDM
Audit and
policy mgmt.
Workflow management
Processing tier
In-memory
MPP database
Distillation tier
HDFS storage
Unstructured and structured data
Business Data Lake architecture
7
Business Data Lake Architectre
Pivotal the way we see it
8. Benefits of the Business Data Lake
• A Business Data Lake is a storage area for all data sources. Data can be pulled/
pushed directly from the data sources into the Storage Area. All data in raw form
are available in one place.
• Limitations on the data volumes and storage cost are significantly reduced through
the use of commodity hardware.
• Once all data is brought into the Lake, users can pull relevant data for analysis.
They can analyze and derive new insights from the data without knowing its initial
structure. APIs that search the data structures in the Business Data Lake and
provide the metadata information are currently being created. These APIs play a
key role in deriving new insights from ad hoc data analysis.
• As new data sources get added to the environment, they can simply be loaded into
the Business Data Lake and a data refinement/enrichment process created, based
on the business need.
• The main drawback of creating a data model up-front is eliminated. Traditional data
modelling, which is done up-front, fails in a Big Data environment for two reasons:
the nature of the incoming data and the limitation on the analysis that it allows.
The Business Data Lake overcomes these two limitations by providing a loosely
coupled architecture that enables flexibility of analysis. Different business users
with different needs can view the same data from different dimensions.
• Based on repetitive requirements, relevant subject areas that are used frequently
for standard / canned reports can be loaded into the data warehouse in a
dimensional form and the rest of the data can continue to reside inside the
Business Data Lake for analytics on need.
• A data governance framework can be built on top of the Business Data Lake for
relevant enterprise data. This framework can be extended to additional data based
on requirements.
• The Business Data Lake meets local business requirements as well as enterprise-wide
needs from the same data store. The enterprise view of the data can be
considered as another local view.
• Being able to move data across from the sources and turn it around quickly to
derive business outcomes is key to the success of a Business Data Lake, an area
where traditional BI implementations fail to meet business needs.
8 Traditional BI vs. Business Data Lake – A comparison
9. 9
Architecture Comparison — Traditional BI and
Business Data Lake
Traditional Top-Down
Approach
Traditional Bottom-Up
Approach
Business Data Lake
{
{
{
{
Enterprise Data Warehouse
Unstructured
Data Sources
Structured
Data Sources
X Unstructured
Data Sources
Structured
Data Sources
Business Models
X Unstructured
Data Sources
Structured
Data Sources
Reports
Analysis on
Data at Rest
Near Real-time
Analysis Reports
Analysis on
Data at Rest
Near Real-time
Analysis Reports
Analysis on
Data at Rest
Near Real-time
Analysis
Data Distillation Analytics
Analytics
X X
Data Marts
Analytics
Data Marts
Business Data Lake
Unstructured
Data Sources
Structured
Data Sources
Processing Insights & Action
Ingestion
Information
Delivery
Data Storage Data Storage Information
Delivery
Pivotal the way we see it
10. Tier Business Data Lake
C
O
M
P
A
R
E
T
O
Top-Down
(EDW)
Bottom-up
(Data Mart)
Storage
Process ALL Data Structured data Structured data
Cost Low High Medium
Effort Low High Low
Ingestion
Process ALL Data Sources
Multiple
structured Data
Sources
Multiple structured Data
Sources
Cost Low High, due to effort Medium, due to effort
Effort Low, as all data flows in to the lake
High, due to data
alignment into
EDW
Medium, due to data
alignment into data mart
Distilation
Done on demand based on business
needs, allowing for identifying new
patterns and relationships in existing
data.
This process is a differentiator
Already distilled
and structured
data, does not
allow for further
distillation
Already distilled,
structured and
aggregated data, does
not allow for further
distillation
Processing
Capable of handling analytics on the
data in the lake
This process is a differentiator
Not possible
directly on the
EDW
Not possible directly on
the data mart
Insights
Ability to analyze data as required. Allows
for data exploration and so enables the
discovery of new insights that were not
directly visible
Analysis needs to
be defined upfront
and hence is rigid
to the business
need
Analysis needs to be
defined upfront and
hence is rigid to the
business need
Action
Ability to integrate with Business
Decisioning systems for the next best
action
Technically
feasible, but not
effective due to
data latency
Technically feasible, but
not effective due to data
latency
Unified Data
Management
MDM and RDM on relevant data.
Effective MDM
and RDM
strategies exist,
but possibility of
over-engineering
Effective MDM and
RDM strategies exist,
but possibility of
over-engineering
10 Traditional BI vs. Business Data Lake – A comparison
11. 11
As we see, a Business Data Lake is able to:
• Receive and store high volume and volatile structured, semi-structured and
unstructured data in near-real time using low cost commodity hardware
• Provide a platform to perform near-real time analytics and business
processing on the data in the lake
• Provide a business view that is tailored to specific LOBs as well the enterprise.
The Business Data Lake does this in a way which enables users to reduce the
business solution implementation time, by:
• Eliminating the dependency of data modelling up-front and thereby letting all
data flow in
• Reducing the time taken to build robust ETL process to load the data into the
structured data stores, which are bound to change
• Eliminating an over-engineered metadata layer
• Providing the capability to view the same data in different dimensions and
derive new patterns and relationships that lie within the data.
A Business Data Lake is a simple but powerful approach to solve
business problems. It caters to ever-changing business needs by
allowing for storage of all data and providing the capability of deriving
actionable insights from any kind of data, yet working in a transparent
and seamless fashion, in an enterprise-wide environment.
Pivotal the way we see it