The document discusses challenges with traditional data warehousing and analytics including high upfront costs, difficulty managing infrastructure, and inability to scale easily. It introduces Amazon Web Services (AWS) and Amazon Redshift as a solution, allowing for easy setup of data warehousing and analytics in the cloud at low costs without large upfront investments. AWS services like Amazon Redshift provide flexible, scalable infrastructure that is easier to manage than traditional on-premise systems and enables organizations to more effectively analyze large amounts of data.
Journey to the Cloud: Database Modernization Best PracticesDatavail
In this presentation from the AWS Dallas workshop, Datavail's migration team discusses the different decision paths to the cloud, how to decide when to migrate to AWS, and 3 case studies examples of migrations to AWS
Part of proper governance in Power BI means taking proper care of what goes on in your tenant. Here's a list of areas you need to watch for and some helpful telemetry to start collecting.
As a data integration professional, it’s almost a guarantee that you’ve heard of real-time stream processing of Big Data. The usual players in the open source world are Apache Kafka, used to move data in real-time, and Spark Streaming, built for in-flight transformations. But what about relational data? Quite often we forget that products incubated in the Apache Foundation can also serve a purpose for “standard” relational databases as well. But how? Well, let’s introduce Oracle GoldenGate and Oracle Data Integrator for Big Data. GoldenGate can extract relational data in real time and produce Kafka messages, ensuring relational data is a part of the enterprise data bus. These messages can then be ingested via ODI through a Spark Streaming process, integrating with additional data sources, such as other relational tables, flat files, etc, as needed. Finally, the output can be sent to multiple locations: on through to a data warehouse for analytical reporting, back to Kafka for additional targets to consume, or any number of targets. Attendees will walk away with a framework on which they can build their data streaming projects, combining relational data with big data and using a common, structured approach via the Oracle Data Integration product stack.
Presented at BIWA Summit 2017.
Apache frameworks provide solutions for processing big and fast data. Traditional APIs use a request/response model with pull-based interactions, while modern data streaming uses a publish/subscribe model. Key concepts for big data architectures include batch processing frameworks like Hadoop, stream processing tools like Storm, and hybrid options like Spark and Flink. Popular data ingestion tools include Kafka for messaging, Flume for log data, and Sqoop for structured data. The best solution depends on requirements like latency, data volume, and workload type.
Accelerate Your ML Pipeline with AutoML and MLflowDatabricks
Building ML models is a time consuming endeavor that requires a thorough understanding of feature engineering, selecting useful features, choosing an appropriate algorithm, and performing hyper-parameter tuning. Extensive experimentation is required to arrive at a robust and performant model. Additionally, keeping track of the models that have been developed and deployed may be complex. Solving these challenges is key for successfully implementing end-to-end ML pipelines at scale.
In this talk, we will present a seamless integration of automated machine learning within a Databricks notebook, thus providing a truly unified analytics lifecycle for data scientists and business users with improved speed and efficiency. Specifically, we will show an app that generates and executes a Databricks notebook to train an ML model with H2O’s Driverless AI automatically. The resulting model will be automatically tracked and managed with MLflow. Furthermore, we will show several deployment options to score new data on a Databricks cluster or with an external REST server, all within the app.
TIQ Solutions - QlikView Data Integration in a Java WorldVizlib Ltd.
The document discusses TIQ Solutions' QlikView data integration landscape. It includes a JDBC connector that connects QlikView with various JDBC data sources like Hadoop, SAP HANA, and Neo4j. It also includes a JSON proxy server that connects QlikView with RESTful APIs and JSON data sources. Additionally, it mentions a QVX and QVD converter that enables creating and converting QlikView files for use in other Java applications and frameworks.
Denodo DataFest 2017: Edge Computing: Collecting vs. Connecting to Streaming ...Denodo
This document discusses connected data and edge computing. It summarizes that connected devices, customers, vehicles, and assets are fueling new business models powered by streaming data, artificial intelligence, cloud computing, and the internet of things. It then describes Hortonworks' data platforms for managing both data at rest and in motion across cloud, on-premises and hybrid environments to enable analytics and power the modern data architecture.
Denodo DataFest 2017: Integrating Big Data and Streaming Data with Enterprise...Denodo
Watch live presentation here: https://goo.gl/UcZEHU
Big data projects are becoming mature and consistent. However, they remain siloed compared to the enterprise data. In addition, now new streaming data needs to integrated as well.
Watch this Denodo DataFest 2017 session to discover:
• How big data projects can be combined with other enterprise data.
• How to integrate streaming data into the mix.
• Benefits of aggregating the data without having to move them into a centralized repository.
Journey to the Cloud: Database Modernization Best PracticesDatavail
In this presentation from the AWS Dallas workshop, Datavail's migration team discusses the different decision paths to the cloud, how to decide when to migrate to AWS, and 3 case studies examples of migrations to AWS
Part of proper governance in Power BI means taking proper care of what goes on in your tenant. Here's a list of areas you need to watch for and some helpful telemetry to start collecting.
As a data integration professional, it’s almost a guarantee that you’ve heard of real-time stream processing of Big Data. The usual players in the open source world are Apache Kafka, used to move data in real-time, and Spark Streaming, built for in-flight transformations. But what about relational data? Quite often we forget that products incubated in the Apache Foundation can also serve a purpose for “standard” relational databases as well. But how? Well, let’s introduce Oracle GoldenGate and Oracle Data Integrator for Big Data. GoldenGate can extract relational data in real time and produce Kafka messages, ensuring relational data is a part of the enterprise data bus. These messages can then be ingested via ODI through a Spark Streaming process, integrating with additional data sources, such as other relational tables, flat files, etc, as needed. Finally, the output can be sent to multiple locations: on through to a data warehouse for analytical reporting, back to Kafka for additional targets to consume, or any number of targets. Attendees will walk away with a framework on which they can build their data streaming projects, combining relational data with big data and using a common, structured approach via the Oracle Data Integration product stack.
Presented at BIWA Summit 2017.
Apache frameworks provide solutions for processing big and fast data. Traditional APIs use a request/response model with pull-based interactions, while modern data streaming uses a publish/subscribe model. Key concepts for big data architectures include batch processing frameworks like Hadoop, stream processing tools like Storm, and hybrid options like Spark and Flink. Popular data ingestion tools include Kafka for messaging, Flume for log data, and Sqoop for structured data. The best solution depends on requirements like latency, data volume, and workload type.
Accelerate Your ML Pipeline with AutoML and MLflowDatabricks
Building ML models is a time consuming endeavor that requires a thorough understanding of feature engineering, selecting useful features, choosing an appropriate algorithm, and performing hyper-parameter tuning. Extensive experimentation is required to arrive at a robust and performant model. Additionally, keeping track of the models that have been developed and deployed may be complex. Solving these challenges is key for successfully implementing end-to-end ML pipelines at scale.
In this talk, we will present a seamless integration of automated machine learning within a Databricks notebook, thus providing a truly unified analytics lifecycle for data scientists and business users with improved speed and efficiency. Specifically, we will show an app that generates and executes a Databricks notebook to train an ML model with H2O’s Driverless AI automatically. The resulting model will be automatically tracked and managed with MLflow. Furthermore, we will show several deployment options to score new data on a Databricks cluster or with an external REST server, all within the app.
TIQ Solutions - QlikView Data Integration in a Java WorldVizlib Ltd.
The document discusses TIQ Solutions' QlikView data integration landscape. It includes a JDBC connector that connects QlikView with various JDBC data sources like Hadoop, SAP HANA, and Neo4j. It also includes a JSON proxy server that connects QlikView with RESTful APIs and JSON data sources. Additionally, it mentions a QVX and QVD converter that enables creating and converting QlikView files for use in other Java applications and frameworks.
Denodo DataFest 2017: Edge Computing: Collecting vs. Connecting to Streaming ...Denodo
This document discusses connected data and edge computing. It summarizes that connected devices, customers, vehicles, and assets are fueling new business models powered by streaming data, artificial intelligence, cloud computing, and the internet of things. It then describes Hortonworks' data platforms for managing both data at rest and in motion across cloud, on-premises and hybrid environments to enable analytics and power the modern data architecture.
Denodo DataFest 2017: Integrating Big Data and Streaming Data with Enterprise...Denodo
Watch live presentation here: https://goo.gl/UcZEHU
Big data projects are becoming mature and consistent. However, they remain siloed compared to the enterprise data. In addition, now new streaming data needs to integrated as well.
Watch this Denodo DataFest 2017 session to discover:
• How big data projects can be combined with other enterprise data.
• How to integrate streaming data into the mix.
• Benefits of aggregating the data without having to move them into a centralized repository.
This document discusses developing a quantum-based production economy using new "green" materials. It describes leveraging growth through quantum metrics to increase performance and conversion rates. A key concept is generating a "quantum ark" production model using string-based "quantum paper" as a new economic paradigm. This would allow for advanced manufacturing through high-speed composite assembly of complex materials using quantum principles like instances and distributions. The goal is establishing a sustainable "green economy" capable of meeting the needs of an advanced civilization through wireless technologies and agile workforce training approaches.
Seamless, Real-Time Data Integration with ConnectPrecisely
As many of our customers have come to learn - integrating legacy data into modern data architecture is easier said than done! View this on-demand webinar to learn all about Precisely's seamless data integration solutions and how they have helped thousands of customers like you trust their data.
Learn about the two flavors of Precisely's Connect:
• Collect, prepare, transform and load your data to various targets using Connect ETL with the flexibility of using clusters and running on many different environments. With our 'design once, deploy anywhere' feature; what is built on prem today, can run on a cloud platform tomorrow with no development or mainframe expertise required.
• Capture data changes in real-time with no coding, tuning or performance impact using Connect CDC. Replicating exactly WHAT you need and HOW you need it with over 80 built-in data transformation methods.
Denodo DataFest 2017: Outpace Your Competition with Real-Time ResponsesDenodo
Watch the presentation on-demand now: https://goo.gl/kceFTe
Today’s digital economy demands a new way of running business. Flexible access to information and responses in real time are essential for outpacing competition.
Watch this Denodo DataFest 2017 session to discover:
• Data access challenges faced by organizations today.
• How data virtualization facilitates real-time analytics.
• Key use cases and customer success stories.
Airbyte @ Airflow Summit - The new modern data stackMichel Tricot
The document introduces the modern data stack of Airbyte, Airflow, and dbt. It discusses how ELT addresses issues with traditional ETL processes by separating extraction, loading, and transformation. Extraction and loading involve general-purpose routines to pull and push raw data, while transformation uses business logic specific to the organization. The stack is presented as an open solution that allows composing with best of breed tools for each part of the data pipeline. Airbyte provides data integration, dbt enables data transformation with SQL, and Airflow handles scheduling. The demo shows how these tools can be combined to build a flexible, autonomous, and future proof modern data stack.
Analytics-Enabled Experiences: The New Secret WeaponDatabricks
Tracking and analyzing how our individual products come together has always been an elusive problem for Steelcase. Our problem can be thought of in the following way: “we know how many Lego pieces we sell, yet we don’t know what Lego set our customers buy.” The Data Science team took over this initiative, which resulted in an evolution of our analytics journey. It is a story of innovation, resilience, agility and grit.
The effects of the COVID-19 pandemic on corporate America shined the spotlight on office furniture manufacturers to solve for ways on which the office can be made safe again. The team would have never imagined how relevant our work on product application analytics would become. Product application analytics became an industry priority overnight.
The proposal presented this year is the story of how data science is helping corporations bring people back to the office and set the path to lead the reinvention of the office space.
After groundbreaking milestones to overcome technical challenges, the most important question is: What do we do with this? How do we scale this? How do we turn this opportunity into a true competitive advantage? The response: stop thinking about this work as a data science project and start to think about this as an analytics-enabled experience.
During our session we will cover the technical elements that we overcame as a team to set-up a pipeline that ingests semi-structured and unstructured data at scale, performs analytics and produces digital experiences for multiple users.
This presentation will be particularly insightful for Data Scientists, Data Engineers and analytics leaders who are seeking to better understand how to augment the value of data for their organization
Empowering Real Time Patient Care Through Spark StreamingDatabricks
Takeda’s Plasma Derived Therapies (PDT) business unit has recently embarked on a project to use Spark Streaming on Databricks to empower how they deliver value to their Plasma Donation centers. As patients come in and interface without clinics, we store and track all of the patient interactions in real time and deliver outputs and results based on said interactions. The current problem with our existing architecture is that it is very expensive to maintain and has an unsustainable number of failure points. Spark Streaming is essential for allowing this use case because it allows for a more robust ETL pipeline. With Spark Streaming, we are able to replace our existing ETL processes (that are based on Lamdbas, step functions, triggered jobs, etc) into a purely stream driven architecture.
Data is brought into our s3 raw layer as a large set of CSV files through AWS DMS and Informatica IICS as these services bring data from on-prem systems into our cloud layer. We have a stream currently running which takes these raw files up and merges them into Delta tables established in the bronze/stage layer. We are using AWS Glue as the metadata provider for all of these operations. From the stage layer, we have another set of streams using the stage Delta tables as their source, which transform and conduct stream to stream lookups before writing the enriched records into RDS (silver/prod layer). Once the data has been merged into RDS we have a DMS task which lifts the data back into S3 as CSV files. We have a small intermediary stream which merge these CSV files into corresponding delta tables, from which we have our gold/analytic streams. The on-prem systems are able to speak to the silver layer and allow for the near real-time latency that our patient care centers require.
M|18 How We Made the Move to MariaDB at FNIMariaDB plc
FNI, a multi-tenant SaaS company providing credit strategy and loan origination services, decided to migrate from Oracle to MariaDB due to rising costs and need for a more scalable and secure solution. They evaluated several open source and commercial databases and selected MariaDB in 2015 as it met their requirements for high volume processing, failover capabilities, hardware agnosticism, scalability, and encryption. FNI implemented MariaDB in a phased approach starting with proof of concept and has now migrated 6 production and 64 test databases. MariaDB has provided cost savings and allowed FNI to standardize processes and code while improving products and services for their financial customers.
According to a recent Harvard Business Review study, there’s only a 43% chance that customers who have a poor experience will stick with you for the next 12 months. Contrast that to the 74% that will remain your customer if they have a great experience. Learn how Macy’s, a leading American department store chain founded in 1858 with over 750 stores in North America, is transforming their customer experience with DataStax Enterprise.
Webinar recording: https://youtu.be/CiUVxh6Ov_E
View current and past DataStax webinars: http://www.datastax.com/resources/webinars
This webinar follows the process of evaluating different big data platforms based on varying use cases and business requirements, and explains how big data professionals can choose the right technology to transform their business. During this session, Ooyala CTO, Sean Knapp will discuss why Ooyala selected DataStax as the big data platform powering their business, and how they provide real-time video analytics that help media companies create deeply personalized viewing experiences for more than 1/4 of all Internet video viewers each month.
View the webinar here - https://bit.ly/2ErkxYY
Enterprises are moving their data warehouse to the cloud to take advantage of reduced operational and administrative overheads, improved business agility, and unmatched simplicity.
The Impetus Workload Transformation Solution makes the journey to the cloud easier by automating the DW migration to cloud-native data warehouse platforms like Snowflake. The solution enables enterprises to automate conversion of source DDL, DML scripts, business logic, and procedural constructs. Enterprises can preserve their existing investments, eliminate error-prone, slow, and expensive manual practices, mitigate any risk, and accelerate time-to-market with the solution.
Join our upcoming webinar where Impetus experts will detail:
Cloud migration strategy
Critical considerations for moving to the cloud
Nuances of migration journey to Snowflake
Demo – Automated workload transformation to Snowflake.
To view - visit https://bit.ly/2ErkxYY
Maxis is a data services company founded in 2010 that provides Maximo consulting, upgrades, and archival services using their Alchemize software. Alchemize started as a Maximo archive solution in 2015 and has since expanded to support full data migration, transformation, and analytics capabilities. Key features of Alchemize include its ability to archive and migrate data across different systems quickly and flexibly while maintaining data integrity. It is used by various organizations for projects such as compliance, system performance optimization, and application retirement.
The document discusses Oracle Analytics Cloud and its capabilities for data visualization and storytelling. It describes how the tool allows anyone to access and analyze data from various sources to gain insights. It provides rich visualization features, collaborative sharing abilities, and can be accessed on mobile, desktop or browsers to tell data-driven stories. The key benefits highlighted are that it offers powerful yet easy-to-use analytics accessible to all users.
Jan van Ansem - Help a friend: how the Developers community can help to get Data Warehousing development up to date with modern development technology.
Helsinki Cassandra Meetup #2: Introduction to CQL3 and DataModelingBruno Amaro Almeida
- Introduction to CQL3 and DataModeling (Johnny Miller, Cassandra Solutions Architect, Datastax):
Johnny Miller is an experience developer, architect, team
lead and agile coach with a history of working at Sky, AOL
Broadband and Alcatel-Lucent. Johnny has architected and
delivered a number of platforms using Cassandra as a key
component for achieving high availability and efficient scaling.
The document discusses Red Hat's CloudForms product and its capabilities for managing containers and Kubernetes/OpenShift environments. It provides an overview of CloudForms' integration with Kubernetes and OpenShift, how it allows monitoring and management of containers, pods, images, nodes and other resources. It also demonstrates CloudForms' topology views and dashboards for containers. The objectives of the event are to share knowledge about Red Hat's container solutions and how CloudForms addresses common concerns around managing containers.
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...Databricks
Zalando transitioned from a centralized data platform to a data mesh architecture. This decentralized their data infrastructure by having individual domains own datasets and pipelines rather than a central team. It provided self-service data infrastructure tools and governance to enable domains to operate independently while maintaining global interoperability. This improved data quality by making domains responsible for their data and empowering them through the data mesh approach.
Audience Level
All levels
Synopsis
Our journey towards solving our Application and Infrastructure Problems using Immutability, Codification, Mesos, Docker and Ironic.
OpenStack Australia Day Melbourne 2017
https://events.aptira.com/openstack-australia-day-melbourne-2017/
Snowflake + Power BI: Cloud Analytics for EveryoneAngel Abundez
This document discusses architectures for using Snowflake and Power BI together. It begins by describing the benefits of each technology. It then outlines several architectural scenarios for connecting Snowflake to Power BI, including using a Power BI gateway, without a gateway, and connecting to Analysis Services. The document also provides examples of usage scenarios and developer best practices. It concludes with a section on data governance considerations for architectures with and without a Power BI gateway.
This webinar discussed how Jaspersoft delivered Cloud BI 60% faster for Kony by using Jaspersoft on AWS. It covered an introduction to AWS Cloud and big data services, Jaspersoft capabilities for building reports and dashboards from various data sources, and a customer case study of how Kony was able to build reports in 5 days instead of 4 weeks using this solution. The webinar emphasized that Jaspersoft on AWS provides flexible, affordable, self-service business intelligence without risks through its pay-as-you-go model starting at less than $1 per hour.
This document discusses developing a quantum-based production economy using new "green" materials. It describes leveraging growth through quantum metrics to increase performance and conversion rates. A key concept is generating a "quantum ark" production model using string-based "quantum paper" as a new economic paradigm. This would allow for advanced manufacturing through high-speed composite assembly of complex materials using quantum principles like instances and distributions. The goal is establishing a sustainable "green economy" capable of meeting the needs of an advanced civilization through wireless technologies and agile workforce training approaches.
Seamless, Real-Time Data Integration with ConnectPrecisely
As many of our customers have come to learn - integrating legacy data into modern data architecture is easier said than done! View this on-demand webinar to learn all about Precisely's seamless data integration solutions and how they have helped thousands of customers like you trust their data.
Learn about the two flavors of Precisely's Connect:
• Collect, prepare, transform and load your data to various targets using Connect ETL with the flexibility of using clusters and running on many different environments. With our 'design once, deploy anywhere' feature; what is built on prem today, can run on a cloud platform tomorrow with no development or mainframe expertise required.
• Capture data changes in real-time with no coding, tuning or performance impact using Connect CDC. Replicating exactly WHAT you need and HOW you need it with over 80 built-in data transformation methods.
Denodo DataFest 2017: Outpace Your Competition with Real-Time ResponsesDenodo
Watch the presentation on-demand now: https://goo.gl/kceFTe
Today’s digital economy demands a new way of running business. Flexible access to information and responses in real time are essential for outpacing competition.
Watch this Denodo DataFest 2017 session to discover:
• Data access challenges faced by organizations today.
• How data virtualization facilitates real-time analytics.
• Key use cases and customer success stories.
Airbyte @ Airflow Summit - The new modern data stackMichel Tricot
The document introduces the modern data stack of Airbyte, Airflow, and dbt. It discusses how ELT addresses issues with traditional ETL processes by separating extraction, loading, and transformation. Extraction and loading involve general-purpose routines to pull and push raw data, while transformation uses business logic specific to the organization. The stack is presented as an open solution that allows composing with best of breed tools for each part of the data pipeline. Airbyte provides data integration, dbt enables data transformation with SQL, and Airflow handles scheduling. The demo shows how these tools can be combined to build a flexible, autonomous, and future proof modern data stack.
Analytics-Enabled Experiences: The New Secret WeaponDatabricks
Tracking and analyzing how our individual products come together has always been an elusive problem for Steelcase. Our problem can be thought of in the following way: “we know how many Lego pieces we sell, yet we don’t know what Lego set our customers buy.” The Data Science team took over this initiative, which resulted in an evolution of our analytics journey. It is a story of innovation, resilience, agility and grit.
The effects of the COVID-19 pandemic on corporate America shined the spotlight on office furniture manufacturers to solve for ways on which the office can be made safe again. The team would have never imagined how relevant our work on product application analytics would become. Product application analytics became an industry priority overnight.
The proposal presented this year is the story of how data science is helping corporations bring people back to the office and set the path to lead the reinvention of the office space.
After groundbreaking milestones to overcome technical challenges, the most important question is: What do we do with this? How do we scale this? How do we turn this opportunity into a true competitive advantage? The response: stop thinking about this work as a data science project and start to think about this as an analytics-enabled experience.
During our session we will cover the technical elements that we overcame as a team to set-up a pipeline that ingests semi-structured and unstructured data at scale, performs analytics and produces digital experiences for multiple users.
This presentation will be particularly insightful for Data Scientists, Data Engineers and analytics leaders who are seeking to better understand how to augment the value of data for their organization
Empowering Real Time Patient Care Through Spark StreamingDatabricks
Takeda’s Plasma Derived Therapies (PDT) business unit has recently embarked on a project to use Spark Streaming on Databricks to empower how they deliver value to their Plasma Donation centers. As patients come in and interface without clinics, we store and track all of the patient interactions in real time and deliver outputs and results based on said interactions. The current problem with our existing architecture is that it is very expensive to maintain and has an unsustainable number of failure points. Spark Streaming is essential for allowing this use case because it allows for a more robust ETL pipeline. With Spark Streaming, we are able to replace our existing ETL processes (that are based on Lamdbas, step functions, triggered jobs, etc) into a purely stream driven architecture.
Data is brought into our s3 raw layer as a large set of CSV files through AWS DMS and Informatica IICS as these services bring data from on-prem systems into our cloud layer. We have a stream currently running which takes these raw files up and merges them into Delta tables established in the bronze/stage layer. We are using AWS Glue as the metadata provider for all of these operations. From the stage layer, we have another set of streams using the stage Delta tables as their source, which transform and conduct stream to stream lookups before writing the enriched records into RDS (silver/prod layer). Once the data has been merged into RDS we have a DMS task which lifts the data back into S3 as CSV files. We have a small intermediary stream which merge these CSV files into corresponding delta tables, from which we have our gold/analytic streams. The on-prem systems are able to speak to the silver layer and allow for the near real-time latency that our patient care centers require.
M|18 How We Made the Move to MariaDB at FNIMariaDB plc
FNI, a multi-tenant SaaS company providing credit strategy and loan origination services, decided to migrate from Oracle to MariaDB due to rising costs and need for a more scalable and secure solution. They evaluated several open source and commercial databases and selected MariaDB in 2015 as it met their requirements for high volume processing, failover capabilities, hardware agnosticism, scalability, and encryption. FNI implemented MariaDB in a phased approach starting with proof of concept and has now migrated 6 production and 64 test databases. MariaDB has provided cost savings and allowed FNI to standardize processes and code while improving products and services for their financial customers.
According to a recent Harvard Business Review study, there’s only a 43% chance that customers who have a poor experience will stick with you for the next 12 months. Contrast that to the 74% that will remain your customer if they have a great experience. Learn how Macy’s, a leading American department store chain founded in 1858 with over 750 stores in North America, is transforming their customer experience with DataStax Enterprise.
Webinar recording: https://youtu.be/CiUVxh6Ov_E
View current and past DataStax webinars: http://www.datastax.com/resources/webinars
This webinar follows the process of evaluating different big data platforms based on varying use cases and business requirements, and explains how big data professionals can choose the right technology to transform their business. During this session, Ooyala CTO, Sean Knapp will discuss why Ooyala selected DataStax as the big data platform powering their business, and how they provide real-time video analytics that help media companies create deeply personalized viewing experiences for more than 1/4 of all Internet video viewers each month.
View the webinar here - https://bit.ly/2ErkxYY
Enterprises are moving their data warehouse to the cloud to take advantage of reduced operational and administrative overheads, improved business agility, and unmatched simplicity.
The Impetus Workload Transformation Solution makes the journey to the cloud easier by automating the DW migration to cloud-native data warehouse platforms like Snowflake. The solution enables enterprises to automate conversion of source DDL, DML scripts, business logic, and procedural constructs. Enterprises can preserve their existing investments, eliminate error-prone, slow, and expensive manual practices, mitigate any risk, and accelerate time-to-market with the solution.
Join our upcoming webinar where Impetus experts will detail:
Cloud migration strategy
Critical considerations for moving to the cloud
Nuances of migration journey to Snowflake
Demo – Automated workload transformation to Snowflake.
To view - visit https://bit.ly/2ErkxYY
Maxis is a data services company founded in 2010 that provides Maximo consulting, upgrades, and archival services using their Alchemize software. Alchemize started as a Maximo archive solution in 2015 and has since expanded to support full data migration, transformation, and analytics capabilities. Key features of Alchemize include its ability to archive and migrate data across different systems quickly and flexibly while maintaining data integrity. It is used by various organizations for projects such as compliance, system performance optimization, and application retirement.
The document discusses Oracle Analytics Cloud and its capabilities for data visualization and storytelling. It describes how the tool allows anyone to access and analyze data from various sources to gain insights. It provides rich visualization features, collaborative sharing abilities, and can be accessed on mobile, desktop or browsers to tell data-driven stories. The key benefits highlighted are that it offers powerful yet easy-to-use analytics accessible to all users.
Jan van Ansem - Help a friend: how the Developers community can help to get Data Warehousing development up to date with modern development technology.
Helsinki Cassandra Meetup #2: Introduction to CQL3 and DataModelingBruno Amaro Almeida
- Introduction to CQL3 and DataModeling (Johnny Miller, Cassandra Solutions Architect, Datastax):
Johnny Miller is an experience developer, architect, team
lead and agile coach with a history of working at Sky, AOL
Broadband and Alcatel-Lucent. Johnny has architected and
delivered a number of platforms using Cassandra as a key
component for achieving high availability and efficient scaling.
The document discusses Red Hat's CloudForms product and its capabilities for managing containers and Kubernetes/OpenShift environments. It provides an overview of CloudForms' integration with Kubernetes and OpenShift, how it allows monitoring and management of containers, pods, images, nodes and other resources. It also demonstrates CloudForms' topology views and dashboards for containers. The objectives of the event are to share knowledge about Red Hat's container solutions and how CloudForms addresses common concerns around managing containers.
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...Databricks
Zalando transitioned from a centralized data platform to a data mesh architecture. This decentralized their data infrastructure by having individual domains own datasets and pipelines rather than a central team. It provided self-service data infrastructure tools and governance to enable domains to operate independently while maintaining global interoperability. This improved data quality by making domains responsible for their data and empowering them through the data mesh approach.
Audience Level
All levels
Synopsis
Our journey towards solving our Application and Infrastructure Problems using Immutability, Codification, Mesos, Docker and Ironic.
OpenStack Australia Day Melbourne 2017
https://events.aptira.com/openstack-australia-day-melbourne-2017/
Snowflake + Power BI: Cloud Analytics for EveryoneAngel Abundez
This document discusses architectures for using Snowflake and Power BI together. It begins by describing the benefits of each technology. It then outlines several architectural scenarios for connecting Snowflake to Power BI, including using a Power BI gateway, without a gateway, and connecting to Analysis Services. The document also provides examples of usage scenarios and developer best practices. It concludes with a section on data governance considerations for architectures with and without a Power BI gateway.
This webinar discussed how Jaspersoft delivered Cloud BI 60% faster for Kony by using Jaspersoft on AWS. It covered an introduction to AWS Cloud and big data services, Jaspersoft capabilities for building reports and dashboards from various data sources, and a customer case study of how Kony was able to build reports in 5 days instead of 4 weeks using this solution. The webinar emphasized that Jaspersoft on AWS provides flexible, affordable, self-service business intelligence without risks through its pay-as-you-go model starting at less than $1 per hour.
FSI201 FINRA’s Managed Data Lake – Next Gen Analytics in the CloudAmazon Web Services
FINRA’s Data Lake unlocks the value in its data to accelerate analytics and machine learning at scale. FINRA's Technology group has changed its customer's relationship with data by creating a Managed Data Lake that enables discovery on Petabytes of capital markets data, while saving time and money over traditional analytics solutions. FINRA’s Managed Data Lake includes a centralized data catalog and separates storage from compute, allowing users to query from petabytes of data in seconds. Learn how FINRA uses Spot instances and services such as Amazon S3, Amazon EMR, Amazon Redshift, and AWS Lambda to provide the 'right tool for the right job' at each step in the data processing pipeline. All of this is done while meeting FINRA’s security and compliance responsibilities as a financial regulator.
Unlock Data-driven Insights in Databricks Using Location IntelligencePrecisely
Today’s data-driven organisations are turning to Databricks for a cloud-based, open, unified platform for data and AI. Yet many companies struggle to unlock the value of the data they have in Databricks. To capitalise on the promise of a competitive edge through increased efficiency and insight, data scientists are turning to location to make sense of massive volumes of business data.
Watch this on-demand to hear from The Spatial Distillery Co. and Databricks on how to leverage advanced location intelligence and enrichment solutions in Databricks to:
- Simplify the complexity of location data and transform it into valuable insights
- Enrich data with thousands of attributes for better, more accurate analytics, AI, and ML models
- Leverage the power of Databricks to integrate geospatial data into business processes for real-time answers
- Create more meaningful and timely customer interactions by streamlining customer-facing and operational tasks
Attributes of a Modern Data Warehouse - Gartner CatalystJack Mardack
Most data-driven enterprises continue to struggle to generate the insights they need from their data. More data volumes from more data sources, combined with escalating user concurrency, have led to declining query throughput performance and skyrocketing data warehouse costs. Moreover, modern use cases such as customer-360 and hyper-personalization have blurred the boundaries between operational and analytics systems, making even greater demands on data warehouse solutions.
Join us for a series of introductory and technical sessions on AWS Big Data solutions. Gain a thorough understanding of what Amazon Web Services offers across the big data lifecycle and learn architectural best practices for applying those solutions to your projects.
We will kick off this technical seminar in the morning with an introduction to the AWS Big Data platform, including a discussion of popular use cases and reference architectures. In the afternoon, we will deep dive into Machine Learning and Streaming Analytics. We will then walk everyone through building your first Big Data application with AWS.
AWS Summit Berlin 2013 - Big Data AnalyticsAWS Germany
Learn more about the tools, techniques and technologies for working productively with data at any scale. This session will introduce the family of data analytics tools on AWS which you can use to collect, compute and collaborate around data, from gigabytes to petabytes. We'll discuss Amazon Elastic MapReduce, Hadoop, structured and unstructured data, and the EC2 instance types which enable high performance analytics.
This session provides an introduction to the AWS platform and services. It explains how you can get started on your cloud journey and what resources you can use build sophisticated applications with increased flexibility, scalability and reliability. The session also covers the benefits customers are enjoying by moving to AWS cloud; increased agility, faster decision making and the ability to fail fast and innovate.
ADV Slides: Comparing the Enterprise Analytic SolutionsDATAVERSITY
Data is the foundation of any meaningful corporate initiative. Fully master the necessary data, and you’re more than halfway to success. That’s why leverageable (i.e., multiple use) artifacts of the enterprise data environment are so critical to enterprise success.
Build them once (keep them updated), and use again many, many times for many and diverse ends. The data warehouse remains focused strongly on this goal. And that may be why, nearly 40 years after the first database was labeled a “data warehouse,” analytic database products still target the data warehouse.
Businesses are generating more data than ever before.
Doing real time data analytics requires IT infrastructure that often needs to be scaled up quickly and running an on-premise environment in this setting has its limitations.
Organisations often require a massive amount of IT resources to analyse their data and the upfront capital cost can deter them from embarking on these projects.
What’s needed is scalable, agile and secure cloud-based infrastructure at the lowest possible cost so they can spin up servers that support their data analysis projects exactly when they are required. This infrastructure must enable them to create proof-of-concepts quickly and cheaply – to fail fast and move on.
Amazon Redshift Update and How Equinox Fitness Clubs Migrated to a Modern Dat...Amazon Web Services
In this session, we provide an update on Amazon Redshift, and look at a case study from Equinox Fitness Clubs. We show you how Amazon Redshift queries data across your data warehouse and data lake, without the need or delay of loading data, to deliver insights you cannot obtain by querying independent data silos. Discover how Equinox Fitness Clubs transitioned from on-premises data warehouses and data marts to a cloud-based, integrated data platform, built on AWS and Amazon Redshift. Learn about their journey from static reports, redundant data, and inefficient data integration to a modern and flexible data lake and data warehouse architecture that delivers dynamic reports based on trusted data.
MongoDB World 2019: re:Innovate from Siloed to Deep Insights on Your DataMongoDB
Are you tired of tedious and long data-to-insights journey, siloed data and unleveraged Data? Would you like existing demographic data help you drive business outcome? Would you like NOT to create any data lake and direct insights on data with pre-fabricated data structure without any efforts?
DataLakes kan skalere i takt med skyen, nedbryde integrationsbarrierer og data gemt i siloer og bane vejen for nye forretningsmuligheder. Det er alt sammen med til at give et bedre beslutningsgrundlag for ledelse og medarbejdere. Kom og hør hvordan.
David Bojsen, Arkitekt, Microsoft
This overview presentation discusses big data challenges and provides an overview of the AWS Big Data Platform by covering:
- How AWS customers leverage the platform to manage massive volumes of data from a variety of sources while containing costs.
- Reference architectures for popular use cases, including, connected devices (IoT), log streaming, real-time intelligence, and analytics.
- The AWS big data portfolio of services, including, Amazon S3, Kinesis, DynamoDB, Elastic MapReduce (EMR), and Redshift.
- The latest relational database engine, Amazon Aurora— a MySQL-compatible, highly-available relational database engine, which provides up to five times better performance than MySQL at one-tenth the cost of a commercial database.
Created by: Rahul Pathak,
Sr. Manager of Software Development
This document provides an overview of Amazon Web Services (AWS) and why customers use AWS. It discusses how AWS enables agility for customers, allows them to avoid undifferentiated heavy lifting of managing infrastructure, provides a broad platform for innovation at scale, and offers cost savings and flexibility through various pricing models. The document then highlights how a variety of Nordic companies are using AWS across different use cases like development and testing, new workloads, supplementing existing workloads and infrastructure, migrating applications, data center migration, and moving entire IT operations to the cloud.
Last week, June 11th, AWS hosted a successful Partner Day in London, targeted at our existing APN partners.
This is what we've covered during the sessions:
- AWS product and services update
- The AWS partner program benefits and opportunities
- How to develop your partnership with AWS
- AWS competency program
- How to resell AWS services
¿Cómo modernizar una arquitectura de TI con la virtualización de datos?Denodo
Ver: https://bit.ly/347ImDf
En la era digital, la gestión eficiente de los datos es un factor fundamental para optimizar la competitividad de las empresas. Sin embargo, la mayoría de ellas se enfrentan a silos de datos, lo que hace que su tratamiento sea lento y costoso. Además, la velocidad, la diversidad y el volumen de los datos pueden superar las arquitecturas de TI tradicionales.
¿Cómo mejorar la entrega de datos para extraer todo su valor?
¿Cómo conseguir que los datos estén disponibles y poder utilizarlos en tiempo real?
Los expertos de Vault IT y Denodo te proponen este webinar para descubrir cómo la virtualización de datos permite modernizar una arquitectura de TI en un contexto de transformación digital.
Analytics in a Day Ft. Synapse Virtual WorkshopCCG
Say goodbye to data silos! Analytics in a Day will simplify and accelerate your journey towards the modern data warehouse. Join CCG and Microsoft for a half-day virtual workshop, hosted by James McAuliffe.
This document summarizes a webinar about using Informatica Cloud to load big data into AWS services like Amazon Redshift for analytics. It discusses how Informatica Cloud can help consolidate and analyze customer data from multiple sources for a company called UBM to improve customer insights. The webinar also provides an example of how UBM used Informatica Cloud and Redshift to better understand customer behaviors and identify potential event attendees through analytics.
The document provides an overview of AWSome Day, an AWS event. It discusses the top reasons customers use AWS, including agility, platform breadth, innovation at scale, and cost savings/flexibility. It then discusses how various organizations in the Nordics are using AWS for development/testing, new workloads, supplementing existing workloads, migrating applications, data center migration, and moving their entire IT to the cloud.
사례로 알아보는 Database Migration Service : 데이터베이스 및 데이터 이관, 통합, 분리, 분석의 도구 - 발표자: ...Amazon Web Services Korea
Database Migration Service(DMS)는 RDBMS 이외에도 다양한 데이터베이스 이관을 지원합니다. 실제 고객사 사례를 통해 DMS가 데이터베이스 이관, 통합, 분리를 수행하는 데 어떻게 활용되는지 알아보고, 동시에 데이터 분석을 위한 데이터 수집(Data Ingest)에도 어떤 역할을 하는지 살펴보겠습니다.
Amazon Elasticache - Fully managed, Redis & Memcached Compatible Service (Lev...Amazon Web Services Korea
Amazon ElastiCache는 Redis 및 MemCached와 호환되는 완전관리형 서비스로서 현대적 애플리케이션의 성능을 최적의 비용으로 실시간으로 개선해 줍니다. ElastiCache의 Best Practice를 통해 최적의 성능과 서비스 최적화 방법에 대해 알아봅니다.
Internal Architecture of Amazon Aurora (Level 400) - 발표자: 정달영, APAC RDS Speci...Amazon Web Services Korea
ccAmazon Aurora 데이터베이스는 클라우드용으로 구축된 관계형 데이터베이스입니다. Aurora는 상용 데이터베이스의 성능과 가용성, 그리고 오픈소스 데이터베이스의 단순성과 비용 효율성을 모두 제공합니다. 이 세션은 Aurora의 고급 사용자들을 위한 세션으로써 Aurora의 내부 구조와 성능 최적화에 대해 알아봅니다.
[Keynote] 슬기로운 AWS 데이터베이스 선택하기 - 발표자: 강민석, Korea Database SA Manager, WWSO, A...Amazon Web Services Korea
오랫동안 관계형 데이터베이스가 가장 많이 사용되었으며 거의 모든 애플리케이션에서 널리 사용되었습니다. 따라서 애플리케이션 아키텍처에서 데이터베이스를 선택하기가 더 쉬웠지만, 구축할 수 있는 애플리케이션의 유형이 제한적이었습니다. 관계형 데이터베이스는 스위스 군용 칼과 같아서 많은 일을 할 수 있지만 특정 업무에는 완벽하게 적합하지는 않습니다. 클라우드 컴퓨팅의 등장으로 경제적인 방식으로 더욱 탄력적이고 확장 가능한 애플리케이션을 구축할 수 있게 되면서 기술적으로 가능한 일이 달라졌습니다. 이러한 변화는 전용 데이터베이스의 부상으로 이어졌습니다. 개발자는 더 이상 기본 관계형 데이터베이스를 사용할 필요가 없습니다. 개발자는 애플리케이션의 요구 사항을 신중하게 고려하고 이러한 요구 사항에 맞는 데이터베이스를 선택할 수 있습니다.
Demystify Streaming on AWS - 발표자: 이종혁, Sr Analytics Specialist, WWSO, AWS :::...Amazon Web Services Korea
실시간 분석은 AWS 고객의 사용 사례가 점점 늘어나고 있습니다. 이 세션에 참여하여 스트리밍 데이터 기술이 어떻게 데이터를 즉시 분석하고, 시스템 간에 데이터를 실시간으로 이동하고, 실행 가능한 통찰력을 더 빠르게 얻을 수 있는지 알아보십시오. 일반적인 스트리밍 데이터 사용 사례, 비즈니스에서 실시간 분석을 쉽게 활성화하는 단계, AWS가 Amazon Kinesis와 같은 AWS 스트리밍 데이터 서비스를 사용하도록 지원하는 방법을 다룹니다.
Amazon EMR - Enhancements on Cost/Performance, Serverless - 발표자: 김기영, Sr Anal...Amazon Web Services Korea
Amazon EMR은 Apache Spark, Hive, Presto, Trino, HBase 및 Flink와 같은 오픈 소스 프레임워크를 사용하여 분석 애플리케이션을 쉽게 실행할 수 있는 관리형 서비스를 제공합니다. Spark 및 Presto용 Amazon EMR 런타임에는 오픈 소스 Apache Spark 및 Presto에 비해 두 배 이상의 성능 향상을 제공하는 최적화 기능이 포함되어 있습니다. Amazon EMR Serverless는 Amazon EMR의 새로운 배포 옵션이지만 데이터 엔지니어와 분석가는 클라우드에서 페타바이트 규모의 데이터 분석을 쉽고 비용 효율적으로 실행할 수 있습니다. 이 세션에 참여하여 개념, 설계 패턴, 라이브 데모를 사용하여 Amazon EMR/EMR 서버리스를 살펴보고 Spark 및 Hive 워크로드, Amazon EMR 스튜디오 및 Amazon SageMaker Studio와의 Amazon EMR 통합을 실행하는 것이 얼마나 쉬운지 알아보십시오.
Amazon OpenSearch - Use Cases, Security/Observability, Serverless and Enhance...Amazon Web Services Korea
로그 및 지표 데이터를 쉽게 가져오고, OpenSearch 검색 API를 사용하고, OpenSearch 대시보드를 사용하여 시각화를 구축하는 등 Amazon OpenSearch의 새로운 기능과 기능에 대해 자세히 알아보십시오. 애플리케이션 문제를 디버깅할 수 있는 OpenSearch의 Observability 기능에 대해 알아보세요. Amazon OpenSearch Service를 통해 인프라 관리에 대해 걱정하지 않고 검색 또는 모니터링 문제에 집중할 수 있는 방법을 알아보십시오.
Enabling Agility with Data Governance - 발표자: 김성연, Analytics Specialist, WWSO,...Amazon Web Services Korea
데이터 거버넌스는 전체 프로세스에서 데이터를 관리하여 데이터의 정확성과 완전성을 보장하고 필요한 사람들이 데이터에 액세스할 수 있도록 하는 프로세스입니다. 이 세션에 참여하여 AWS가 어떻게 분석 서비스 전반에서 데이터 준비 및 통합부터 데이터 액세스, 데이터 품질 및 메타데이터 관리에 이르기까지 포괄적인 데이터 거버넌스를 제공하는지 알아보십시오. AWS에서의 스트리밍에 대해 자세히 알아보십시오.
Amazon Redshift Deep Dive - Serverless, Streaming, ML, Auto Copy (New feature...Amazon Web Services Korea
이 세션에 참여하여 Amazon Redshift의 새로운 기능을 자세히 살펴보십시오. Amazon Data Sharing, Amazon Redshift Serverless, Redshift Streaming, Redshift ML 및 자동 복사 등에 대한 자세한 내용과 데모를 통해 Amazon Redshift의 새로운 기능을 알고 싶은 사용자에게 적합합니다.
From Insights to Action, How to build and maintain a Data Driven Organization...Amazon Web Services Korea
데이터는 혁신과 변혁의 토대입니다. 비즈니스 혁신을 이끄는 혁신은 특정 시점의 전략이나 솔루션이 아니라 성장을 위한 반복적이고 집단적인 계획입니다. 혁신에 이러한 접근 방식을 채택하는 기업은 전략과 비즈니스 문화에서 데이터를 기반으로 하는 경우가 많습니다. 이러한 접근 방식을 개발하려면 리더가 데이터를 조직의 자산처럼 취급하고 조직이 더 나은 비즈니스 성과를 위해 데이터를 활용할 수 있도록 권한을 부여해야 합니다. AWS와 Amazon이 어떻게 데이터와 분석을 활용하여 확장 가능한 비즈니스 효율성을 창출하고 고객의 가장 복잡한 문제를 해결하는 메커니즘을 개발했는지 알아보십시오.
[Keynote] Accelerating Business Outcomes with AWS Data - 발표자: Saeed Gharadagh...Amazon Web Services Korea
데이터는 최종 소비자의 성공에 초점을 맞춘 디지털 혁신에서 중추적인 역할을 하고 있습니다. 모든 기업들은 데이터를 자산으로 사용하여 사례 제공을 추진하고 까다로운 결과를 해결하고 있습니다. AWS 클라우드 기술과 분석 솔루션의 강력한 성능을 통해 고객은 혁신 여정을 가속화할 수 있습니다. 이 세션에서는 기업 고객들이 클라우드에서 데이터의 힘을 활용하여 혁신 목표를 달성하고 필요한 결과를 제공하는 방법에 대해 다룹니다.
LG전자 - Amazon Aurora 및 RDS 블루/그린 배포를 이용한 데이터베이스 업그레이드 안정성 확보 - 발표자: 이은경 책임, L...Amazon Web Services Korea
LG ThinQ는 LG전자의 가전제품과 서비스를 아우르는 플랫폼 브랜드로서 앱 하나로 간편한 컨트롤, 똑똑한 케어, 스마트한 쇼핑까지 한번에 가능한 플랫폼입니다. ThinQ 플랫폼은 글로벌 서비스로 제공되고 있어, 작업 시간을 최소화하고, 서비스의 영향을 최소화 할 필요가 있었습니다. 따라서 DB 버전 업그레이드 작업 시 애플리케이션 배포가 필요없는 Blue/Green Deployment 방식은 최선의 선택이 되었습니다.
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...Amazon Web Services Korea
온프레미스 분석 플랫폼에는 자원 증설 비용, 자원 관리 비용, 신규 자원 도입 및 환경 설정의 리드타임 등 다양한 측면에서의 한계가 존재합니다. 이에 KB국민카드에서는 기존 분석 플랫폼의 한계를 극복함과 동시에 시너지를 낼 수 있는 클라우드 기반 분석 플랫폼을 설계 및 도입하였습니다. 본 사례 소개는 KB국민카드의 데이터 혁신 여정과 노하우를 소개합니다.
SK Telecom - 망관리 프로젝트 TANGO의 오픈소스 데이터베이스 전환 여정 - 발표자 : 박승전, Project Manager, ...Amazon Web Services Korea
SK Telecom의 망관리 프로젝트인 TANGO에서는 오라클을 기반으로 시스템을 구축하여 운영해 왔습니다. 하지만 늘어나는 사용자와 데이터로 인해 유연하고 비용 효율적인 인프라가 필요하게 되었고, 이에 클라우드 도입을 검토 및 실행에 옮기게 되었습니다. TANGO 프로젝트의 클라우드 도입을 위한 검토부터 준비, 실행 및 이를 통해 얻게 된 교훈과 향후 계획에 대해 소개합니다.
코리안리 - 데이터 분석 플랫폼 구축 여정, 그 시작과 과제 - 발표자: 김석기 그룹장, 데이터비즈니스센터, 메가존클라우드 ::: AWS ...Amazon Web Services Korea
2022년 코리안리는 핵심업무시스템(기간계/정보계 시스템)을 AWS 클라우드로 전환하는 사업과 AWS 클라우드 기반에서 손익분석을 위한 어플리케이션 구축 사업을 동시에 진행하고 있었습니다. 이에 따라 클라우드 전환 이후 시스템 간 상호운용성과 호환성을갖춘 데이터 분석 플랫폼 또한 필요하게 되었습니다. 코리안리 IT 환경에 적합한 플랫폼 선정을 위하여 AWS Native Analytics Platform, 3rd Party Analytics Platform (클라우데라, 데이터브릭스)과의 PoC를 진행하고, 최종적으로 AWS Native Analytics Platform 으로 확정하였습니다. 코리안리는 메가존클라우드와 함께 2022년 10월부터 4개월(구축 3개월, 안정화 및 교육 1개월) 동안 AWS 기반 데이터 분석 플랫폼을 구축하고 활용 범위를 지속적으로 확대하고 있습니다.
LG 이노텍 - Amazon Redshift Serverless를 활용한 데이터 분석 플랫폼 혁신 과정 - 발표자: 유재상 선임, LG이노...Amazon Web Services Korea
LG 이노텍은 세계 시장을 선도하는 글로벌 소재·부품기업으로, Amazon Redshift 을 데이터 분석 플랫폼의 핵심 서비스로 활용하고 있습니다.지속적인 데이터 증가와 업무 확대에 따른 유연한 아키텍처 개선의 필요성에 대처하기 위해, 2022년에 AWS 에서 발표된 Redshift Serverless 를 활용한, 비용 최적화된 아키텍처 개선 과정의 실사례를 엿볼수 있는 기회가 됩니다.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
8. Take a look a data processing “pipeline”
Generation
Collection & storage
Analytics & computation
Collaboration & sharing
9. What has changed in this pipeline
Data is available
everywhere, contains
customer insight and
costs little to generate,
but..,
Generation
Collection & storage
Analytics & computation
Collaboration & sharing
11. Big Gap in turning data into actionable
information
12. The Explosion of Data
Existing Challenges with Analytics
The Cloud
13. Challenge 1: Capex Intensive
Provision all your infrastructure and tools before you get results
Cost of your infrastructure dictates what analytics you can perform
Source: Oracle technology global price list 11/1/2012
14. Most data never makes it to a data warehouse
The Data Analysis Gap
Enterprise Data is growing at over 50%
yearly
Data Warehousing growing at less than
10% yearly
1990
2000
2010
2020
Enterprise Data
Data in Warehouse
Sources:
Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011
IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares
Most data is left on the floor
15. Challenge 2: Hard to setup, manage and scale
Setup takes months of planning and work
Extending your data-warehouse can be heavy on time and cost
Managing a data analytics platform requires expensive staff
Complex tuning and management skills required
Enterprises average between 3 and 4 DBAs per data
warehouse
Gartner: Critical factors in calculating the data warehouse TCO, July 2009
16. Very hard to move up the stack
These make it extremely hard to
move up the Business Intelligence
Maturity Stack
17. The Explosion of Data
Existing Challenges with Analytics
The Cloud
24. Value proposition of the AWS cloud
No Upfront Investment
Low ongoing cost
Flexible capacity
Replace capital expenditure with
variable expense
Customers leverage our
economies of scale
No need to guess capacity
requirements and overprovision
37
PRICE
REDUCTIONS
Speed and agility
Focus on business
Global Reach
Infrastructure in minutes not
weeks
Not undifferentiated heavy
lifting
Go global in minutes and reach
a global audience
25. Architected for Enterprise Security Requirements
“The Amazon Virtual Private Cloud
[Amazon VPC] was a unique option that
offered an additional level of security and
an ability to integrate with other aspects of
our infrastructure.”
Dr. Michael Miller, Head of HPC for R&D
26. Gartner Magic Quadrant for Cloud Infrastructure as a Service
(August 19, 2013)
Gartner “Magic Quadrant for Cloud Infrastructure as a Service,” Lydia Leong, Douglas Toombs, Bob Gill, Gregor Petri, Tiny Haynes, August 19, 2013. This Magic Quadrant graphic was published by Gartner, Inc. as part of a
larger research note and should be evaluated in the context of the entire report.. The Gartner report is available upon request from Steven Armstrong (asteven@amazon.com). Gartner does not endorse any vendor, product or
service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings. Gartner research publications consist of the opinions of Gartner's research organization
and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
27. Summarizing the problem and the opportunity
The Explosion of Data
Data is a competitive edge
Existing challenges with
analytics
Hard and expensive to setup,
manage and scale
The Cloud
Lowers cost and improves
agility
28. The Solution
Data Analytics in the Cloud
Easy and inexpensive to get started
Easy to setup, scale and manage
Low cost to enable analytics on all your data
Open and flexible
29. Technology Process View
Data
source 1
Data
Data
source n
source 1
Extract Transform,
Load and Cleanse
Data
warehouse
Analytics
Analytics
Unstructur
ed data
sources
The diagram above shows functional architecture components of any data warehousing
project.
30. Source systems
Data
source 1
Data
Data
source n
source 1
Extract Transform,
Load and Cleanse
Data
warehouse
Analytics
Analytics
Unstructur
ed data
sources
The diagram above shows functional architecture components of any data warehousing
project.
31. Data Integration
Data
source 1
Data
Data
source n
source 1
Extract Transform,
Load and Cleanse
Data
warehouse
Analytics
Analytics
Unstructur
ed data
sources
The diagram above shows functional architecture components of any data warehousing
project.
32. The Data Warehouse
Data
source 1
Data
Data
source n
source 1
Extract Transform,
Load and Cleanse
Data
warehouse
Analytics
Analytics
Unstructur
ed data
sources
The diagram above shows functional architecture components of any data warehousing
project.
33. Business Intelligence and Analytics
Data
source 1
Data
Data
source n
source 1
Extract Transform,
Load and Cleanse
Data
warehouse
Analytics
Analytics
Unstructur
ed data
sources
The diagram above shows functional architecture components of any data warehousing
project.
34. Data Analytics -Technology Stack
Amazon Redshift
Data
Integration
Data
Warehouse
AWS Cloud
Business
Intelligence
36. Data warehousing done the AWS way
Deploy
• Easy to provision
• Pay as you go, no up front costs
• Fast, cheap, easy to use
• SQL
37. Customer quotes
“Queries that used to take hours came back in seconds. Our analysts
are orders of magnitude more productive.”
“Redshift is twenty times faster than Hive…The cost saving is even
more impressive…Our analysts like [it] so much they don’t want to go
back.”
“[Amazon Redshift] took an industry famous for its opaque pricing,
high TCO and unreliable results and completely turned it on its head.”
“Team played with Redshift today and concluded it is awesome. Unindexed complex queries returning in < 10s.”
38. Amazon Redshift lets you start small and grow big
Extra Large Node (HS1.XL)
Eight Extra Large Node (HS1.8XL)
3 spindles, 2 TB, 16 GB RAM, 2 cores
24 spindles, 16 TB, 128 GB RAM, 16 cores, 10 GigE
Single Node (2 TB)
Cluster 2-100 Nodes (32 TB – 1.6 PB)
Cluster 2-32 Nodes (4 TB – 64 TB)
Note: Nodes not to scale
39. Amazon Redshift Pricing – Singapore & Sydney
Price Per Hour for
XL Node ($US)
On-Demand
$ 1.25
1 Year Reservation
$ 0.75
3 Year Reservation
$ 0.45
Simple Pricing
Number of Nodes x Cost per Hour
No charge for Leader Node
Pay as you go
40. So for example…….
•
1 XL node reserved for 3 years:
= 0.45c x number of hours in a month
=
$340 per month
• 1 XL node cluster gives you:
• 2 Cores
• 16 GB RAM
• 2 TB Disk
• Plus 2 TB storage in S3 for backups & snapshots
41. Amazon Redshift is easy to use
•
Provision in minutes
•
Monitor query performance
•
Point and click resize
•
Built in security
•
Automatic backups
42. Use cases
• Reporting Data-warehouse behind an OLTP system
• Data Mart to take load off the existing data warehouse
• Log file analysis for clickstream or gaming data (e.g.
Advertising, Retail, Gaming)
• Query-able archive for data compliance (e.g. Telco - Call
detail Records)
• Machine generated sensor data analysis (e.g. Utility smart meters, Resources - equipment failure prediction)
• As a data analytics system for live data (Gaming,
Advertising)
43. Flexibility & choice are key in the Cloud
Amazon Partner Network
(Technology Partners)
Deployment & Administration
Application Services
Compute
Storage
Database
Networking
AWS Global Infrastructure
47. Informatica:
The Industry Leader in Cloud Integration
#1 by Customer Count
2000+ companies
#1 by Customers/Analysts
AppExchange
Gartner
#1 by Data Processed
+40B transactions/month
#1 by Connectivity
Informatica Cloud Marketplace
52. Cloud Integration Customer Success Stories
Data Migration
App Integration
Consolidated Smith
Barney and Morgan
Stanley data on
Day 1
of merger
Synchronizing
Salesforce CRM
with Netsuite and
other business apps
Managers didn’t
lose momentum in
ongoing recruiting
efforts
1.5M rows of data
synchronized daily
iPaaS *(Build)
Extend
PowerCenter
Decreased
operational issues
from 70% to 30%
of IT workload
Reduce time to
build and distribute
connectivity to 3rd
party data sources
Enabled faster, more
accurate decisionmaking based on
timely, trusted data
Customize cloud
integration
templates to execute
sophisticated
integration workflows
Hybrid deployment
gives integration
flexibility and
scalability to meet
various use cases
Data Replication
Lowered time and
resources needed for
integrations by 80%
53. Informatica Cloud
The Industry’s Most Comprehensive Cloud Integration
and Data Management Solution
Cloud Process Automation
Guiding users to work efficiently with the data
Cloud Data Quality and MDM
Delivering the “Single Customer View”
Cloud Integration
Connecting your cloud apps
57. Challenges with Traditional Approaches to Cloud Integration
Mainframe based
Integration
Prism
ETI
Client / Server based
Integration
Cloud based
Integration
58. Move to the Cloud…
IT transitions from skeptic to partner to driver
Cloud First
(IT Led)
Increasing IT
involvement
in Cloud
decision
making
Business-IT
Collaboration
LOB Led
(IT Approved)
LOB Owned
(Outside of IT)
2012-2013
Pre-2010
2010-2012
2013
59. Cloud is the Reality in the Enterprise
Large, Accelerating Market
4-6x
growth rate of
on-premise IT
20-27% CAGR
$20-40B market
SaaS
largest category
PaaS
fastest growing
(Forrester)
Led by Large
Enterprises
76%
enterprises
have a formal
cloud strategy
(Forrester)
(Forrester, IDC, Gartner, 451Group)
Driven by IT
90%
Cloud decisions
and operations
involve IT
(IDC)
60%
84%
of all companies
using SaaS w/in 12
months
of net new
software is
now SaaS
(Forrester)
(IDC)
74%
using cloud
will increase cloud
spend
> 20%
(IDC)
66%
SaaS POs
signed by IT
(IDC)
60. Informatica Cloud and Amazon Redshift:
Enabling cost-effective data warehousing
•
•
Redshift Connector pre-release announced in February
General availability in August 2013
InformaticaCloud.com/Amazon-Redshift
61. What did it use to take…
•
•
•
•
•
•
Budget large capital expenditure
Schedule a sales meeting with Oracle, IBM, Teradata, etc…
Formal POC (Proof of Concept)
Procure software and hardware
Install and setup
Start project
62. What it takes now…
•
•
Go to the web and sign-up
Start project!
64. Informatica Cloud Amazon Redshift demonstration
6
Metadata Mappings
4
5
1
Firewall
1
Build mapping and execute job
2
Retrieve Account Data
3
Put Account Data into Flat File
4
Transfer compressed Flat File to S3
5
Initiate copy from S3
6
Load data into Amazon Redshift
3
Informatica Cloud
Secure Agent
2
65. Best practices to remember…
•
The Amazon S3 bucket that holds the data files must be created in the same
region as your cluster
– Files are deleted from Amazon S3 bucket when upload is complete
•
Choose a batch size where the number of batches matches the number of
slices in your cluster
– Each XL node has 2 slices, each 8XL node has 16
– If you have a 2 node XL cluster and 40,000 rows of data, choose a batch size of
10,000
– The Informatica Cloud Redshift connector can maximize Amazon’s parallel
processing capabilities this way
66. Next Steps
•
Get started with Amazon Redshift
•
Get started with Informatica Cloud
– InformaticaCloud.com
•
Learn more about our Redshift Connector
– InformaticaCloud.com/Amazon-Redshift
85. Jaspersoft: The Intelligence Inside
Embeddable Architecture
Cloud Ready
Open web standard
architecture makes
integration with any
app easy to perform
Multi-tenant architecture,
100’s of SaaS
customers, top selling BI
solution on Amazon
Full Self-Service BI Suite
Address all user requirements with
interactive reports, dashboards,
analysis, and data integration
Affordable
Proven Platform
Up to 80% less than
traditional BI platforms
while delivering significant
power & capabilities
Millions of users,
380,000 community
members, deployed in
130,000+ applications
93. … with a World-Class BI Platform
Reporting, Dashboards, Visualization, OLAP
Analysis
Columnar-Based In-Memory Engine
Business Metadata Layer
Data
Integration
Data
Virtualization
Direct
Extensive APIs: HTTP, SOAP, REST
100% Web Standards: CSS, .JS, .JSP, Java
HTML5 Browser, Native Mobile Apps
Data Connectivity to Any Data
RDS
Redshift
EMR
SaaS
On-Premises
94
100. Jaspersoft Pro on AWS
•
Jaspersoft is the first BI service that you can buy per hour
– No user limitations, no monthly fee,
– less than $1 per hour
•
First BI service to automatically
connect to your AWS data
– 10 minutes from launch to visualizing your data in RDS or Redshift
– AWS Security Integration
•
Released February, 2013
– Over 500 customers
101