SlideShare a Scribd company logo
New Features and End to End Lineage
Paige Roberts, Big Data Product Marketing Manager
Ron Cichoski, Sr. Solutions Architect
March 2018
Agenda
Product News
– Recent improvements - in DMX/DMX-h versions 9.5, 9.6, and since
– About to Release – Early releases and testing now, GA in April
– Coming Soon – Features on the drawing board for June or later
End to End Data Lineage
Demo of Lineage with Cloudera Navigator
Where to Find More Info
2Syncsort Confidential and Proprietary - do not copy or distribute
Disclaimer
3
Syncsort Confidential and Proprietary - do not copy or distribute
Any information about our roadmap outlines our general product direction and is subject to
change at any time without notice. It is for informational purposes only and shall not, be
incorporated into any contract or other commitment.
Syncsort undertakes no obligation either to develop the features or functionality described
or to include any such feature or functionality in a future release.
RECENT IMPROVEMENTS
Product News
Syncsort Confidential and Proprietary - do not copy or distribute
4
Combine batch and streaming data sources
Single Interface for Streaming & Batch
Spark 2!
Easy development in GUI No need
to write Scala, C or Java code
Now supports cluster mode!
5
Syncsort Confidential and Proprietary - do not copy or distribute
Simplify Streaming Data Integration
Syncsort Confidential and Proprietary - do not copy or distribute 5
Progress Monitoring
Track the progress of
DMX/DMX-h jobs as
they’re running!
Settable time intervals
See exactly how fast jobs are running
Know how much memory and CPU jobs
use at any point
Know when there’s a problem, even in
the middle of long-running jobs
6Syncsort Confidential and Proprietary - do not copy or distribute
C:PROGRAM FILESDMEXPRESSPROGRAMSdmsmonitor.exe /jobid J_readVSAM_20171006_001743_13572 /task
T_readVSAM /interactive 2 /logdir .
Timestamp: 2017-10-06 00:19:09
Status: RUNNING for 00:01:28
User: aramachandran
Data directory: C:UsersaramachandranDocumentsProjectsCompanyNameVSAM_test
Memory: 32MB
CPU: 12%
/MVS/WWCDMX/AZR.VSM (Source): 7689557 records [1689372 records/sec], 246065824 bytes [5405992 bytes/sec]
Vsam_out.dat (Target): 7685704 records [1687590 records/sec], 245942528 bytes [54002880 bytes/sec]
C:PROGRAM FILESDMEXPRESSPROGRAMSdmsmonitor.exe /jobid J_readVSAM_20171006_001743_13572 /task
T_readVSAM /interactive 2 /logdir .
Timestamp: 2017-10-06 00:19:11
Status: RUNNING for 00:01:30
User: aramachandran
Data directory: C:UsersaramachandranDocumentsProjectsCompanyNameVSAM_test
Memory: 32MB
CPU: 12%
/MVS/WWCDMX/AZR.VSM (Source): 10718776 records [1514609 records/sec], 343000832 bytes [48467504 bytes/sec]
Vsam_out.dat (Target): 10716748 records [1515522 records/sec], 342935936 bytes [48496704 bytes/sec]
Keybreak Processing Made Easy
Running Totals
Counters
Group Numbering
7Syncsort Confidential and Proprietary - do not copy or distribute
Hive Enhancements
Improvements to Hive support
JDBC connectivity
Support for partitioned tables: ORC, Parquet, AVRO, HDFS
Support for Truncate and Insert
Automatic creation of Hive and other Hcat supported tables
Direct distributed processing of Hive
Update of Hive statistics
Use Hive tables for lookups
Hive Merge support – Updates, inserts, deletes, and upserts in Hive
8Syncsort Confidential and Proprietary - do not copy or distribute
DB2 on i Series
9Syncsort Confidential and Proprietary - do not copy or distribute
EARLY RELEASE READY
Product News
Syncsort Confidential and Proprietary - do not copy or distribute
10
Impala Support
11Syncsort Confidential and Proprietary - do not copy or distribute
Syncsort Real-Time Change Data Capture
12Syncsort Confidential and Proprietary - do not copy or distribute
Keep data in sync in real-time:
Without overloading networks.
Without affecting source database performance.
Without coding or tuning.
• HDFS
• Hive
• Impala
• IBM DB2
• IBM Informix
• Oracle
• Oracle RAC
• Sybase
• MS SQL Server
• Teradata
• MySQL
• PostgreSQL
• IBM DB2
• VSAM
• IBM Informix
• Oracle
• Oracle RAC
• Sybase
• MS SQL Server
Dependable – Reliable transfer of
data even if connectivity fails on
either side.
Fast – Captures changes in source
as they happen. Updates table
statistics for faster queries.
Flexible – Writes to most major
RDBMSs, as well as HDFS, all Hive
and Impala tables, including those
backed by text, ORC, Parquet, Avro
or Kudu.
Even updates Hive versions that
don’t support updates.
Real-Time Replication with Transformation
Conflict Resolution, Collision Monitoring, Tracking and Auditing
Govern and Track Everything for Compliance
• Metadata and data lineage for Hive, Avro and
Parquet through HCatalog
• Metadata lineage export and API from DMX/DMX-h
– Simplify audits, analytics dashboards, metrics
– Integrate with enterprise metadata repositories
• Cloudera Navigator certified integration
– Audit and track data from source to cluster
– HDFS, YARN, Spark and other metadata
– Lineage, tagging
– Business and structural metadata
• Apache Atlas ingestion lineage integration
– Audit and track data from source to cluster
– Lineage, tagging
Syncsort Confidential and Proprietary - do not copy or distribute 13
Get Your Database data into Hadoop, At the Press of a Button
• Funnel hundreds of tables at once into your data lake
‒ Extract, map and move whole DB schemas in one invocation
‒ Extract from Oracle, DB2/z, MS SQL Server, Teradata, Netezza and Redshift
‒ To SQL Server, Postgres, Hive, HDFS, Redshift and Amazon S3
‒ Automatically create target Hive and HCat tables
• Process multiple funnels in parallel on edge node or data nodes
‒ Order data flows by dependencies
‒ Leverage DMX-h high performance data processing engine
• Extract only the data you want
‒ Data type filtering
‒ Table, record or column exclusion / inclusion
• In-flight transformations and cleansing
• User specified access methods: Native, ODBC or JDBC
14
Syncsort Confidential and Proprietary - do not copy or distribute
DataFunnel™
Move thousands of tables at once!
Syncsort Confidential and Proprietary - do not copy or distribute 14
New User Experience for DataFunnel
15Syncsort Confidential and Proprietary - do not copy or distribute
DMX
DataFunnel™
Syncsort Confidential and Proprietary - do not copy or distribute 15
New User Experience for DataFunnel
16Syncsort Confidential and Proprietary - do not copy or distribute
DMX
DataFunnel™
Syncsort Confidential and Proprietary - do not copy or distribute 16
New UI Wizard Flow Creation
17Syncsort Confidential and Proprietary - do not copy or distribute
DMX
DataFunnel™
Syncsort Confidential and Proprietary - do not copy or distribute 17
New UI Wizard Flow Creation
18Syncsort Confidential and Proprietary - do not copy or distribute
DMX
DataFunnel™
Syncsort Confidential and Proprietary - do not copy or distribute 18
GUI Execution
19Syncsort Confidential and Proprietary - do not copy or distribute
COMING SOON
Product News
Syncsort Confidential and Proprietary - do not copy or distribute
20
Change Data Capture in DataFunnel
21Syncsort Confidential and Proprietary - do not copy or distribute
Keep data in sync in real-time:
Without overloading networks.
Without affecting source database performance.
Without coding or tuning.
• HDFS
• Hive
• Impala
• IBM DB2
• IBM Informix
• Oracle
• Oracle RAC
• Sybase
• MS SQL Server
• Teradata
• MySQL
• PostgreSQL
• IBM DB2
• VSAM
• IBM Informix
• Oracle
• Oracle RAC
• Sybase
• MS SQL Server
Dependable – Reliable transfer of
data even if connectivity fails on
either side.
Fast – Captures changes in source
as they happen. Updates table
statistics for faster queries.
Flexible – Writes to most major
RDBMSs, as well as HDFS, all Hive
and Impala tables, including those
backed by text, ORC, Parquet, Avro
or Kudu.
Even updates Hive versions that
don’t support updates.
Real-Time Replication with Transformation
Conflict Resolution, Collision Monitoring, Tracking and Auditing
Job Monitoring in DataFunnel GUI
22Syncsort Confidential and Proprietary - do not copy or distribute
DMX
DataFunnel™
Syncsort Confidential and Proprietary - do not copy or distribute 22
END TO END DATA LINEAGE
23Syncsort Confidential and Proprietary - do not copy or distribute
End to End Data Lineage in Cloudera Navigator
24Syncsort Confidential and Proprietary - do not copy or distribute
Data Sources
End to End Data Lineage in Cloudera Navigator
25Syncsort Confidential and Proprietary - do not copy or distribute
Data Sources
Syncsort accesses
data from
sources outside
cluster.
End to End Data Lineage in Cloudera Navigator
26Syncsort Confidential and Proprietary - do not copy or distribute
Syncsort onboards
data, modifies
on-the-fly to match
Hadoop storage
model.
Data Sources
Syncsort accesses
data from
sources outside
cluster.
End to End Data Lineage in Cloudera Navigator
27Syncsort Confidential and Proprietary - do not copy or distribute
Syncsort onboards
data, modifies
on-the-fly to match
Hadoop storage
model.
Data Sources
Syncsort accesses
data from
sources outside
cluster.
Syncsort changes,
enhances, joins
data in cluster with
MapReduce or
Spark.
Data Hub
End to End Data Lineage in Cloudera Navigator
28Syncsort Confidential and Proprietary - do not copy or distribute
Syncsort onboards
data, modifies
on-the-fly to match
Hadoop storage
model.
Data Sources
Syncsort accesses
data from
sources outside
cluster.
Syncsort changes,
enhances, joins
data in cluster with
MapReduce or
Spark.
Syncsort passes
source-to-
cluster data
lineage info to
Navigator.
Data Hub
End to End Data Lineage in Cloudera Navigator
29Syncsort Confidential and Proprietary - do not copy or distribute
Syncsort onboards
data, modifies
on-the-fly to match
Hadoop storage
model.
Data Sources
Syncsort accesses
data from
sources outside
cluster.
Syncsort changes,
enhances, joins
data in cluster with
MapReduce or
Spark.
Navigator gathers
any other changes
made to data on
cluster.
Syncsort passes
source-to-
cluster data
lineage info to
Navigator.
Data changes made
by MapReduce,
Spark, HiveQL.
Data Hub
End to End Data Lineage in Cloudera Navigator
30Syncsort Confidential and Proprietary - do not copy or distribute
Syncsort onboards
data, modifies
on-the-fly to match
Hadoop storage
model.
Data Sources
Syncsort accesses
data from
sources outside
cluster.
Syncsort changes,
enhances, joins
data in cluster with
MapReduce or
Spark.
Analytics and
visualizations get
complete data.
Navigator gathers
any other changes
made to data on
cluster.
Data analyst
gets end-to-end
data lineage
info from
Navigator.
Syncsort passes
source-to-
cluster data
lineage info to
Navigator.
Data changes made
by MapReduce,
Spark, HiveQL.
Data Hub
Analytics,
Visualization
Syncsort DMX-h + Cloudera Navigator for End-to-End Lineage
31Syncsort Confidential and Proprietary - do not copy or distribute
End to End Data Lineage Outside the Cluster
32Syncsort Confidential and Proprietary - do not copy or distribute
Data Sources
Data Hub
Analytics,
Visualization
Data Lineage
Other
Syncsort onboards
data, modifies
on-the-fly to match
Hadoop storage
model.
Syncsort accesses
data from
sources outside
cluster.
Syncsort changes,
enhances, joins
data in cluster with
MapReduce or
Spark.
Analytics and
visualizations get
complete data.
MDM solutions like
ASG, Collibra, etc.
gather the lineage
from the API.
Syncsort passes
source-to-
cluster data
lineage info to a
REST API.
Data analyst
gets end-to-end
data lineage
info from
Navigator.
REST API
Data Lineage + Data Quality = Foundations of Data Governance
33Syncsort Confidential and Proprietary - do not copy or distribute
Discovery
and
Profiling
Data Sources
Multi-field fuzzy matching, de-duplication,
cleansing, enrichment, standardization,
business rule enforcement.
Analytics and
visualizations on
clean, complete data
you can trust.
Data Hub
Analytics,
Visualization
Data Lineage
DEMO OF CLOUDERA NAVIGATOR LINEAGE
34Syncsort Confidential and Proprietary - do not copy or distribute
What Next?
35Syncsort Confidential and Proprietary - do not copy or distribute
Contact Syncsort sales to get the latest info: http://www.syncsort.com/en/ContactSales
If you would like to start using the new features in the DataFunnel UI, and provide feedback,
contact Product Manager, Ashwin Ramachandran - ARamachandran@Syncsort.com
Questions
36
THANK YOU

More Related Content

What's hot

Handling Kernel Upgrades at Scale - The Dirty Cow Story
Handling Kernel Upgrades at Scale - The Dirty Cow StoryHandling Kernel Upgrades at Scale - The Dirty Cow Story
Handling Kernel Upgrades at Scale - The Dirty Cow Story
DataWorks Summit
 
Akka 2.4 plus commercial features in Typesafe Reactive Platform
Akka 2.4 plus commercial features in Typesafe Reactive PlatformAkka 2.4 plus commercial features in Typesafe Reactive Platform
Akka 2.4 plus commercial features in Typesafe Reactive Platform
Legacy Typesafe (now Lightbend)
 
Oracle Drivers configuration for High Availability
Oracle Drivers configuration for High AvailabilityOracle Drivers configuration for High Availability
Oracle Drivers configuration for High Availability
Ludovico Caldara
 
The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017
The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017
The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017
Luciano Resende
 
Extending The Yahoo Streaming Benchmark to Apache Apex
Extending The Yahoo Streaming Benchmark to Apache ApexExtending The Yahoo Streaming Benchmark to Apache Apex
Extending The Yahoo Streaming Benchmark to Apache Apex
Apache Apex
 
Rapid Home Provisioning
Rapid Home ProvisioningRapid Home Provisioning
Rapid Home Provisioning
Ludovico Caldara
 
Oracle Active Data Guard and Global Data Services in Action!
Oracle Active Data Guard and Global Data Services in Action!Oracle Active Data Guard and Global Data Services in Action!
Oracle Active Data Guard and Global Data Services in Action!
Ludovico Caldara
 
Release and patching strategy
Release and patching strategyRelease and patching strategy
Release and patching strategy
Jitendra Singh
 
Migrating pipelines into Docker
Migrating pipelines into DockerMigrating pipelines into Docker
Migrating pipelines into Docker
DataWorks Summit/Hadoop Summit
 
Beyond the Brokers | Emma Humber and Andrew Borley, IBM
Beyond the Brokers | Emma Humber and Andrew Borley, IBMBeyond the Brokers | Emma Humber and Andrew Borley, IBM
Beyond the Brokers | Emma Humber and Andrew Borley, IBM
HostedbyConfluent
 
Boost your Oracle RAC manageability with Policy-Managed Databases
Boost your Oracle RAC manageability with Policy-Managed DatabasesBoost your Oracle RAC manageability with Policy-Managed Databases
Boost your Oracle RAC manageability with Policy-Managed Databases
Ludovico Caldara
 
Slide 1 - Parallels Plesk Control Panel 8.6.0
Slide 1 - Parallels Plesk Control Panel 8.6.0Slide 1 - Parallels Plesk Control Panel 8.6.0
Slide 1 - Parallels Plesk Control Panel 8.6.0
webhostingguy
 
Oracle GoldenGate Architecture Performance
Oracle GoldenGate Architecture PerformanceOracle GoldenGate Architecture Performance
Oracle GoldenGate Architecture Performance
Enkitec
 
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
alanfgates
 
Become a MySQL DBA: performing live database upgrades - webinar slides
Become a MySQL DBA: performing live database upgrades - webinar slidesBecome a MySQL DBA: performing live database upgrades - webinar slides
Become a MySQL DBA: performing live database upgrades - webinar slides
Severalnines
 
C19013010 the tutorial to build shared ai services session 2
C19013010 the tutorial to build shared ai services session 2C19013010 the tutorial to build shared ai services session 2
C19013010 the tutorial to build shared ai services session 2
Bill Liu
 
How InfluxDB Enables NodeSource to Run Extreme Levels of Node.js Processes
How InfluxDB Enables NodeSource to Run Extreme Levels of Node.js ProcessesHow InfluxDB Enables NodeSource to Run Extreme Levels of Node.js Processes
How InfluxDB Enables NodeSource to Run Extreme Levels of Node.js Processes
InfluxData
 
Webinar slides: Managing MySQL Replication for High Availability
Webinar slides: Managing MySQL Replication for High AvailabilityWebinar slides: Managing MySQL Replication for High Availability
Webinar slides: Managing MySQL Replication for High Availability
Severalnines
 
Oracle GoldenGate 12c CDR Presentation for ECO
Oracle GoldenGate 12c CDR Presentation for ECOOracle GoldenGate 12c CDR Presentation for ECO
Oracle GoldenGate 12c CDR Presentation for ECO
Bobby Curtis
 
Migration to Oracle Multitenant
Migration to Oracle MultitenantMigration to Oracle Multitenant
Migration to Oracle Multitenant
Jitendra Singh
 

What's hot (20)

Handling Kernel Upgrades at Scale - The Dirty Cow Story
Handling Kernel Upgrades at Scale - The Dirty Cow StoryHandling Kernel Upgrades at Scale - The Dirty Cow Story
Handling Kernel Upgrades at Scale - The Dirty Cow Story
 
Akka 2.4 plus commercial features in Typesafe Reactive Platform
Akka 2.4 plus commercial features in Typesafe Reactive PlatformAkka 2.4 plus commercial features in Typesafe Reactive Platform
Akka 2.4 plus commercial features in Typesafe Reactive Platform
 
Oracle Drivers configuration for High Availability
Oracle Drivers configuration for High AvailabilityOracle Drivers configuration for High Availability
Oracle Drivers configuration for High Availability
 
The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017
The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017
The Analytic Platform behind IBM’s Watson Data Platform - Big Data Spain 2017
 
Extending The Yahoo Streaming Benchmark to Apache Apex
Extending The Yahoo Streaming Benchmark to Apache ApexExtending The Yahoo Streaming Benchmark to Apache Apex
Extending The Yahoo Streaming Benchmark to Apache Apex
 
Rapid Home Provisioning
Rapid Home ProvisioningRapid Home Provisioning
Rapid Home Provisioning
 
Oracle Active Data Guard and Global Data Services in Action!
Oracle Active Data Guard and Global Data Services in Action!Oracle Active Data Guard and Global Data Services in Action!
Oracle Active Data Guard and Global Data Services in Action!
 
Release and patching strategy
Release and patching strategyRelease and patching strategy
Release and patching strategy
 
Migrating pipelines into Docker
Migrating pipelines into DockerMigrating pipelines into Docker
Migrating pipelines into Docker
 
Beyond the Brokers | Emma Humber and Andrew Borley, IBM
Beyond the Brokers | Emma Humber and Andrew Borley, IBMBeyond the Brokers | Emma Humber and Andrew Borley, IBM
Beyond the Brokers | Emma Humber and Andrew Borley, IBM
 
Boost your Oracle RAC manageability with Policy-Managed Databases
Boost your Oracle RAC manageability with Policy-Managed DatabasesBoost your Oracle RAC manageability with Policy-Managed Databases
Boost your Oracle RAC manageability with Policy-Managed Databases
 
Slide 1 - Parallels Plesk Control Panel 8.6.0
Slide 1 - Parallels Plesk Control Panel 8.6.0Slide 1 - Parallels Plesk Control Panel 8.6.0
Slide 1 - Parallels Plesk Control Panel 8.6.0
 
Oracle GoldenGate Architecture Performance
Oracle GoldenGate Architecture PerformanceOracle GoldenGate Architecture Performance
Oracle GoldenGate Architecture Performance
 
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
Hive2.0 sql speed-scale--hadoop-summit-dublin-apr-2016
 
Become a MySQL DBA: performing live database upgrades - webinar slides
Become a MySQL DBA: performing live database upgrades - webinar slidesBecome a MySQL DBA: performing live database upgrades - webinar slides
Become a MySQL DBA: performing live database upgrades - webinar slides
 
C19013010 the tutorial to build shared ai services session 2
C19013010 the tutorial to build shared ai services session 2C19013010 the tutorial to build shared ai services session 2
C19013010 the tutorial to build shared ai services session 2
 
How InfluxDB Enables NodeSource to Run Extreme Levels of Node.js Processes
How InfluxDB Enables NodeSource to Run Extreme Levels of Node.js ProcessesHow InfluxDB Enables NodeSource to Run Extreme Levels of Node.js Processes
How InfluxDB Enables NodeSource to Run Extreme Levels of Node.js Processes
 
Webinar slides: Managing MySQL Replication for High Availability
Webinar slides: Managing MySQL Replication for High AvailabilityWebinar slides: Managing MySQL Replication for High Availability
Webinar slides: Managing MySQL Replication for High Availability
 
Oracle GoldenGate 12c CDR Presentation for ECO
Oracle GoldenGate 12c CDR Presentation for ECOOracle GoldenGate 12c CDR Presentation for ECO
Oracle GoldenGate 12c CDR Presentation for ECO
 
Migration to Oracle Multitenant
Migration to Oracle MultitenantMigration to Oracle Multitenant
Migration to Oracle Multitenant
 

Similar to End-to-End, Source to Analytics, Data Lineage with Syncsort DMX-h

Keeping Data in Sync with Syncsort
Keeping Data in Sync with SyncsortKeeping Data in Sync with Syncsort
Keeping Data in Sync with Syncsort
Precisely
 
Big Data Q2 Customer Education Webcast: New DMX Change Data Capture for Hadoo...
Big Data Q2 Customer Education Webcast: New DMX Change Data Capture for Hadoo...Big Data Q2 Customer Education Webcast: New DMX Change Data Capture for Hadoo...
Big Data Q2 Customer Education Webcast: New DMX Change Data Capture for Hadoo...
Precisely
 
Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...
Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...
Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...
Precisely
 
The Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and StreamingThe Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and Streaming
Timothy Spann
 
Simplifying Big Data Integration with Syncsort DMX and DMX-h
Simplifying Big Data Integration with Syncsort DMX and DMX-hSimplifying Big Data Integration with Syncsort DMX and DMX-h
Simplifying Big Data Integration with Syncsort DMX and DMX-h
Precisely
 
PartnerSkillUp_Enable a Streaming CDC Solution
PartnerSkillUp_Enable a Streaming CDC SolutionPartnerSkillUp_Enable a Streaming CDC Solution
PartnerSkillUp_Enable a Streaming CDC Solution
Timothy Spann
 
What’s New in Syncsort Integrate? New User Experience for Fast Data Onboarding
What’s New in Syncsort Integrate? New User Experience for Fast Data OnboardingWhat’s New in Syncsort Integrate? New User Experience for Fast Data Onboarding
What’s New in Syncsort Integrate? New User Experience for Fast Data Onboarding
Precisely
 
Customer Education Webcast: New Features in Data Integration and Streaming CDC
Customer Education Webcast: New Features in Data Integration and Streaming CDCCustomer Education Webcast: New Features in Data Integration and Streaming CDC
Customer Education Webcast: New Features in Data Integration and Streaming CDC
Precisely
 
Apache Kafka® and the Data Mesh
Apache Kafka® and the Data MeshApache Kafka® and the Data Mesh
Apache Kafka® and the Data Mesh
ConfluentInc1
 
PCM Vision 2019 Breakout: Quest Software
PCM Vision 2019 Breakout: Quest SoftwarePCM Vision 2019 Breakout: Quest Software
PCM Vision 2019 Breakout: Quest Software
PCM
 
Get a "Farm to Table" View of Your Data: Tracking Data Quality and Lineage, o...
Get a "Farm to Table" View of Your Data: Tracking Data Quality and Lineage, o...Get a "Farm to Table" View of Your Data: Tracking Data Quality and Lineage, o...
Get a "Farm to Table" View of Your Data: Tracking Data Quality and Lineage, o...
Precisely
 
Unconference Round Table Notes
Unconference Round Table NotesUnconference Round Table Notes
Unconference Round Table Notes
Timothy Spann
 
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Anant Corporation
 
Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
Best Practices for Building Hybrid-Cloud Architectures | Hans JespersenBest Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
confluent
 
GSJUG: Mastering Data Streaming Pipelines 09May2023
GSJUG: Mastering Data Streaming Pipelines 09May2023GSJUG: Mastering Data Streaming Pipelines 09May2023
GSJUG: Mastering Data Streaming Pipelines 09May2023
Timothy Spann
 
Mastering the move
Mastering the moveMastering the move
Mastering the move
Trivadis
 
Why Hadoop is important to Syncsort
Why Hadoop is important to SyncsortWhy Hadoop is important to Syncsort
Why Hadoop is important to Syncsort
huguk
 
Journey to SAS Analytics Grid with SAS, R, Python
Journey to SAS Analytics Grid with SAS, R, PythonJourney to SAS Analytics Grid with SAS, R, Python
Journey to SAS Analytics Grid with SAS, R, Python
Sumit Sarkar
 
Simplifying and Future-Proofing Hadoop
Simplifying and Future-Proofing HadoopSimplifying and Future-Proofing Hadoop
Simplifying and Future-Proofing Hadoop
Precisely
 
What's New in Apache Spark 2.3 & Why Should You Care
What's New in Apache Spark 2.3 & Why Should You CareWhat's New in Apache Spark 2.3 & Why Should You Care
What's New in Apache Spark 2.3 & Why Should You Care
Databricks
 

Similar to End-to-End, Source to Analytics, Data Lineage with Syncsort DMX-h (20)

Keeping Data in Sync with Syncsort
Keeping Data in Sync with SyncsortKeeping Data in Sync with Syncsort
Keeping Data in Sync with Syncsort
 
Big Data Q2 Customer Education Webcast: New DMX Change Data Capture for Hadoo...
Big Data Q2 Customer Education Webcast: New DMX Change Data Capture for Hadoo...Big Data Q2 Customer Education Webcast: New DMX Change Data Capture for Hadoo...
Big Data Q2 Customer Education Webcast: New DMX Change Data Capture for Hadoo...
 
Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...
Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...
Big Data Customer Education Webcast: The Latest Advancements in Syncsort DMX ...
 
The Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and StreamingThe Never Landing Stream with HTAP and Streaming
The Never Landing Stream with HTAP and Streaming
 
Simplifying Big Data Integration with Syncsort DMX and DMX-h
Simplifying Big Data Integration with Syncsort DMX and DMX-hSimplifying Big Data Integration with Syncsort DMX and DMX-h
Simplifying Big Data Integration with Syncsort DMX and DMX-h
 
PartnerSkillUp_Enable a Streaming CDC Solution
PartnerSkillUp_Enable a Streaming CDC SolutionPartnerSkillUp_Enable a Streaming CDC Solution
PartnerSkillUp_Enable a Streaming CDC Solution
 
What’s New in Syncsort Integrate? New User Experience for Fast Data Onboarding
What’s New in Syncsort Integrate? New User Experience for Fast Data OnboardingWhat’s New in Syncsort Integrate? New User Experience for Fast Data Onboarding
What’s New in Syncsort Integrate? New User Experience for Fast Data Onboarding
 
Customer Education Webcast: New Features in Data Integration and Streaming CDC
Customer Education Webcast: New Features in Data Integration and Streaming CDCCustomer Education Webcast: New Features in Data Integration and Streaming CDC
Customer Education Webcast: New Features in Data Integration and Streaming CDC
 
Apache Kafka® and the Data Mesh
Apache Kafka® and the Data MeshApache Kafka® and the Data Mesh
Apache Kafka® and the Data Mesh
 
PCM Vision 2019 Breakout: Quest Software
PCM Vision 2019 Breakout: Quest SoftwarePCM Vision 2019 Breakout: Quest Software
PCM Vision 2019 Breakout: Quest Software
 
Get a "Farm to Table" View of Your Data: Tracking Data Quality and Lineage, o...
Get a "Farm to Table" View of Your Data: Tracking Data Quality and Lineage, o...Get a "Farm to Table" View of Your Data: Tracking Data Quality and Lineage, o...
Get a "Farm to Table" View of Your Data: Tracking Data Quality and Lineage, o...
 
Unconference Round Table Notes
Unconference Round Table NotesUnconference Round Table Notes
Unconference Round Table Notes
 
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
Data Engineer's Lunch #82: Automating Apache Cassandra Operations with Apache...
 
Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
Best Practices for Building Hybrid-Cloud Architectures | Hans JespersenBest Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
Best Practices for Building Hybrid-Cloud Architectures | Hans Jespersen
 
GSJUG: Mastering Data Streaming Pipelines 09May2023
GSJUG: Mastering Data Streaming Pipelines 09May2023GSJUG: Mastering Data Streaming Pipelines 09May2023
GSJUG: Mastering Data Streaming Pipelines 09May2023
 
Mastering the move
Mastering the moveMastering the move
Mastering the move
 
Why Hadoop is important to Syncsort
Why Hadoop is important to SyncsortWhy Hadoop is important to Syncsort
Why Hadoop is important to Syncsort
 
Journey to SAS Analytics Grid with SAS, R, Python
Journey to SAS Analytics Grid with SAS, R, PythonJourney to SAS Analytics Grid with SAS, R, Python
Journey to SAS Analytics Grid with SAS, R, Python
 
Simplifying and Future-Proofing Hadoop
Simplifying and Future-Proofing HadoopSimplifying and Future-Proofing Hadoop
Simplifying and Future-Proofing Hadoop
 
What's New in Apache Spark 2.3 & Why Should You Care
What's New in Apache Spark 2.3 & Why Should You CareWhat's New in Apache Spark 2.3 & Why Should You Care
What's New in Apache Spark 2.3 & Why Should You Care
 

More from Precisely

Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
信頼できるデータでESGイニシアチブを成功に導く方法.pdf How to drive success with ESG initiatives with...
信頼できるデータでESGイニシアチブを成功に導く方法.pdf How to drive success with ESG initiatives with...信頼できるデータでESGイニシアチブを成功に導く方法.pdf How to drive success with ESG initiatives with...
信頼できるデータでESGイニシアチブを成功に導く方法.pdf How to drive success with ESG initiatives with...
Precisely
 
AI-Ready Data - The Key to Transforming Projects into Production.pptx
AI-Ready Data - The Key to Transforming Projects into Production.pptxAI-Ready Data - The Key to Transforming Projects into Production.pptx
AI-Ready Data - The Key to Transforming Projects into Production.pptx
Precisely
 
Building a Multi-Layered Defense for Your IBM i Security
Building a Multi-Layered Defense for Your IBM i SecurityBuilding a Multi-Layered Defense for Your IBM i Security
Building a Multi-Layered Defense for Your IBM i Security
Precisely
 
Optimierte Daten und Prozesse mit KI / ML + SAP Fiori.pdf
Optimierte Daten und Prozesse mit KI / ML + SAP Fiori.pdfOptimierte Daten und Prozesse mit KI / ML + SAP Fiori.pdf
Optimierte Daten und Prozesse mit KI / ML + SAP Fiori.pdf
Precisely
 
Chaining, Looping, and Long Text for Script Development and Automation.pdf
Chaining, Looping, and Long Text for Script Development and Automation.pdfChaining, Looping, and Long Text for Script Development and Automation.pdf
Chaining, Looping, and Long Text for Script Development and Automation.pdf
Precisely
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Precisely
 
Navigating the Cloud: Best Practices for Successful Migration
Navigating the Cloud: Best Practices for Successful MigrationNavigating the Cloud: Best Practices for Successful Migration
Navigating the Cloud: Best Practices for Successful Migration
Precisely
 
Unlocking the Power of Your IBM i and Z Security Data with Google Chronicle
Unlocking the Power of Your IBM i and Z Security Data with Google ChronicleUnlocking the Power of Your IBM i and Z Security Data with Google Chronicle
Unlocking the Power of Your IBM i and Z Security Data with Google Chronicle
Precisely
 
How to Build Data Governance Programs That Last - A Business-First Approach.pdf
How to Build Data Governance Programs That Last - A Business-First Approach.pdfHow to Build Data Governance Programs That Last - A Business-First Approach.pdf
How to Build Data Governance Programs That Last - A Business-First Approach.pdf
Precisely
 
Zukuntssichere SAP Prozesse dank automatisierter Massendaten
Zukuntssichere SAP Prozesse dank automatisierter MassendatenZukuntssichere SAP Prozesse dank automatisierter Massendaten
Zukuntssichere SAP Prozesse dank automatisierter Massendaten
Precisely
 
Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power Systems
Precisely
 
Crucial Considerations for AI-ready Data.pdf
Crucial Considerations for AI-ready Data.pdfCrucial Considerations for AI-ready Data.pdf
Crucial Considerations for AI-ready Data.pdf
Precisely
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Precisely
 
Justifying Capacity Managment Webinar 4/10
Justifying Capacity Managment Webinar 4/10Justifying Capacity Managment Webinar 4/10
Justifying Capacity Managment Webinar 4/10
Precisely
 
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Precisely
 
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Precisely
 
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3fTestjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Precisely
 
Data Innovation Summit: Data Integrity Trends
Data Innovation Summit: Data Integrity TrendsData Innovation Summit: Data Integrity Trends
Data Innovation Summit: Data Integrity Trends
Precisely
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
Precisely
 

More from Precisely (20)

Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
信頼できるデータでESGイニシアチブを成功に導く方法.pdf How to drive success with ESG initiatives with...
信頼できるデータでESGイニシアチブを成功に導く方法.pdf How to drive success with ESG initiatives with...信頼できるデータでESGイニシアチブを成功に導く方法.pdf How to drive success with ESG initiatives with...
信頼できるデータでESGイニシアチブを成功に導く方法.pdf How to drive success with ESG initiatives with...
 
AI-Ready Data - The Key to Transforming Projects into Production.pptx
AI-Ready Data - The Key to Transforming Projects into Production.pptxAI-Ready Data - The Key to Transforming Projects into Production.pptx
AI-Ready Data - The Key to Transforming Projects into Production.pptx
 
Building a Multi-Layered Defense for Your IBM i Security
Building a Multi-Layered Defense for Your IBM i SecurityBuilding a Multi-Layered Defense for Your IBM i Security
Building a Multi-Layered Defense for Your IBM i Security
 
Optimierte Daten und Prozesse mit KI / ML + SAP Fiori.pdf
Optimierte Daten und Prozesse mit KI / ML + SAP Fiori.pdfOptimierte Daten und Prozesse mit KI / ML + SAP Fiori.pdf
Optimierte Daten und Prozesse mit KI / ML + SAP Fiori.pdf
 
Chaining, Looping, and Long Text for Script Development and Automation.pdf
Chaining, Looping, and Long Text for Script Development and Automation.pdfChaining, Looping, and Long Text for Script Development and Automation.pdf
Chaining, Looping, and Long Text for Script Development and Automation.pdf
 
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial IntelligenceRevolutionizing SAP® Processes with Automation and Artificial Intelligence
Revolutionizing SAP® Processes with Automation and Artificial Intelligence
 
Navigating the Cloud: Best Practices for Successful Migration
Navigating the Cloud: Best Practices for Successful MigrationNavigating the Cloud: Best Practices for Successful Migration
Navigating the Cloud: Best Practices for Successful Migration
 
Unlocking the Power of Your IBM i and Z Security Data with Google Chronicle
Unlocking the Power of Your IBM i and Z Security Data with Google ChronicleUnlocking the Power of Your IBM i and Z Security Data with Google Chronicle
Unlocking the Power of Your IBM i and Z Security Data with Google Chronicle
 
How to Build Data Governance Programs That Last - A Business-First Approach.pdf
How to Build Data Governance Programs That Last - A Business-First Approach.pdfHow to Build Data Governance Programs That Last - A Business-First Approach.pdf
How to Build Data Governance Programs That Last - A Business-First Approach.pdf
 
Zukuntssichere SAP Prozesse dank automatisierter Massendaten
Zukuntssichere SAP Prozesse dank automatisierter MassendatenZukuntssichere SAP Prozesse dank automatisierter Massendaten
Zukuntssichere SAP Prozesse dank automatisierter Massendaten
 
Unlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power SystemsUnlocking the Potential of the Cloud for IBM Power Systems
Unlocking the Potential of the Cloud for IBM Power Systems
 
Crucial Considerations for AI-ready Data.pdf
Crucial Considerations for AI-ready Data.pdfCrucial Considerations for AI-ready Data.pdf
Crucial Considerations for AI-ready Data.pdf
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
Justifying Capacity Managment Webinar 4/10
Justifying Capacity Managment Webinar 4/10Justifying Capacity Managment Webinar 4/10
Justifying Capacity Managment Webinar 4/10
 
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
Automate Studio Training: Materials Maintenance Tips for Efficiency and Ease ...
 
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
Leveraging Mainframe Data in Near Real Time to Unleash Innovation With Cloud:...
 
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3fTestjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
Testjrjnejrvnorno4rno3nrfnfjnrfnournfou3nfou3f
 
Data Innovation Summit: Data Integrity Trends
Data Innovation Summit: Data Integrity TrendsData Innovation Summit: Data Integrity Trends
Data Innovation Summit: Data Integrity Trends
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 

Recently uploaded

leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
alexjohnson7307
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Jeffrey Haguewood
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
LucaBarbaro3
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!
GDSC PJATK
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Safe Software
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
saastr
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
saastr
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
fredae14
 
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStrDeep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
saastr
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 

Recently uploaded (20)

leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
leewayhertz.com-AI in predictive maintenance Use cases technologies benefits ...
 
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
Letter and Document Automation for Bonterra Impact Management (fka Social Sol...
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Trusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process MiningTrusted Execution Environment for Decentralized Process Mining
Trusted Execution Environment for Decentralized Process Mining
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!Finale of the Year: Apply for Next One!
Finale of the Year: Apply for Next One!
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Driving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success StoryDriving Business Innovation: Latest Generative AI Advancements & Success Story
Driving Business Innovation: Latest Generative AI Advancements & Success Story
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
Overcoming the PLG Trap: Lessons from Canva's Head of Sales & Head of EMEA Da...
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
Deep Dive: AI-Powered Marketing to Get More Leads and Customers with HyperGro...
 
Recommendation System using RAG Architecture
Recommendation System using RAG ArchitectureRecommendation System using RAG Architecture
Recommendation System using RAG Architecture
 
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStrDeep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
Deep Dive: Getting Funded with Jason Jason Lemkin Founder & CEO @ SaaStr
 
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 

End-to-End, Source to Analytics, Data Lineage with Syncsort DMX-h

  • 1. New Features and End to End Lineage Paige Roberts, Big Data Product Marketing Manager Ron Cichoski, Sr. Solutions Architect March 2018
  • 2. Agenda Product News – Recent improvements - in DMX/DMX-h versions 9.5, 9.6, and since – About to Release – Early releases and testing now, GA in April – Coming Soon – Features on the drawing board for June or later End to End Data Lineage Demo of Lineage with Cloudera Navigator Where to Find More Info 2Syncsort Confidential and Proprietary - do not copy or distribute
  • 3. Disclaimer 3 Syncsort Confidential and Proprietary - do not copy or distribute Any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not, be incorporated into any contract or other commitment. Syncsort undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release.
  • 4. RECENT IMPROVEMENTS Product News Syncsort Confidential and Proprietary - do not copy or distribute 4
  • 5. Combine batch and streaming data sources Single Interface for Streaming & Batch Spark 2! Easy development in GUI No need to write Scala, C or Java code Now supports cluster mode! 5 Syncsort Confidential and Proprietary - do not copy or distribute Simplify Streaming Data Integration Syncsort Confidential and Proprietary - do not copy or distribute 5
  • 6. Progress Monitoring Track the progress of DMX/DMX-h jobs as they’re running! Settable time intervals See exactly how fast jobs are running Know how much memory and CPU jobs use at any point Know when there’s a problem, even in the middle of long-running jobs 6Syncsort Confidential and Proprietary - do not copy or distribute C:PROGRAM FILESDMEXPRESSPROGRAMSdmsmonitor.exe /jobid J_readVSAM_20171006_001743_13572 /task T_readVSAM /interactive 2 /logdir . Timestamp: 2017-10-06 00:19:09 Status: RUNNING for 00:01:28 User: aramachandran Data directory: C:UsersaramachandranDocumentsProjectsCompanyNameVSAM_test Memory: 32MB CPU: 12% /MVS/WWCDMX/AZR.VSM (Source): 7689557 records [1689372 records/sec], 246065824 bytes [5405992 bytes/sec] Vsam_out.dat (Target): 7685704 records [1687590 records/sec], 245942528 bytes [54002880 bytes/sec] C:PROGRAM FILESDMEXPRESSPROGRAMSdmsmonitor.exe /jobid J_readVSAM_20171006_001743_13572 /task T_readVSAM /interactive 2 /logdir . Timestamp: 2017-10-06 00:19:11 Status: RUNNING for 00:01:30 User: aramachandran Data directory: C:UsersaramachandranDocumentsProjectsCompanyNameVSAM_test Memory: 32MB CPU: 12% /MVS/WWCDMX/AZR.VSM (Source): 10718776 records [1514609 records/sec], 343000832 bytes [48467504 bytes/sec] Vsam_out.dat (Target): 10716748 records [1515522 records/sec], 342935936 bytes [48496704 bytes/sec]
  • 7. Keybreak Processing Made Easy Running Totals Counters Group Numbering 7Syncsort Confidential and Proprietary - do not copy or distribute
  • 8. Hive Enhancements Improvements to Hive support JDBC connectivity Support for partitioned tables: ORC, Parquet, AVRO, HDFS Support for Truncate and Insert Automatic creation of Hive and other Hcat supported tables Direct distributed processing of Hive Update of Hive statistics Use Hive tables for lookups Hive Merge support – Updates, inserts, deletes, and upserts in Hive 8Syncsort Confidential and Proprietary - do not copy or distribute
  • 9. DB2 on i Series 9Syncsort Confidential and Proprietary - do not copy or distribute
  • 10. EARLY RELEASE READY Product News Syncsort Confidential and Proprietary - do not copy or distribute 10
  • 11. Impala Support 11Syncsort Confidential and Proprietary - do not copy or distribute
  • 12. Syncsort Real-Time Change Data Capture 12Syncsort Confidential and Proprietary - do not copy or distribute Keep data in sync in real-time: Without overloading networks. Without affecting source database performance. Without coding or tuning. • HDFS • Hive • Impala • IBM DB2 • IBM Informix • Oracle • Oracle RAC • Sybase • MS SQL Server • Teradata • MySQL • PostgreSQL • IBM DB2 • VSAM • IBM Informix • Oracle • Oracle RAC • Sybase • MS SQL Server Dependable – Reliable transfer of data even if connectivity fails on either side. Fast – Captures changes in source as they happen. Updates table statistics for faster queries. Flexible – Writes to most major RDBMSs, as well as HDFS, all Hive and Impala tables, including those backed by text, ORC, Parquet, Avro or Kudu. Even updates Hive versions that don’t support updates. Real-Time Replication with Transformation Conflict Resolution, Collision Monitoring, Tracking and Auditing
  • 13. Govern and Track Everything for Compliance • Metadata and data lineage for Hive, Avro and Parquet through HCatalog • Metadata lineage export and API from DMX/DMX-h – Simplify audits, analytics dashboards, metrics – Integrate with enterprise metadata repositories • Cloudera Navigator certified integration – Audit and track data from source to cluster – HDFS, YARN, Spark and other metadata – Lineage, tagging – Business and structural metadata • Apache Atlas ingestion lineage integration – Audit and track data from source to cluster – Lineage, tagging Syncsort Confidential and Proprietary - do not copy or distribute 13
  • 14. Get Your Database data into Hadoop, At the Press of a Button • Funnel hundreds of tables at once into your data lake ‒ Extract, map and move whole DB schemas in one invocation ‒ Extract from Oracle, DB2/z, MS SQL Server, Teradata, Netezza and Redshift ‒ To SQL Server, Postgres, Hive, HDFS, Redshift and Amazon S3 ‒ Automatically create target Hive and HCat tables • Process multiple funnels in parallel on edge node or data nodes ‒ Order data flows by dependencies ‒ Leverage DMX-h high performance data processing engine • Extract only the data you want ‒ Data type filtering ‒ Table, record or column exclusion / inclusion • In-flight transformations and cleansing • User specified access methods: Native, ODBC or JDBC 14 Syncsort Confidential and Proprietary - do not copy or distribute DataFunnel™ Move thousands of tables at once! Syncsort Confidential and Proprietary - do not copy or distribute 14
  • 15. New User Experience for DataFunnel 15Syncsort Confidential and Proprietary - do not copy or distribute DMX DataFunnel™ Syncsort Confidential and Proprietary - do not copy or distribute 15
  • 16. New User Experience for DataFunnel 16Syncsort Confidential and Proprietary - do not copy or distribute DMX DataFunnel™ Syncsort Confidential and Proprietary - do not copy or distribute 16
  • 17. New UI Wizard Flow Creation 17Syncsort Confidential and Proprietary - do not copy or distribute DMX DataFunnel™ Syncsort Confidential and Proprietary - do not copy or distribute 17
  • 18. New UI Wizard Flow Creation 18Syncsort Confidential and Proprietary - do not copy or distribute DMX DataFunnel™ Syncsort Confidential and Proprietary - do not copy or distribute 18
  • 19. GUI Execution 19Syncsort Confidential and Proprietary - do not copy or distribute
  • 20. COMING SOON Product News Syncsort Confidential and Proprietary - do not copy or distribute 20
  • 21. Change Data Capture in DataFunnel 21Syncsort Confidential and Proprietary - do not copy or distribute Keep data in sync in real-time: Without overloading networks. Without affecting source database performance. Without coding or tuning. • HDFS • Hive • Impala • IBM DB2 • IBM Informix • Oracle • Oracle RAC • Sybase • MS SQL Server • Teradata • MySQL • PostgreSQL • IBM DB2 • VSAM • IBM Informix • Oracle • Oracle RAC • Sybase • MS SQL Server Dependable – Reliable transfer of data even if connectivity fails on either side. Fast – Captures changes in source as they happen. Updates table statistics for faster queries. Flexible – Writes to most major RDBMSs, as well as HDFS, all Hive and Impala tables, including those backed by text, ORC, Parquet, Avro or Kudu. Even updates Hive versions that don’t support updates. Real-Time Replication with Transformation Conflict Resolution, Collision Monitoring, Tracking and Auditing
  • 22. Job Monitoring in DataFunnel GUI 22Syncsort Confidential and Proprietary - do not copy or distribute DMX DataFunnel™ Syncsort Confidential and Proprietary - do not copy or distribute 22
  • 23. END TO END DATA LINEAGE 23Syncsort Confidential and Proprietary - do not copy or distribute
  • 24. End to End Data Lineage in Cloudera Navigator 24Syncsort Confidential and Proprietary - do not copy or distribute Data Sources
  • 25. End to End Data Lineage in Cloudera Navigator 25Syncsort Confidential and Proprietary - do not copy or distribute Data Sources Syncsort accesses data from sources outside cluster.
  • 26. End to End Data Lineage in Cloudera Navigator 26Syncsort Confidential and Proprietary - do not copy or distribute Syncsort onboards data, modifies on-the-fly to match Hadoop storage model. Data Sources Syncsort accesses data from sources outside cluster.
  • 27. End to End Data Lineage in Cloudera Navigator 27Syncsort Confidential and Proprietary - do not copy or distribute Syncsort onboards data, modifies on-the-fly to match Hadoop storage model. Data Sources Syncsort accesses data from sources outside cluster. Syncsort changes, enhances, joins data in cluster with MapReduce or Spark. Data Hub
  • 28. End to End Data Lineage in Cloudera Navigator 28Syncsort Confidential and Proprietary - do not copy or distribute Syncsort onboards data, modifies on-the-fly to match Hadoop storage model. Data Sources Syncsort accesses data from sources outside cluster. Syncsort changes, enhances, joins data in cluster with MapReduce or Spark. Syncsort passes source-to- cluster data lineage info to Navigator. Data Hub
  • 29. End to End Data Lineage in Cloudera Navigator 29Syncsort Confidential and Proprietary - do not copy or distribute Syncsort onboards data, modifies on-the-fly to match Hadoop storage model. Data Sources Syncsort accesses data from sources outside cluster. Syncsort changes, enhances, joins data in cluster with MapReduce or Spark. Navigator gathers any other changes made to data on cluster. Syncsort passes source-to- cluster data lineage info to Navigator. Data changes made by MapReduce, Spark, HiveQL. Data Hub
  • 30. End to End Data Lineage in Cloudera Navigator 30Syncsort Confidential and Proprietary - do not copy or distribute Syncsort onboards data, modifies on-the-fly to match Hadoop storage model. Data Sources Syncsort accesses data from sources outside cluster. Syncsort changes, enhances, joins data in cluster with MapReduce or Spark. Analytics and visualizations get complete data. Navigator gathers any other changes made to data on cluster. Data analyst gets end-to-end data lineage info from Navigator. Syncsort passes source-to- cluster data lineage info to Navigator. Data changes made by MapReduce, Spark, HiveQL. Data Hub Analytics, Visualization
  • 31. Syncsort DMX-h + Cloudera Navigator for End-to-End Lineage 31Syncsort Confidential and Proprietary - do not copy or distribute
  • 32. End to End Data Lineage Outside the Cluster 32Syncsort Confidential and Proprietary - do not copy or distribute Data Sources Data Hub Analytics, Visualization Data Lineage Other Syncsort onboards data, modifies on-the-fly to match Hadoop storage model. Syncsort accesses data from sources outside cluster. Syncsort changes, enhances, joins data in cluster with MapReduce or Spark. Analytics and visualizations get complete data. MDM solutions like ASG, Collibra, etc. gather the lineage from the API. Syncsort passes source-to- cluster data lineage info to a REST API. Data analyst gets end-to-end data lineage info from Navigator. REST API
  • 33. Data Lineage + Data Quality = Foundations of Data Governance 33Syncsort Confidential and Proprietary - do not copy or distribute Discovery and Profiling Data Sources Multi-field fuzzy matching, de-duplication, cleansing, enrichment, standardization, business rule enforcement. Analytics and visualizations on clean, complete data you can trust. Data Hub Analytics, Visualization Data Lineage
  • 34. DEMO OF CLOUDERA NAVIGATOR LINEAGE 34Syncsort Confidential and Proprietary - do not copy or distribute
  • 35. What Next? 35Syncsort Confidential and Proprietary - do not copy or distribute Contact Syncsort sales to get the latest info: http://www.syncsort.com/en/ContactSales If you would like to start using the new features in the DataFunnel UI, and provide feedback, contact Product Manager, Ashwin Ramachandran - ARamachandran@Syncsort.com