Achieving Real-Time Analytics at Hermes | Zulf Qureshi, HVR and Dr. Stefan Roessier, Hermes
•Download as PPTX, PDF•
0 likes•521 views
Hermes, Germany's largest post-independent logistics service provider for deliveries, had one main goal—make faster and smarter data-driven business decisions. But with high volumes of diverse and disparate data, how can you effectively leverage it as an asset for real-time insights and business intelligence? During this session, Hermes will share their data challenges and how HVR's high volume data replication capabilities enabled Hermes to securely and seamlessly integrate data into Kafka for real-time decision-making and greater visibility into the entire logistics process.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Achieving Real-Time Analytics at Hermes | Zulf Qureshi, HVR and Dr. Stefan Roessier, Hermes
Similar to Achieving Real-Time Analytics at Hermes | Zulf Qureshi, HVR and Dr. Stefan Roessier, Hermes (20)
More from HostedbyConfluent
More from HostedbyConfluent (20)
Recently uploaded
Recently uploaded (20)
Achieving Real-Time Analytics at Hermes | Zulf Qureshi, HVR and Dr. Stefan Roessier, Hermes
- 1. 1 With Big Data and Your Data Warehouse Achieving Real-Time Analytics
- 2. 2 Speakers Dr. Stefan Roessler Product Owner BI Hermes Zulf Qureshi Team Lead Sales Engineering EMEA HVR
- 4. 4 1972 >280 443m / year
- 5. 5 Experts with More Than 40 Years of Experience in B2C Parcel Shipping 1972 2021 Started as an exclusive shipping services provider for OTTO 443 million shipments (Germany) in the 2020/21 business year – client business rapidly expanding client business otto group & private customer business 30% 70% Share of European parcel shipping
- 7. 7 About HVR
- 8. 8 We are dedicated to helping the enterprise circulate high volumes of data quickly, accurately, and securely where they need it, when they need it. Empowering organizations to realize full potential of their data
- 9. 9 COMPLEXITY SCALABILITY PERFORMANCE HVR customers continually process terabytes of transactional data every hour. Why HVR?
- 11. 11 Hermes’ Challenge and Approach for Real-Time BI
- 12. 12 • Why is there a need? • How did they handle the request? • What was their process?
- 13. 13 More Data—Faster Generated by Humans Generated by Interaction Generated by Companies Generated by Machines
- 14. 14 Big Data Query Grid v2 DBMS OLTP Still batch processing Hadoop Teradata Teradata Aster Source CLOUD OR ON-PREM Target CLOUD OR ON-PREM DBMS OLTP Real-time needs ETL Tool HVR Hub Teradata Listener Hermes Needed a New Tool for CDC
- 15. 15 Our Comparison—Short List for Final Three Providers Site N.N. N.N. Product N.N. N.N. Installation very simple GUI P O P available Stability and Reliability very stable Support of Source and Target Systems many connectors Customer Support local / German-speaking / Holland Trigger Machanism supporting CDC / incl. RAC Throughput Rate very high Initialized Data Transfer available and easy Maintenance and Administration easy License Model different options Price Medium price segment Supplier https://www.hvr-software.com/solutions/real-time-data/ HVR HVR Inc 135 Main Street, Suite 850 San Francisco, CA 94105 USA Product A Product B Site N.N. N.N. Product N.N. N.N. Installation very simple GUI P O P available Stability and Reliability very stable Support of Source and Target Systems many connectors Customer Support local / German-speaking / Holland Trigger Machanism supporting CDC / incl. RAC Throughput Rate very high Initialized Data Transfer available and easy Maintenance and Administration easy License Model different options Price Medium price segment Supplier https://www.hvr-software.com/solutions/real-time-data/ HVR HVR Inc 135 Main Street, Suite 850 San Francisco, CA 94105 USA Product A Product B
- 17. 17 HQ in Hamburg Data Center in Germany Logistic Applications on iSeries DB2/400 DBMS Workstations Logistic Applications on Oracle DBMS (incl. RAC) Housing Datawarehouse Hadoop Teradata DBMS Cloud VM Jump Host Logistic Applications on Oracle DBMS Cloud HVR Hub Solution Architecture
- 18. 18 Logistics in the field Data Center in Germany Streaming Data Enrichment HVR Hub Logistic Application Example Case – Delivery Heatmap
- 19. 19 Improvement for the Swap Body Disposition Current Customer Service Reporting Implemented Use Cases with HVR
- 20. 20 Transmission of Delivery Events Digital Tour Sorting Additional Use Cases using Kafka
- 21. 21 Digital Tour Sorting • Digital support for sorting parcels and plan delivery tours ahead • Process Kafka event stream of shipment data and events to calculate forecast
- 22. 22 Data Processing & Storage Streaming Digital Tour Sorting Shipment data & events
- 23. 23 Transmition of Delivery Event Customer giving signature on label Delivering Parcel, 1 step backwards Transmission of Delivery Events
- 24. 24 Data Processing & Storage Streaming Transmission of Delivery Events Shipment data & events
- 25. 25 Thank you H V R - S O F T W A R E . C O M / C O N TA C T Dr. Stefan Roessler Product Owner BI Hermes Zulf Qureshi Senior Solutions Architect HVR
- 26. 26 Thank you
Editor's Notes
- Like many companies, we are constantly changing - currently much more so than in the past Entire environment of logistics is changing Customers have different requirements than before (more transparency, more information, faster delivery speed) Competitors are changing We have been on the market for 46 years. Put more than 600 million pacts a year -> are thus at the capacity limits Many new buildings started (Logistic Center) and also modernization of the IT systems: basis for digitization and further growth Big reorganizations also in the IT: we see ourselves as partners of the business The way of working has also changed a lot: agile, cross-functional teams, IT and business move together 16 employees at BI
- End to end data replication solution Low-impact, high volume data movement with log-based CDC HVR automates many aspects of data movement through a robust feature set and distributed architect that eliminates complexity in the management of high-volume data flows
- Support for complex heterogeneous environments
- The experience report is structured in 3 stages 1) Why do we do that anyway? - It needs a goal 2) How do we make the change? - Try a lot / many POCs 3) What was our solution? - New architecture and fundamental change of the DWH
- Why? More and more data is being generated faster and faster: People generated: customer service via telephone contacts, e-mail (evaluation with text mining), social networks Machines generated: sorter systems with sensors, volume scanners, Generated through interaction: time recording, hand-held scanner on delivery tour (over 1 million house door contacts per day / 10,000 delivery tours per day) Company generates: Financials Our goals: We want to focus more and more on the data - making business decisions ever more data-driven BI must and can deliver the basics! Change from data warehouse to business analytics
- Need for action: Used CDC tool from Oracle (streams) will no longer be supported in future Oracle versions Complement to the classic ETL tool required Action: Different POCs to convince us of the power of the tools
- Zusammenfassung unserer Produktevaluierung Was war uns wichtig und wie haben die unterschiedlichen Tools hier abgeschnitten? Erfolgsfaktor für uns: viel selber ausprobieren mit POC Z.B. Lizenzmodell: Einige Anbieter haben es im Cloud-Zeitalter noch nicht verstanden, dass eine CPU-basierte Lizenzsierung nicht mehr zeitgemäß ist
- HVR had different modules, initial load At the hub, design, deployment and management is centralized HVR consists of initial charge, incremental changes, and data validation Connecting between source-> hub and hub> goals is efficient and secure HVR Architecture suits the wished architecture of Hermes Optionally, HVR can be configured with SSL certificates to make network traffic visible only with agents on the source and target systems.
- Need for action: Used CDC tool from Oracle (streams) will no longer be supported in future Oracle versions Complement to the classic ETL tool required Action: Different POCs to convince us of the power of the tools For safety reasons, the HVR hub has only been managed by a springhost
- Need for action: Used CDC tool from Oracle (streams) will no longer be supported in future Oracle versions Complement to the classic ETL tool required Action: Different POCs to convince us of the power of the tools For safety reasons, the HVR hub has only been managed by a springhost
- Example 1: Use in the Customer Service center Real-time information / reports for our clients in 1/2 hour time possible Example 2: Use in the disposition of our swap bodies (WAB) -- (a type of standard freight container with an interchangeable unit) Earlier: The data from the disposition system was transferred to the DWH ½ hourly via timed ETL routes (= near-realtime) Today: Takeover with HVR in "real" realtime for reporting Advantage: better control / utilization / path optimization of the WAB by the dispatchers possible
- Example 1: Use in the Customer Service center Real-time information / reports for our clients in 1/2 hour time possible Example 2: Use in the disposition of our swap bodies (WAB) -- (a type of standard freight container with an interchangeable unit) Earlier: The data from the disposition system was transferred to the DWH ½ hourly via timed ETL routes (= near-realtime) Today: Takeover with HVR in "real" realtime for reporting Advantage: better control / utilization / path optimization of the WAB by the dispatchers possible
- Die Sendungsdaten werden über Kafka an eine Mongo übertragen Dazu gibt es weitere Systeme (unter anderem mit Spring Boot) die diese Daten als Sendungsforecast an das Frontend übertragen Der Abwickler benutzt das Frontend (zsb-cockpit) um die Disposition und Tourenplanung für einen Tag durchzuführen Auf Basis der vom Abwickler durchgeführten Disposition hat jede Sendung ein Sortierziel, dass über den Scanner angezeigt wird
- Need for action: Used CDC tool from Oracle (streams) will no longer be supported in future Oracle versions Complement to the classic ETL tool required Action: Different POCs to convince us of the power of the tools For safety reasons, the HVR hub has only been managed by a springhost
- Need for action: Used CDC tool from Oracle (streams) will no longer be supported in future Oracle versions Complement to the classic ETL tool required Action: Different POCs to convince us of the power of the tools For safety reasons, the HVR hub has only been managed by a springhost
- Ich hoffe, Ihnen einen Einblick in unsere Entwicklungen und Erfahrungen gegeben zu haben … sprechen Sie mich oder Herman gerne an … falls Interesse an einem weiteren Austausch besteht …