Apache Flink Adoption @ Shopify

•

1 like•211 views

In this talk, we go over the history and future of Apache Flink adoption at Shopify. We'll talk about how and why we went from choosing Apache Flink as the replacement for our existing streaming technologies in 2021, to a year later with a flourishing streaming community. Today, we have tens of prototypes and several large use-cases running production. Along the way, we'll overview the Flink ecosystem at Shopify, the tools and libraries Shopify built, the decision to fork Flink, how we drove adoption of streaming at the company, and what's next for the platform.

Software

Kevin Lam
Twitter: @CodesKlam
Apache Flink
Adoption @ Shopify

Team Mission:
enable near real-time
stream processing
applications
@CodesKlam

Too specialized
Broken Ownership
Problems
@CodesKlam

Too specialized
Broken Ownership
Unstable
Problems
@CodesKlam

Driven by pilot use case
Why Flink?
@CodesKlam

Driven by pilot use case
Large complex stateful transforms
Why Flink?
@CodesKlam

Driven by pilot use case
Large complex stateful transforms
Consolidate o
ff
ers
Why Flink?
@CodesKlam

1. Streaming Primitives
Why Flink?
@CodesKlam

1. Streaming Primitives
2. Infrastructure
Why Flink?
@CodesKlam

1. Streaming Primitives
2. Infrastructure
3. Scalability
Why Flink?
@CodesKlam

1. Streaming Primitives
2. Infrastructure
3. Scalability
4. Community Support
5. Ease of Programming + Languages
Why Flink?
@CodesKlam

Enable Flink Adoption
Deprecated and shutdown the
old services
The plan
@CodesKlam

Our strategy:
an ecosystem of tools
to make everything
possible, then make it
easy
@CodesKlam

Library
Deployment Infra & Tooling
Foundations
@CodesKlam

Library
Deployment Infra & Tooling
Cookbook
Foundations
@CodesKlam

Library
Deployment Infra & Tooling
Cookbook
Foundations
Hands-on Support
@CodesKlam

Library
Deployment Infra & Tooling
Cookbook
Foundations
Hands-on Support
Project Generator
@CodesKlam

implicit val env: StreamExecutionEnvironment = Trickle.createEnv()
@CodesKlam

implicit val env: StreamExecutionEnvironment = Trickle.createEnv()
val checkoutTrackSource: DataStream[CheckoutTrack_V3] =
MonorailKafkaSource.asDataStream[CheckoutTrack_V3](…)
val lineItemsSource: DataStream[Change[LineItemRecord]] =
CDCKafkaSource.asDataStream[LineItemRecord](…)
val sink: Sink[VendorPopularityTopology.Result] = BigtableSink[Result]
(…)
@CodesKlam

Made Flink generally available in Jan 2022
How did it go?
@CodesKlam

Made Flink generally available in Jan 2022
>75 applications in production
Millions of events processed per second
How did it go?
@CodesKlam

Made Flink generally available in Jan 2022
>75 applications in production
Millions of events processed per second
How did it go?
Legacy Systems shut down
@CodesKlam

Treasure
f
irst customers
🧠 Learnings
@CodesKlam

Treasure
f
irst customers
Be nimble
🧠 Learnings
@CodesKlam

Treasure
f
irst customers
Be nimble
Distributed ownership
🧠 Learnings
@CodesKlam

Low Code Flink Applications
Infrastructure Automation
And more!
Next Steps
@CodesKlam

Thanks!
twitter: @CodesKlam
Also, we’re hiring! shopify.com/careers

What's hot

In a typical deployment of Kafka with many topics and partitions, scaling the Kafka consumer efficiently is one of the important tasks in maintaining overall smooth Kafka operations. The traditional Kubernetes Horizontal Pod Scaling (HPA) that uses basic CPU and/or memory metrics is not suitable for scaling Kafka consumers. The more appropriate workload metric for Kafka consumer is the number of messages in Kafka broker queue. More specifically, the message production rate of a specific topic would be the right workload metric for a Kafka consumer. While using message production rate is a better way to decide the number of consumer replicas, this is still a reaction based auto-scaling. Using machine-learning based forecasting, it is possible for predict the upcoming increase or decrease of message production rate. With predicted workload, scaling the Kafka consumers could be achieved in a more timely manner, resulting with better performance KPI's.

Intelligent Auto-scaling of Kafka Consumers with Workload Prediction | Ming S...

HostedbyConfluent

Apache Flink internals

Kostas Tzoumas

Spark SQL is a highly scalable and efficient relational processing engine with ease-to-use APIs and mid-query fault tolerance. It is a core module of Apache Spark. Spark SQL can process, integrate and analyze the data from diverse data sources (e.g., Hive, Cassandra, Kafka and Oracle) and file formats (e.g., Parquet, ORC, CSV, and JSON). This talk will dive into the technical details of SparkSQL spanning the entire lifecycle of a query execution. The audience will get a deeper understanding of Spark SQL and understand how to tune Spark SQL performance.

Deep Dive into Spark SQL with Advanced Performance Tuning with Xiao Li & Wenc...

Databricks

Flink Forward San Francisco 2022. The Table API is one of the most actively developed components of Flink in recent time. Inspired by databases and SQL, it encapsulates concepts many developers are familiar with. It can be used with both bounded and unbounded streams in a unified way. But from afar it can be difficult to keep track of what this API is capable of and how it relates to Flink's other APIs. In this talk, we will explore the current state of Table API. We will show how it can be used as a batch processor, a changelog processor, or a streaming ETL tool with many built-in functions and operators for deduplicating, joining, and aggregating data. By comparing it to the DataStream API we will highlight differences and elaborate on when to use which API. We will demonstrate hybrid pipelines in which both APIs interact with one another and contribute their unique strengths. Finally, we will take a look at some of the most recent additions as a first step to stateful upgrades. by David Andreson

The Current State of Table API in 2022

Flink Forward

Search and recommendation system for Alibaba’s e-commerce platform use batch and streaming processing heavily. Flink SQL and Table API (which is a SQL-like DSL) provide simple, flexible, and powerful language to express the data processing logic. More importantly, it opens the door to unify the semantics of batch and streaming jobs. Blink is a project at Alibaba which improves Apache Flink to make it ready for large scale production use. To support our products, we made lots of improvements to Flink SQL & TableAPI in Alibaba's Blink project. We added the support for User-Defined Table function (UDTF), User-Defined Aggregates (UDAGG), Window Aggregate, and retraction, etc. We are actively working with the Flink community to contribute these improvements back. In this talk, we will present the rationale, semantics, design and implementation of these improvements. We will also share the experience of running large scale Flink SQL and TableAPI jobs at Alibaba.

Flink SQL & TableAPI in Large Scale Production at Alibaba

DataWorks Summit

Reading The Source Code of Presto

Taro L. Saito

How Adobe Does 2 Million Records Per Second Using Apache Spark!

Databricks

Mvcc (oracle, innodb, postgres)

frogd

Apache Hudi: The Path Forward

Alluxio, Inc.

Flink Forward San Francisco 2022. Probably everyone who has written stateful Apache Flink applications has used one of the fault-tolerant keyed state primitives ValueState, ListState, and MapState. With RocksDB, however, retrieving and updating items comes at an increased cost that you should be aware of. Sometimes, these may not be avoidable with the current API, e.g., for efficient event-time stream-sorting or streaming joins where you need to iterate one or two buffered streams in the right order. With FLIP-220, we are introducing a new state primitive: BinarySortedMultiMapState. This new form of state offers you to (a) efficiently store lists of values for a user-provided key, and (b) iterate keyed state in a well-defined sort order. Both features can be backed efficiently by RocksDB with a 2x performance improvement over the current workarounds. This talk will go into the details of the new API and its implementation, present how to use it in your application, and talk about the process of getting it into Flink. by Nico Kruber

Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...

Flink Forward

Intro to Delta Lake

Databricks

Flink Forward San Francisco 2022. Apache Flink and Delta Lake together allow you to build the foundation for your data lakehouses by ensuring the reliability of your concurrent streams from processing to the underlying cloud object-store. Together, the Flink/Delta Connector enables you to store data in Delta tables such that you harness Delta’s reliability by providing ACID transactions and scalability while maintaining Flink’s end-to-end exactly-once processing. This ensures that the data from Flink is written to Delta Tables in an idempotent manner such that even if the Flink pipeline is restarted from its checkpoint information, the pipeline will guarantee no data is lost or duplicated thus preserving the exactly-once semantics of Flink. by Scott Sandre & Denny Lee

Building Reliable Lakehouses with Apache Flink and Delta Lake

Flink Forward

Flink Forward San Francisco 2022. At Stripe we have created a complete end to end exactly-once processing pipeline to process financial data at scale, by combining the exactly-once power from Flink, Kafka, and Pinot together. The pipeline provides exactly-once guarantee, end-to-end latency within a minute, deduplication against hundreds of billions of keys, and sub-second query latency against the whole dataset with trillion level rows. In this session we will discuss the technical challenges of designing, optimizing, and operating the whole pipeline, including Flink, Kafka, and Pinot. We will also share our lessons learned and the benefits gained from exactly-once processing. by Xiang Zhang & Pratyush Sharma & Xiaoman Dong

Exactly-Once Financial Data Processing at Scale with Flink and Pinot

Flink Forward

Building a Data Pipeline using Apache Airflow (on AWS / GCP)

Yohei Onishi

Flink Forward San Francisco 2022. With a real-time processing engine like Flink and a transactional storage layer like Hudi, it has never been easier to build end-to-end low-latency data platforms connecting sources like Kafka to data lake storage. Come learn how to blend Lakehouse architectural patterns with real-time processing pipelines with Flink and Hudi. We will dive deep on how Flink can leverage the newest features of Hudi like multi-modal indexing that dramatically improves query and write performance, data skipping that reduces the query latency by 10x for large datasets, and many more innovations unique to Flink and Hudi. by Ethan Guo & Kyle Weller

How to build a streaming Lakehouse with Flink, Kafka, and Hudi

Flink Forward

Flink Forward San Francisco 2022. This talk will take you on the long journey of Apache Flink into the cloud-native era. It started all the way from where Hadoop and YARN were the standard way of deploying and operating data applications. We're going to deep dive into the cloud-native set of principles and how they map to the Apache Flink internals and recent improvements. We'll cover fast checkpointing, fault tolerance, resource elasticity, minimal infrastructure dependencies, industry-standard tooling, ease of deployment and declarative APIs. After this talk you'll get a broader understanding of the operational requirements for a modern streaming application and where the current limits are. by David Moravek

Apache Flink in the Cloud-Native Era

Flink Forward

Optimising Geospatial Queries with Dynamic File Pruning

Databricks

Flink Forward San Francisco 2022. What if my data is already in order? Stream Processing has given us an elegant and powerful solution for running analytic queries and logic over high volumes of continuously arriving data. However, in both Apache Flink and Apache Beam, the notion of time-ordering is baked in at a very low level, making it difficult to express computations that are interested in a semantic-, rather than time-ordering of the data. In financial services, what often matters the most about the data moving between systems is not when the data was created, but in what order, to the extent that many institutions engineer a global sequencing over all data entering and produced by their systems to achieve complete determinism. How, then, can financial institutions and others best employ Stream Processing on streams of data that are already ordered? I will cover various techniques that can make this work, as well as seek input from the community on how Flink might be improved to better support these use-cases. by Patrick Lucas

Processing Semantically-Ordered Streams in Financial Services

Flink Forward

Fast data arrives in real time and potentially high volume. Rapid processing, filtering and aggregation is required to ensure timely reaction and actual information in user interfaces. Doing so is a challenge, make this happen in a scalable and reliable fashion is even more interesting. This session introduces Apache Kafka as the scalable event bus that takes care of the events as they flow in and Kafka Streams for the streaming analytics. Both Java and Node applications are demonstrated that interact with Kafka and leverage Server Sent Events and WebSocket channels to update the Web UI in real time. User activity performed by the audience in the Web UI is processed by the Kafka powered back end and results in live updates on all clients. Kafka Streams and KSQL are used to analyze the real time events in real time and publish events with the live findings.

Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push

Lucas Jellema

Flink Forward San Francisco 2022. Next time you want to integrate with a new destination for a demo, concept or production application, the Async Sink framework will bootstrap development, allowing you to move quickly without compromise. In Flink 1.15 we introduced the Async Sink base (FLIP-171), with the goal to encapsulate common logic and allow developers to focus on the key integration code. The new framework handles things like request batching, buffering records, applying backpressure, retry strategies, and at least once semantics. It allows you to focus on your business logic, rather than spending time integrating with your downstream consumers. During the session we will dive deep into the internals to uncover how it works, why it was designed this way, and how to use it. We will code up a new sink from scratch and demonstrate how to quickly push data to a destination. At the end of this talk you will be ready to start implementing your own Flink sink using the new Async Sink framework. by Steffen Hausmann & Danny Cranmer

One sink to rule them all: Introducing the new Async Sink

Flink Forward

What's hot (20)

Intelligent Auto-scaling of Kafka Consumers with Workload Prediction | Ming S...

Apache Flink internals

Deep Dive into Spark SQL with Advanced Performance Tuning with Xiao Li & Wenc...

The Current State of Table API in 2022

Flink SQL & TableAPI in Large Scale Production at Alibaba

Reading The Source Code of Presto

How Adobe Does 2 Million Records Per Second Using Apache Spark!

Mvcc (oracle, innodb, postgres)

Apache Hudi: The Path Forward

Introducing BinarySortedMultiMap - A new Flink state primitive to boost your ...

Intro to Delta Lake

Building Reliable Lakehouses with Apache Flink and Delta Lake

Exactly-Once Financial Data Processing at Scale with Flink and Pinot

Building a Data Pipeline using Apache Airflow (on AWS / GCP)

How to build a streaming Lakehouse with Flink, Kafka, and Hudi

Apache Flink in the Cloud-Native Era

Optimising Geospatial Queries with Dynamic File Pruning

Processing Semantically-Ordered Streams in Financial Services

Real Time UI with Apache Kafka Streaming Analytics of Fast Data and Server Push

One sink to rule them all: Introducing the new Async Sink

Similar to Apache Flink Adoption @ Shopify

ColdFusion Keynote: Building the Agile Web Since 1995

ColdFusionConference

Despite what the Ghostbusters said, we’re going to go ahead and cross (or, join) the streams. This session covers getting started with streaming data pipelines, maximizing Pulsar’s messaging system alongside one of the most flexible streaming frameworks available, Apache Flink. Specifically, we’ll demonstrate the use of Flink SQL, which provides various abstractions and allows your pipeline to be language-agnostic. So, if you want to leverage the power of a high-speed, highly customizable stream processing engine without the usual overhead and learning curves of the technologies involved (and their interconnected relationships), then this talk is for you. Watch the step-by-step demo to build a unified batch and streaming pipeline from scratch with Pulsar, via the Flink SQL client. This means you don’t need to be familiar with Flink, (or even a specific programming language). The examples provided are built for highly complex systems, but the talk itself will be accessible to any experience level.

Cross the Streams! Creating Streaming Data Pipelines with Apache Flink + Apac...

StreamNative

Have your cake and eat it too

Gwen (Chen) Shapira

Have your Cake and Eat it Too - Architecture for Batch and Real-time processing

DataWorks Summit

When building service-based systems, we don’t generally think too much about data. If we need data from another service, we ask for it. This pattern works well for whole swathes of use cases, particularly ones where datasets are small and requirements are simple. But real business services have to join and operate on datasets from many different sources. This can be slow and cumbersome in practice. These problems stem from an underlying dichotomy. Data systems are built to make data as accessible as possible—a mindset that focuses on getting the job done. Services, instead, focus on encapsulation—a mindset that allows independence and autonomy as we evolve and grow. But these two forces inevitably compete in most serious service-based architectures. Ben Stopford explains why understanding and accepting this dichotomy is an important part of designing service-based systems at any significant scale. Ben looks at how companies make use of a shared, immutable sequence of records to balance data that sits inside their services with data that is shared, an approach that allows the likes of Uber, Netflix, and LinkedIn to scale to millions of events per second. Ben concludes by examining the potential of stream processors as a mechanism for joining significant, event-driven datasets across a whole host of services and explains why stream processing provides much of the benefits of data warehousing but without the same degree of centralization.

NDC London 2017 - The Data Dichotomy- Rethinking Data and Services with Streams

Ben Stopford

Scaling Rails With Torquebox Presented at JUDCon:2011 Boston

benbrowning

Despite being a well-established standard for bidirectional persistent connection between client and server, WebSocket is complicated to set up on Android. When Tinder migrated to WebSocket for its realtime chat experience, they developed Scarlet: an extensible Kotlin library inspired by Retrofit that eliminates the boilerplate code required to handle data serialization and specify when to connect and retry. After this talk, you’ll be able to configure Scarlet using many of its plugins and integrate any WebSocket API in 10 minutes. https://tech.gotinder.com/taming-websocket-with-scarlet/ https://github.com/Tinder/Scarlet https://github.com/Tinder/StateMachine

Taming WebSocket with Scarlet

Zhixuan Lai

What's Missing? Microservices Meetup at Cisco

Adrian Cockcroft

Full-Stack Reativo com Spring WebFlux + Angular - Devs Java Girl

Loiane Groner

session_01_react_.pptx

AyaBenkabbour1

AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. Using Step Functions, you can design and run workflows that stitch together services such as AWS Lambda and Amazon ECS into feature-rich applications. Application development is simpler and more intuitive using Step Functions, because it translates your workflow into a state machine diagram that is easy to understand, easy to explain to others, and easy to change.

Building Advanced Serverless Workflows with AWS Step Functions | AWS Floor28

Amazon Web Services

The use case of a scalable architecture

Toru Wonyoung Choi

SQL is the lingua franca of data processing and everybody working with data knows SQL. Apache Flink provides SQL support for querying and processing batch and streaming data. Flink’s SQL support powers large-scale production systems at Alibaba, Huawei, and Uber. Based on Flink SQL, these companies have built systems for their internal users as well as publicly offered services for paying customers. In our talk, we will discuss why you should and how you can (not being Alibaba or Uber) leverage the simplicity and power of SQL on Flink. We will start exploring the use cases that Flink SQL was designed for and present real-world problems that it can solve. In particular, you will learn why unified batch and stream processing is important and what it means to run SQL queries on streams of data. After we explored why you should use Flink SQL, we will show how you can leverage its full potential. Since recently, the Flink community is working on a service that integrates a query interface, (external) table catalogs, and result serving functionality for static, appending, and updating result sets. We will discuss the design and feature set of this query service and how it can be used for exploratory batch and streaming queries, ETL pipelines, and live updating query results that serve applications, such as real-time dashboards. The talk concludes with a brief demo of a client running queries against the service.

Flink Forward San Francisco 2018: Fabian Hueske & Timo Walther - "Why and how...

Flink Forward

Real time dashboards with Kafka and Druid

Venu Ryali

Presentation given by Renzo Tomà as "Tech and Use Case Deep Dive", during the Elastic{ON}Tour 2015 event in Amsterdam on October 29th. Explanation of how bol.com is using the Elastic ELK stack to power a logsearch platform. Lots of details on the types of sources and number of feeds. Some history and reasoning why the current set of in-process JSON based logshippers are used. Links to the bol.com github account for the logshipper projects. The presentation ends with two special sauces: fun things you can do with lots of data in Elasticsearch. The 1st sauce is 'the call stack' - tagging each request with a unique ID, passing that ID along to all service calls and making sure this ID ends up in all access logging, enables you to group all calls together and get a call stack. The 2nd sauce is a way of generating a service map using access logging and some logstash magic. I love questions and feedback. My mail address can be found in the presentation.

How bol.com makes sense of its logs, using the Elastic technology stack.

Renzo Tomà

Vertica the convertro way

Zvika Gutkin

Real-time data streaming is a hot topic in the Telecommunications Industry / Telecom Sector. As telecommunications companies strive to offer high speed, integrated networks with reduced connection times, connect countless devices at reduced latency, and transform the digital experience worldwide, more and more companies are turning to Apache Kafka’s data stream processing solutions to deliver a scalable, real-time infrastructure for OSS and BSS scenarios. Enabling a combination of on-premise data centers, edge processing, and multi-cloud architectures is becoming the new normal in the Telco Industry. This combination is enabling accelerated growth from value-added services delivered over mobile networks. Join Kai Waehner, Technology Evangelist at Confluent, for this session which explores various telecommunications use cases, including data integration, infrastructure monitoring, data distribution, data processing and business applications. Different architectures and components from the Kafka ecosystem are also discussed. This talk explores: - Overcome challenges for building a modern hybrid telco infrastructure - Build a real time infrastructure to correlate relevant events - Connect thousands of devices, networks, infrastructures, and people - Work together with different companies, organisations and business models - Leverage open source and fully managed solutions from the Apache Kafka ecosystem, Confluent Platform and Confluent Cloud

Apache Kafka in the Telco Industry (OSS, BSS, OTT, IMS, NFV, Middleware, Main...

Kai Wähner

Real-time data streaming is a hot topic in the Telecommunications Industry. As telecommunications companies strive to offer high speed, integrated networks with reduced connection times, connect countless devices at reduced latency, and transform the digital experience worldwide, more and more companies are turning to Apache Kafka’s data stream processing solutions to deliver a scalable, real-time infrastructure for OSS and BSS scenarios. Enabling a combination of on-premise data centres, edge processing, and multi-cloud architectures is becoming the new normal in the Telco Industry. This combination is enabling accelerated growth from value-added services delivered over mobile networks. Join Kai Waehner, Technology Evangelist at Confluent, for this session which explores various telecommunications use cases, including data integration, infrastructure monitoring, data distribution, data processing and business applications. Different architectures and components from the Kafka ecosystem are also discussed. Review this online talk to learn how to: - Overcome challenges for building a modern hybrid telco infrastructure - Build a real time infrastructure to correlate relevant events - Connect thousands of devices, networks, infrastructures, and people - Work together with different companies, organisations and business models - Leverage open source and fully managed solutions from the Apache Kafka ecosystem, Confluent Platform and Confluent Cloud Speaker: Kai Waehner

Event Streaming in the Telco Industry with Apache Kafka® and Confluent

confluent

Deploying and Managing PowerPivot for SharePoint

Denny Lee

Mike Taulty MIX10 Silverlight Frameworks and Patterns

ukdpe

Similar to Apache Flink Adoption @ Shopify (20)

ColdFusion Keynote: Building the Agile Web Since 1995

Cross the Streams! Creating Streaming Data Pipelines with Apache Flink + Apac...

Have your cake and eat it too

Have your Cake and Eat it Too - Architecture for Batch and Real-time processing

NDC London 2017 - The Data Dichotomy- Rethinking Data and Services with Streams

Scaling Rails With Torquebox Presented at JUDCon:2011 Boston

Taming WebSocket with Scarlet

What's Missing? Microservices Meetup at Cisco

Full-Stack Reativo com Spring WebFlux + Angular - Devs Java Girl

session_01_react_.pptx

Building Advanced Serverless Workflows with AWS Step Functions | AWS Floor28

The use case of a scalable architecture

Flink Forward San Francisco 2018: Fabian Hueske & Timo Walther - "Why and how...

Real time dashboards with Kafka and Druid

How bol.com makes sense of its logs, using the Elastic technology stack.

Vertica the convertro way

Apache Kafka in the Telco Industry (OSS, BSS, OTT, IMS, NFV, Middleware, Main...

Event Streaming in the Telco Industry with Apache Kafka® and Confluent

Deploying and Managing PowerPivot for SharePoint

Mike Taulty MIX10 Silverlight Frameworks and Patterns

Recently uploaded

WSO2Con204 - Hard Rock Presentation - Keynote

WSO2

WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...

WSO2

WSO2Con2024 - Low-Code Integration Tooling

WSO2

WSO2Con2024 - GitOps in Action: Navigating Application Deployment in the Plat...

WSO2

WSO2Con2024 - Organization Management: The Revolution in B2B CIAM

WSO2

WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation

WSO2

WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...

WSO2

WSO2Con2024 - Software Delivery in Hybrid Environments

WSO2

WSO2CON 2024 - Software Engineering for Digital Businesses

WSO2

Driving Innovation: Scania's API Revolution with WSO2

WSO2

WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...

WSO2

WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!

WSO2

WSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity

WSO2

WSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration

WSO2

Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...

Bert Jan Schrijver

Announcing Codolex 2.0 from GDK Software

Jim McKeeth

WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...

WSO2

Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in Tembisa ● Abortion Pills For Sale in Tembisa ● Tembisa 🏥🚑!! Abortion Clinic Near Me Cost, Price, Women's Clinic Near Me, Abortion Clinic Near, Abortion Doctors Near me, Abortion Services Near Me, Abortion Pills Over The Counter, Abortion Pill Doctors' Offices, Abortion Clinics, Abortion Places Near Me, Cheap Abortion Places Near Me, Medical Abortion & Surgical Abortion, approved cyctotec pills and womb cleaning pills too plus all the instructions needed This Discrete women’s Termination Clinic offers same day services that are safe and pain free, we use approved pills and we clean the womb so that no side effects are present. Our main goal is that of preventing unintended pregnancies and unwanted births every day to enable more women to have children by choice, not chance. We offer Terminations by Pill and The Morning After Pill.” Our Private VIP Abortion Service offers the ultimate in privacy, efficiency and discretion. we do safe and same day termination and we do also womb cleaning as well its done from 1 week up to 28 weeks. We do delivery of our services world wide SAFE ABORTION CLINICS/PILLS ON SALE WE DO DELIVERY OF PILLS ALSO Abortion clinic at very low costs, 100% Guaranteed and it’s safe, pain free and a same day service. It Is A 45 Minutes Procedure, we use tested abortion pills and we do womb cleaning as well. Alternatively the medical abortion pill and womb cleansing !!!

Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...

Medical / Health Care (+971588192166) Mifepristone and Misoprostol tablets 200mg

Artyushina_Guest lecture_YorkU CS May 2024.pptx

AnnaArtyushina1

Azure Native Qumulo scales elastically for common High Performance Compute (HPC) workloads based on application requirements for: Financial Services, Automotive, Genomics / Life Sciences, Media and Entertainment, Energy, Oil and Gas, and more. Performance can be increased (and elastically decreased) much higher than the examples shown here. These slides offer a glimpse into ANQ's HPC capabilities, although at a smaller scale. We invite YOU to do your own testing (with a free ANQ trial) and work with us to test your HPC workloads in Azure.

AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf

ryanfarris8

Recently uploaded (20)

WSO2Con204 - Hard Rock Presentation - Keynote

WSO2CON 2024 - Unlocking the Identity: Embracing CIAM 2.0 for a Competitive A...

WSO2Con2024 - Low-Code Integration Tooling

WSO2Con2024 - GitOps in Action: Navigating Application Deployment in the Plat...

WSO2Con2024 - Organization Management: The Revolution in B2B CIAM

WSO2CON 2024 - OSU & WSO2: A Decade Journey in Integration & Innovation

WSO2Con2024 - Facilitating Broadband Switching Services for UK Telecoms Provi...

WSO2Con2024 - Software Delivery in Hybrid Environments

WSO2CON 2024 - Software Engineering for Digital Businesses

Driving Innovation: Scania's API Revolution with WSO2

WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...

WSO2CON 2024 - Not Just Microservices: Rightsize Your Services!

WSO2Con2024 - Enabling Transactional System's Exponential Growth With Simplicity

WSO2CON2024 - Why Should You Consider Ballerina for Your Next Integration

Devoxx UK 2024 - Going serverless with Quarkus, GraalVM native images and AWS...

Announcing Codolex 2.0 from GDK Software

WSO2CON 2024 - Cloud Native Middleware: Domain-Driven Design, Cell-Based Arch...

Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...

Artyushina_Guest lecture_YorkU CS May 2024.pptx

AzureNativeQumulo_HPC_Cloud_Native_Benchmarks.pdf

Apache Flink Adoption @ Shopify

1. Kevin Lam Twitter: @CodesKlam Apache Flink Adoption @ Shopify

2. Team Mission: enable near real-time stream processing applications @CodesKlam

3. 2M+ ~175 Merchants Countries @CodesKlam

4. N to 1 @CodesKlam

5. @CodesKlam

6. @CodesKlam

7. Problems @CodesKlam

8. Too specialized Problems @CodesKlam

9. Too specialized Broken Ownership Problems @CodesKlam

10. Too specialized Broken Ownership Unstable Problems @CodesKlam

11. 👋 @CodesKlam

12. Why Flink? @CodesKlam

13. Driven by pilot use case Why Flink? @CodesKlam

14. Driven by pilot use case Large complex stateful transforms Why Flink? @CodesKlam

15. Driven by pilot use case Large complex stateful transforms Consolidate o ff ers Why Flink? @CodesKlam

16. Why Flink? @CodesKlam

17. 1. Streaming Primitives Why Flink? @CodesKlam

18. 1. Streaming Primitives 2. Infrastructure Why Flink? @CodesKlam

19. 1. Streaming Primitives 2. Infrastructure 3. Scalability Why Flink? @CodesKlam

20. 1. Streaming Primitives 2. Infrastructure 3. Scalability 4. Community Support 5. Ease of Programming + Languages Why Flink? @CodesKlam

21. Enable Flink Adoption Deprecated and shutdown the old services The plan @CodesKlam

22. Enable Flink Adoption Deprecated and shutdown the old services The plan @CodesKlam

23. Our strategy: an ecosystem of tools to make everything possible, then make it easy @CodesKlam

24. Foundations @CodesKlam

25. Library Foundations @CodesKlam

26. Library Deployment Infra & Tooling Foundations @CodesKlam

27. Library Deployment Infra & Tooling Cookbook Foundations @CodesKlam

28. Library Deployment Infra & Tooling Cookbook Foundations Hands-on Support @CodesKlam

29. Library Deployment Infra & Tooling Cookbook Foundations Hands-on Support Project Generator @CodesKlam

30. @CodesKlam

31. @CodesKlam

32. @CodesKlam

33. @CodesKlam

34. @CodesKlam

35. @CodesKlam

36. @CodesKlam

37. implicit val env: StreamExecutionEnvironment = Trickle.createEnv() @CodesKlam

38. implicit val env: StreamExecutionEnvironment = Trickle.createEnv() val checkoutTrackSource: DataStream[CheckoutTrack_V3] = MonorailKafkaSource.asDataStream[CheckoutTrack_V3](…) val lineItemsSource: DataStream[Change[LineItemRecord]] = CDCKafkaSource.asDataStream[LineItemRecord](…) val sink: Sink[VendorPopularityTopology.Result] = BigtableSink[Result] (…) @CodesKlam

39. implicit val env: StreamExecutionEnvironment = Trickle.createEnv() val checkoutTrackSource: DataStream[CheckoutTrack_V3] = MonorailKafkaSource.asDataStream[CheckoutTrack_V3](…) val lineItemsSource: DataStream[Change[LineItemRecord]] = CDCKafkaSource.asDataStream[LineItemRecord](…) val sink: Sink[VendorPopularityTopology.Result] = BigtableSink[Result] (…) val checkouts = processCheckoutTrackSource(checkoutTrackSource) val lineItems = processLineItemsSource(lineItemsSource) val results = aggregateJoinResults( joinCheckoutsAndLineItems(checkouts, lineItems) ) @CodesKlam

40. implicit val env: StreamExecutionEnvironment = Trickle.createEnv() val checkoutTrackSource: DataStream[CheckoutTrack_V3] = MonorailKafkaSource.asDataStream[CheckoutTrack_V3](…) val lineItemsSource: DataStream[Change[LineItemRecord]] = CDCKafkaSource.asDataStream[LineItemRecord](…) val sink: Sink[VendorPopularityTopology.Result] = BigtableSink[Result] (…) val checkouts = processCheckoutTrackSource(checkoutTrackSource) val lineItems = processLineItemsSource(lineItemsSource) val results = aggregateJoinResults( joinCheckoutsAndLineItems(checkouts, lineItems) ) results.sinkTo(sink) @CodesKlam

41. implicit val env: StreamExecutionEnvironment = Trickle.createEnv() val checkoutTrackSource: DataStream[CheckoutTrack_V3] = MonorailKafkaSource.asDataStream[CheckoutTrack_V3](…) val lineItemsSource: DataStream[Change[LineItemRecord]] = CDCKafkaSource.asDataStream[LineItemRecord](…) val sink: Sink[VendorPopularityTopology.Result] = BigtableSink[Result] (…) val checkouts = processCheckoutTrackSource(checkoutTrackSource) val lineItems = processLineItemsSource(lineItemsSource) val results = aggregateJoinResults( joinCheckoutsAndLineItems(checkouts, lineItems) ) results.sinkTo(sink) @CodesKlam

42. How did it go? @CodesKlam

43. Made Flink generally available in Jan 2022 How did it go? @CodesKlam

44. Made Flink generally available in Jan 2022 >75 applications in production Millions of events processed per second How did it go? @CodesKlam

45. Made Flink generally available in Jan 2022 >75 applications in production Millions of events processed per second How did it go? Legacy Systems shut down @CodesKlam

46. 🧠 Learnings @CodesKlam

47. Treasure f irst customers 🧠 Learnings @CodesKlam

48. Treasure f irst customers Be nimble 🧠 Learnings @CodesKlam

49. Treasure f irst customers Be nimble Distributed ownership 🧠 Learnings @CodesKlam

50. Low Code Flink Applications Infrastructure Automation And more! Next Steps @CodesKlam

51. Thanks! twitter: @CodesKlam Also, we’re hiring! shopify.com/careers