From microservices to serverless - Chicago CTO Summit 2019

Software

@pcalcado - philcalcado.com
We
are
all
about
live
events

@pcalcado - philcalcado.com
I am very biased
towards microservices

@pcalcado - philcalcado.com
Monolith to Services

@pcalcado - philcalcado.com
Monolith to Services
Monolith to Microservices

@pcalcado - philcalcado.com
Monolith to Services
Monolith to Microservices
Monolith to Microservices

@pcalcado - philcalcado.com
Monolith to Services
Monolith to Microservices
Monolith to Microservices
Serverless?

@pcalcado - philcalcado.com
Phil Calçado - @pcalcado - philcalcado.com
From microservices
to serverless:
The Leadership Edition

@pcalcado - philcalcado.com
every time you create
a new application,
you need to make
lots of decisions

@pcalcado - philcalcado.com
What language?
How to deploy?
How to monitor?
How to alert?
What framework?
Do we need to
worry about
compliance?
Which team owns it?

@pcalcado - philcalcado.com
wasn’t the whole point
of Serverless that I
don’t need to think
about this stuff?

@pcalcado - philcalcado.com
Iteration 0 Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5
User Story
Technical Task

@pcalcado - philcalcado.com
User Story
Technical Task
Iteration 0 Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 5

@pcalcado - philcalcado.com
1. Experimentation
2. Checklists and Standards
3. Copy & Paste
4. Libraries and Tools
5. Platform
My ﬁve steps to Microservices

@pcalcado - philcalcado.com
how Serverless breaks
my recipe

@pcalcado - philcalcado.com
so, what to do when
adopting Serverless?

@pcalcado - philcalcado.com
default to tech your
team knows

@pcalcado - philcalcado.com
expect to allocate at
least 1/10 of your
capacity towards
platform

@pcalcado - philcalcado.com
keep tight control of
infrastructure costs

@pcalcado - philcalcado.com
consider paying for
the higher tiers of
AWS Support

@pcalcado - philcalcado.com
…should I even try
Serverless then?

@pcalcado - philcalcado.com
Some version of
Serverless feels like
the future, but we are
not there yet

TALK #2: Microservices vs. The First Law of Object Design We've been breaking systems and application into smaller components for a long time now. From Component-Based Design to Distributed Objects to SOA to what is today's preferred golden hammer: microservices. One definition of microservices is that it is a flavor of SOA that emphasizes many specialize services versus a few more generalist ones. Often these microservices are so small that they take care of a single "object". Distributed objects aren't new to this industry, and in 2003, Martin Fowler wrote a classic article where he discusses several problems with this model, and proposes the First Law of Distributed Objects: "Objects have been around for a while, and sometimes it seems that ever since they were created, folks have wanted to distribute them. However, distribution of objects, or indeed of anything else, has a lot more pitfalls than many people realize, especially when they're under the influence of vendors' cozy brochures. This article is about some of these hard lessons-lessons I've seen many of my clients learn the hard way... my First Law of Distributed Object Design: Don’t distribute your objects!" Reinventing the wheel is nothing new in our field, but if microservices are meant to be small, how can we avoid the same problems from the past? What are the technologies, architectures, protocols, and practices we need in place to make sure that our microservices architecture isn't just the largest bowl of spaghetti this organization has ever cooked? SPEAKER: Phil Calçado, Director of Software Engineering at DigitalOcean Phil Calçado works at DigitalOcean, where he helps build the cloud for developers. Before that, he spent four years building the team and architecture behind SoundCloud's move from a monolith to microservices. He tweets at @pcalcado writes at http://philcalcado.com.

The Economics of Microservices (2017 CraftConf)

what i have learnt working with startups.Phil Calçado

The Not-So-Straightforward Road From Microservices to Serverless

For the last ten years or so, many companies have focused on migrating from larger, monolithic systems and applications towards a specific style of Service-Oriented Architecture called Microservices. The promise was that these smaller, loosely-coupled, and independently developed components would increase productivity and safety for organizations, as large and complex business challenges can be broken down into smaller and simpler components. However, even before most organizations were able to fully migrate to this new architecture and enjoy its promised benefits, a new iteration of cloud computing has been made available in the shape of Serverless platforms such as AWS Lambda, and Google Cloud and Microsoft Azure Functions. Now lots of companies found themselves conflicted between abandoning their traditional microservices approach towards Serverless, often even before they have fully migrated to it from the legacy systems. After leading successful Microservices adoption at SoundCloud and DigitalOcean, recently at Meetup I have faced the challenge of pivoting from a traditional monolith-to-microservices migration to a cloud-native platform. In this talk, let's discuss the fundamental concepts, technologies, and practices behind Microservices and Serverless, and how a software architect used to distributed systems based on microservices needs to change their mindset and approach when adopting Serverless.

Three Years of Microservices at SoundCloud - Distributed Matters Berlin 2015

SoundCloud is the largest repository of audio on the web, used by more than 200 million people every month, who upload more than 11 hours of audio every minute. Like so many others, we have migrated from a typical monolithic architecture to microservices. While the benefits brought by this style of SOA to our productivity and reliability are clear, the architecture required some non-obvious changes in the way we operate systems, and a way to tackle the overhead associated with having hundreds of small moving parts to serve every request. In this talk we’ll share the toolkit and strategy SoundCloud uses to keep its microservices explosion manageable. What do we do about the operations overhead? How to spread devops skills across teams to support the “you build it, you run it” vision? How to deal with breaking changes and asynchronous behaviours? How to deal with chatty interactions? Which protocol? How do I even get a diagram telling me how all this stuff is put together?

Better Functional Design through TDDPhil Calçado

Finagle-Based Microservices at SoundCloud

This is the slide deck from my keynote at the EA User Event in Brussels, September 2015. Micro-services and micro-services architecture are the next hype in software development. Websites and blogs are full of introducing posts, the first books are being written and the first conferences organized. There’s big promises of scalability, flexibility and replaceability of individual elements in your landscape. However, when you are knee deep in the mud as a software architect at an insurance, it is very hard to find help on how to design applications and components in a micro-services architecture. During this talk Sander will show how he used Enterprise Architect to model the micro services architecture, and will explain the difficulties and the lessons learned, using many real-life examples.

Multipathed, Multiplexed, Multilateral Transport Protocols - Decoupling trans...

APNIC

Microservices. The good the bad and the ugly

Slide deck from my keynote at the Software Development 2020 Conference in Breda, The Netherlands, June 2015. Micro-services and micro-services architecture are the next hype in software development. Websites and blogs are full of introducing posts, the first books are being written and the first conferences organized. There’s big promises of scalability, flexibility and replaceability of individual elements in your landscape. However, when you are knee deep in the mud as a software architect at an insurance, it is very hard to find help on how to design applications and components in a micro-services architecture. During this talk Sander Hoogendoorn, discusses the long and winding road the insurance company where he’s acting as the lead software architect has taken to implement their business processes in a micro-landscape. Sander will show how this company is modeling requirements in a micro-landscape using smart use cases, and will explain the difficulties and the lessons learned, using many real-life examples.

Why You Need Sensor Fusion In Your 2016 Retail Analytics Strategy

G3 Communications

The Future Of Information Technology: Welcome…to the Digital Age

Cisco Canada

DIGITAL POP MARKETING Optometrists PowerPoint

DIGITAL POP MARKETING Plumbing Services PowerPoint

Principles of Microservices - NDC 2014

Sam Newman

Migrate to-pg pgconfeu2019

Loxodata

Designing and building a micro-services architecture. Stairway to heaven or a...

Micro-services and micro-services architecture are the next hype in software development. Websites and blogs are full of introducing posts, the first books are being written and the first conferences organized. There’s big promises of scalability, flexibility and replaceability of individual elements in your landscape. However, when you are knee deep in the mud as a software architect at an insurance, it is very hard to find help on how to design applications and components in a micro-services architecture. During this talk Sander Hoogendoorn, discusses the long and winding road the insurance company where he’s acting as the lead software architect has taken to implement their business processes in a micro-landscape. Sander will show how this company is modeling requirements in a micro-landscape using smart use cases, and will explain the difficulties and the lessons learned, using many real-life examples.

Sitecore Digital Survivor Series - How Web and App Performance impacts Custom...

Dynatrace

What's Next in Financial Services Infrastructure Plumbing and APIs

Stephane Dubois

Show slides cio summit monaco vernon turner

bibhuashish

Dell Company Presentation

ssuser5a45b7

IDC Keynote: The Seven Transformations IT Solution Providers Must Confront

Ingram Micro Cloud

Thirty months of microservices. Stairway to heaven or highway to hell

This is the deck of the talks on microservices I did at both Avisi's #ASAS2016 (Arnhem, NL), Microsoft's #TechDaysNL (Amsterdam, NL) and #GeeCon (Prague, Czech Republic) conferences in September and October 2016. Microservices are the next hype. Websites are full of introducing posts, books are being written and conferences organized. There’s big promises of scalability and flexibility. However, when you are knee deep in mud as an architect, developer or tester, it’s hard to find out how to get there. Sander Hoogendoorn, independent craftsman and CTO of Klaverblad Insurances, discusses the long and winding road his projects, both greenfield and brownfield, have travelled. Sander will e.g. address polyglot persistence, DDD, bounded contexts, modeling HTTP/REST, continuous delivery and many lessons learned, using many real-life examples.

La Redoute Quality Engineering Transformation

Antoine Craske

Accelerate is becoming a standard to drive DevOps implementation on the four key metrics. We used the model to improve our pipelines over various increments, aligned with the DevOps principles. We act on various domains from methodology, organization, architecture to remove limiting factors along our software pipeline. Multiple teams are now delivering multiple times per day with stability, a game-changer compared to our previous performance across teams. This talk shares the journey we have been taking, lessons learned and takeaways to apply in your context.

Common Anti-patterns that Impede a Successful Digital Transformation

Gautham Pallapa

Lean, Agile, and DevOps are being used as tools to drive digital transformations across companies. The focus is heavily on technology, and less on people and processes, as the silver bullet to realizing their outcomes. This results in some anti-patterns that impede success of a digital transformation. In this session hosted by GitHub and Object Partners, I talk about the common transformational anti-patterns, and share approaches, experiments, and tips that could help.

the afterparty: refactoring after 100x hypergrowth

PicPay is the largest digital wallet in Latin America. We offer peer-to-peer payments, BNPL, credit cards, personal loans, insurance, investing, trading, and other financial products to 60 million people and 5 million merchants, processing over $20 billion yearly. But just three years ago, in 2019, PicPay was 30 engineers in a small office working on a single product for a few million users. Growing our product portfolio, team size, and the number of transactions by several orders of magnitude so quickly was as chaotic as you might imagine—but we've done it! Our approach gave teams and engineers radical autonomy to choose the best tool, process, and technology for their area. This strategy was invaluable in handling hypergrowth, but as we stabilize as a business and product, we have realized that what took us here isn't what will keep us moving forward. In this talk, let's explore the consequences of how we dealt with our hypergrowth phase and what are the changes and initiatives we have put in place to make sure that we keep growing and pushing the envelope—but at a manageable pace this time around. Phil Calçado is the Global CTO at PicPay. Before PicPay, Phil was Senior Director of Engineering at SeatGeek, leading the team that built the live events platform used by 44 million people worldwide. He has also led the platform team at Meetup/WeWork, worked on Linkerd - the pioneering Service Mesh, and headed product engineering for DigitalOcean and SoundCloud, both pioneers in adopting Microservices architectures. https://plus.qconferences.com/plus2022/presentation/afterparty-refactoring-after-100x-hypergrowth

don't try this at home: self-improvement as a senior leader

Presented at LeadingEng New York 2022 (https://leaddev.com/leadingeng-new-york/video/dont-try-home-how-practice-self-improvement-senior-leader) -- Most of us have developed our expertise as engineers through a mix of literature and experimentation. The software industry moves at a higher speed than others because we can try and study complex, real-world systems in the comfort of our bedrooms and personal Github accounts. When you become a leader, especially one who leads other leaders, things get a little more complicated. There are books, but they tend to contain generic advice. There are patterns, but the nature of the job makes it harder to identify when and how to apply them. In this session, I want to walk through a few things that have worked for me when you don't have a lab to try things out. Let's explore processes, tools, and resources to continuously improve your skills and expertise when there is no StackOverflow and no coding dojos. Phil Calçado is Global CTO at PicPay, the largest digital wallet in Latin America. He leads a team of more than 1,500 engineers building financial products for 60 million users and processing over four billion dollars yearly. Before PicPay, Phil was Senior Director of Engineering at SeatGeek, where he led the team that built the live events platform used by 44 million people worldwide. He also has led the platform team at Meetup/WeWork, worked on Linkerd - the pioneering Service Mesh, and headed product engineering for DigitalOcean and SoundCloud, both pioneers in the adoption of Microservices architectures.

Similar to From microservices to serverless - Chicago CTO Summit 2019

Taking an agile approach to the digital workplace

James Robertson

Welcome to Life in the Digital Vortex

InnoTech

Modeling Microservices

Multipathed, Multiplexed, Multilateral Transport Protocols - Decoupling trans...

APNIC

Microservices. The good the bad and the ugly

Why You Need Sensor Fusion In Your 2016 Retail Analytics Strategy

G3 Communications

The Future Of Information Technology: Welcome…to the Digital Age

Cisco Canada

DIGITAL POP MARKETING Optometrists PowerPoint

DIGITAL POP MARKETING Plumbing Services PowerPoint

Principles of Microservices - NDC 2014

Sam Newman

Migrate to-pg pgconfeu2019

Loxodata

Designing and building a micro-services architecture. Stairway to heaven or a...

Sitecore Digital Survivor Series - How Web and App Performance impacts Custom...

Dynatrace

What's Next in Financial Services Infrastructure Plumbing and APIs

Stephane Dubois

Show slides cio summit monaco vernon turner

bibhuashish

Dell Company Presentation

ssuser5a45b7

IDC Keynote: The Seven Transformations IT Solution Providers Must Confront

Ingram Micro Cloud

Thirty months of microservices. Stairway to heaven or highway to hell

La Redoute Quality Engineering Transformation

Antoine Craske

Common Anti-patterns that Impede a Successful Digital Transformation

Gautham Pallapa

Similar to From microservices to serverless - Chicago CTO Summit 2019 (20)

Taking an agile approach to the digital workplace

Welcome to Life in the Digital Vortex

Modeling Microservices

Multipathed, Multiplexed, Multilateral Transport Protocols - Decoupling trans...

Microservices. The good the bad and the ugly

Why You Need Sensor Fusion In Your 2016 Retail Analytics Strategy

The Future Of Information Technology: Welcome…to the Digital Age

DIGITAL POP MARKETING Optometrists PowerPoint

DIGITAL POP MARKETING Plumbing Services PowerPoint

Principles of Microservices - NDC 2014

Migrate to-pg pgconfeu2019

Designing and building a micro-services architecture. Stairway to heaven or a...

Sitecore Digital Survivor Series - How Web and App Performance impacts Custom...

What's Next in Financial Services Infrastructure Plumbing and APIs

Show slides cio summit monaco vernon turner

Dell Company Presentation

IDC Keynote: The Seven Transformations IT Solution Providers Must Confront

Thirty months of microservices. Stairway to heaven or highway to hell

La Redoute Quality Engineering Transformation

Common Anti-patterns that Impede a Successful Digital Transformation

More from Phil Calçado

the afterparty: refactoring after 100x hypergrowth

don't try this at home: self-improvement as a senior leader

Ten Years of Failing Microservices

The Next Generation of Microservices

How are microservices in 2017 different from how we used to build them at the beginning of the decade? More traditional Service-Oriented Architectures were defined by protocols and standards published and curated by industry consortiums. Knowledge of the architectural style usually called "microservices", on the other hand, is often in the form of patterns, cautionary tales, and tools extracted from real-world reports and software made available by organisations that have adopted this style. Almost ten years since the first wave of such reports, the landscape has changed considerably. Many hard challenges from the past have been eased or completely solved, and a lot of the custom software created by the microservices pioneers have been made off-the-shelf open source software. In this talk, Phil Calçado will contrast what we first found in the first generation of microservices architectures against the current generation's landscape. Let's talk about which previous common knowledge and patterns are deprecated, which ones are still active, and introduce some of the ones that have been recently added to our toolbox.

The Next Generation of Microservices — YOW 2017 Brisbane

Finagle @ SoundCloud

A Brief Talk On High-Performing Organisations

Rhein-Main Scala Enthusiasts — Your microservice as a Function

ScalaItaly 2015 - Your Microservice as a Function

SoundCloud's microservice architecture is built mostly in Scala, using Finagle as its distributed systems workhorse. Finagle is an RPC system for the JVM, and it is based on a pipes-and-filters architecture that maps very nicely to functional programming concepts of higher-order functions and combinators. Over the past few years we have found that it is extremely useful to go even a step further and think of microservices as functions themselves. In this talk let's explore how SoundCloud uses Scala and Finagle, and how we started thinking of a microservices architecture as a special case of a functional system.

An example of Future composition in a real appPhil Calçado

APIs: The Problems with Eating your Own Dog Food

SoundCloud's web and mobile properties are all built on top of our Public API. While building the user-interface atop a RESTful layer has proven itself a sound decision, the one-size-fits-all nature of a Public API is not ideal. When it comes to data transfer and HTTP resource modelling, each client has their own needs, and in the end hacks and workarounds have to be implemented in both clients and servers. Feature development also becomes complicated, with coordination between multiple teams required for every single little feature. SoundCloud is now moving to a different model, where clients have their own façade APIs, modelled after their core characteristics and needs. We are also using the architecture to break away from Conway’s Law and building more cross-functional, end-to-end teams. From: http://gotocon.com/berlin-2013/presentation/APIs:%20The%20Problems%20with%20Eating%20your%20Own%20Dog%20food To implement this, a lot of change in our architecture, tech stack and development processes were required. In this talk we will explore the challenges we had, the options we investigated and how ultimately SoundCloud decided to move forward.

Evolutionary Architecture at WorkPhil Calçado

Structuring apps in ScalaPhil Calçado

From a monolithic Ruby on Rails app to the JVMPhil Calçado

Applying Evolutionary Architecture on a Popular APIPhil Calçado

SoundCloud Masterclass on BrazilPhil Calçado

SpeakerConf: my findings in trying to use this functional programming busines...Phil Calçado

[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil UsersPhil Calçado

(v2.0) Better Functional Design Through Test-Driven Development