An example of Future composition in a real app

•

5 likes•3,782 views

Phil Calçado

Technology Education

api.soundcloud.com
Search
Msgs Metadata
Groups
Stats
Playlists

api.soundcloud.com
MetadataPlaylists Search
Social˝
Graph
Messages StatsGroups

example
50 or so 10-40ms requests
total response time is 152ms

challenge:
interesting
how to detect/type missed
concurrency opportunities?˝
aka. for comprehension abuse

phil calçado
http://philcalcado.com˝
@pcalcado˝
www.soundcloud.com˝

PicPay is the largest digital wallet in Latin America. We offer peer-to-peer payments, BNPL, credit cards, personal loans, insurance, investing, trading, and other financial products to 60 million people and 5 million merchants, processing over $20 billion yearly. But just three years ago, in 2019, PicPay was 30 engineers in a small office working on a single product for a few million users. Growing our product portfolio, team size, and the number of transactions by several orders of magnitude so quickly was as chaotic as you might imagine—but we've done it! Our approach gave teams and engineers radical autonomy to choose the best tool, process, and technology for their area. This strategy was invaluable in handling hypergrowth, but as we stabilize as a business and product, we have realized that what took us here isn't what will keep us moving forward. In this talk, let's explore the consequences of how we dealt with our hypergrowth phase and what are the changes and initiatives we have put in place to make sure that we keep growing and pushing the envelope—but at a manageable pace this time around. Phil Calçado is the Global CTO at PicPay. Before PicPay, Phil was Senior Director of Engineering at SeatGeek, leading the team that built the live events platform used by 44 million people worldwide. He has also led the platform team at Meetup/WeWork, worked on Linkerd - the pioneering Service Mesh, and headed product engineering for DigitalOcean and SoundCloud, both pioneers in adopting Microservices architectures. https://plus.qconferences.com/plus2022/presentation/afterparty-refactoring-after-100x-hypergrowth

Presented at LeadingEng New York 2022 (https://leaddev.com/leadingeng-new-york/video/dont-try-home-how-practice-self-improvement-senior-leader) -- Most of us have developed our expertise as engineers through a mix of literature and experimentation. The software industry moves at a higher speed than others because we can try and study complex, real-world systems in the comfort of our bedrooms and personal Github accounts. When you become a leader, especially one who leads other leaders, things get a little more complicated. There are books, but they tend to contain generic advice. There are patterns, but the nature of the job makes it harder to identify when and how to apply them. In this session, I want to walk through a few things that have worked for me when you don't have a lab to try things out. Let's explore processes, tools, and resources to continuously improve your skills and expertise when there is no StackOverflow and no coding dojos. Phil Calçado is Global CTO at PicPay, the largest digital wallet in Latin America. He leads a team of more than 1,500 engineers building financial products for 60 million users and processing over four billion dollars yearly. Before PicPay, Phil was Senior Director of Engineering at SeatGeek, where he led the team that built the live events platform used by 44 million people worldwide. He also has led the platform team at Meetup/WeWork, worked on Linkerd - the pioneering Service Mesh, and headed product engineering for DigitalOcean and SoundCloud, both pioneers in the adoption of Microservices architectures.

The Economics of Microservices (redux)

Phil Calçado

From microservices to serverless - Chicago CTO Summit 2019

Phil Calçado

The Not-So-Straightforward Road From Microservices to Serverless

Phil Calçado

For the last ten years or so, many companies have focused on migrating from larger, monolithic systems and applications towards a specific style of Service-Oriented Architecture called Microservices. The promise was that these smaller, loosely-coupled, and independently developed components would increase productivity and safety for organizations, as large and complex business challenges can be broken down into smaller and simpler components. However, even before most organizations were able to fully migrate to this new architecture and enjoy its promised benefits, a new iteration of cloud computing has been made available in the shape of Serverless platforms such as AWS Lambda, and Google Cloud and Microsoft Azure Functions. Now lots of companies found themselves conflicted between abandoning their traditional microservices approach towards Serverless, often even before they have fully migrated to it from the legacy systems. After leading successful Microservices adoption at SoundCloud and DigitalOcean, recently at Meetup I have faced the challenge of pivoting from a traditional monolith-to-microservices migration to a cloud-native platform. In this talk, let's discuss the fundamental concepts, technologies, and practices behind Microservices and Serverless, and how a software architect used to distributed systems based on microservices needs to change their mindset and approach when adopting Serverless.

Ten Years of Failing Microservices

Phil Calçado

The Next Generation of Microservices

Phil Calçado

How are microservices in 2017 different from how we used to build them at the beginning of the decade? More traditional Service-Oriented Architectures were defined by protocols and standards published and curated by industry consortiums. Knowledge of the architectural style usually called "microservices", on the other hand, is often in the form of patterns, cautionary tales, and tools extracted from real-world reports and software made available by organisations that have adopted this style. Almost ten years since the first wave of such reports, the landscape has changed considerably. Many hard challenges from the past have been eased or completely solved, and a lot of the custom software created by the microservices pioneers have been made off-the-shelf open source software. In this talk, Phil Calçado will contrast what we first found in the first generation of microservices architectures against the current generation's landscape. Let's talk about which previous common knowledge and patterns are deprecated, which ones are still active, and introduce some of the ones that have been recently added to our toolbox.

The Next Generation of Microservices — YOW 2017 Brisbane

Phil Calçado

The Economics of Microservices (2017 CraftConf)

Phil Calçado

Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...

Phil Calçado

TALK #2: Microservices vs. The First Law of Object Design We've been breaking systems and application into smaller components for a long time now. From Component-Based Design to Distributed Objects to SOA to what is today's preferred golden hammer: microservices. One definition of microservices is that it is a flavor of SOA that emphasizes many specialize services versus a few more generalist ones. Often these microservices are so small that they take care of a single "object". Distributed objects aren't new to this industry, and in 2003, Martin Fowler wrote a classic article where he discusses several problems with this model, and proposes the First Law of Distributed Objects: "Objects have been around for a while, and sometimes it seems that ever since they were created, folks have wanted to distribute them. However, distribution of objects, or indeed of anything else, has a lot more pitfalls than many people realize, especially when they're under the influence of vendors' cozy brochures. This article is about some of these hard lessons-lessons I've seen many of my clients learn the hard way... my First Law of Distributed Object Design: Don’t distribute your objects!" Reinventing the wheel is nothing new in our field, but if microservices are meant to be small, how can we avoid the same problems from the past? What are the technologies, architectures, protocols, and practices we need in place to make sure that our microservices architecture isn't just the largest bowl of spaghetti this organization has ever cooked? SPEAKER: Phil Calçado, Director of Software Engineering at DigitalOcean Phil Calçado works at DigitalOcean, where he helps build the cloud for developers. Before that, he spent four years building the team and architecture behind SoundCloud's move from a monolith to microservices. He tweets at @pcalcado writes at http://philcalcado.com.

Finagle @ SoundCloud

Phil Calçado

A Brief Talk On High-Performing Organisations

Phil Calçado

Three Years of Microservices at SoundCloud - Distributed Matters Berlin 2015

Phil Calçado

SoundCloud is the largest repository of audio on the web, used by more than 200 million people every month, who upload more than 11 hours of audio every minute. Like so many others, we have migrated from a typical monolithic architecture to microservices. While the benefits brought by this style of SOA to our productivity and reliability are clear, the architecture required some non-obvious changes in the way we operate systems, and a way to tackle the overhead associated with having hundreds of small moving parts to serve every request. In this talk we’ll share the toolkit and strategy SoundCloud uses to keep its microservices explosion manageable. What do we do about the operations overhead? How to spread devops skills across teams to support the “you build it, you run it” vision? How to deal with breaking changes and asynchronous behaviours? How to deal with chatty interactions? Which protocol? How do I even get a diagram telling me how all this stuff is put together?

Rhein-Main Scala Enthusiasts — Your microservice as a Function

Phil Calçado

ScalaItaly 2015 - Your Microservice as a Function

Phil Calçado

SoundCloud's microservice architecture is built mostly in Scala, using Finagle as its distributed systems workhorse. Finagle is an RPC system for the JVM, and it is based on a pipes-and-filters architecture that maps very nicely to functional programming concepts of higher-order functions and combinators. Over the past few years we have found that it is extremely useful to go even a step further and think of microservices as functions themselves. In this talk let's explore how SoundCloud uses Scala and Finagle, and how we started thinking of a microservices architecture as a special case of a functional system.

Finagle-Based Microservices at SoundCloud

Phil Calçado

APIs: The Problems with Eating your Own Dog Food

Phil Calçado

SoundCloud's web and mobile properties are all built on top of our Public API. While building the user-interface atop a RESTful layer has proven itself a sound decision, the one-size-fits-all nature of a Public API is not ideal. When it comes to data transfer and HTTP resource modelling, each client has their own needs, and in the end hacks and workarounds have to be implemented in both clients and servers. Feature development also becomes complicated, with coordination between multiple teams required for every single little feature. SoundCloud is now moving to a different model, where clients have their own façade APIs, modelled after their core characteristics and needs. We are also using the architecture to break away from Conway’s Law and building more cross-functional, end-to-end teams. From: http://gotocon.com/berlin-2013/presentation/APIs:%20The%20Problems%20with%20Eating%20your%20Own%20Dog%20food To implement this, a lot of change in our architecture, tech stack and development processes were required. In this talk we will explore the challenges we had, the options we investigated and how ultimately SoundCloud decided to move forward.

Evolutionary Architecture at WorkPhil Calçado

Structuring apps in ScalaPhil Calçado

From a monolithic Ruby on Rails app to the JVMPhil Calçado

Applying Evolutionary Architecture on a Popular APIPhil Calçado

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Product School

Recently uploaded

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Product School

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

The Future of Platform Engineering

Jemma Hussein Allen

UiPath Test Automation using UiPath Test Suite series, part 3

DianaGray10

JMeter webinar - integration with InfluxDB and Grafana

RTTS

Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application. In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics. Length: 30 minutes Session Overview ------------------------------------------- During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana: - What out-of-the-box solutions are available for real-time monitoring JMeter tests? - What are the benefits of integrating InfluxDB and Grafana into the load testing stack? - Which features are provided by Grafana? - Demonstration of InfluxDB and Grafana using a practice web application To view the webinar recording, go to: https://www.rttsweb.com/jmeter-integration-webinar

GraphRAG is All You need? LLM & Knowledge Graph

Guy Korland

Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs. 1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. https://arxiv.org/abs/2306.08302 2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs: https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Product School

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

Neuro-symbolic is not enough, we need neuro-*semantic*

Frank van Harmelen

Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”. All of this illustrated with link prediction over knowledge graphs, but the argument is general.

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams. Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

Search and Society: Reimagining Information Access for Radical Futures

Bhaskar Mitra

The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

Abida Shariff

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

How world-class product teams are winning in the AI era by CEO and Founder, P...

Product School

Accelerate your Kubernetes clusters with Varnish Caching

Thijs Feryn

ODC, Data Fabric and Architecture User Group

CatarinaPereira64715

Connector Corner: Automate dynamic content and events by pushing a button

DianaGray10

Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to: Create a campaign using Mailchimp with merge tags/fields Send an interactive Slack channel message (using buttons) Have the message received by managers and peers along with a test email for review But there’s more: In a second workflow supporting the same use case, you’ll see: Your campaign sent to target colleagues for approval If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team But—if the “Reject” button is pushed, colleagues will be alerted via Slack message Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors. And... Speakers: Akshay Agnihotri, Product Manager Charlie Greenberg, Host

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Ramesh Iyer

In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.