Making sense of streaming

•

0 likes•264 views

Streaming processes data continuously as it arrives, allowing for real-time analysis and response. Common streaming technologies include Apache Kafka for messaging, Apache Storm and Apache Flink for stream processing, and Apache Spark for both batch and stream processing. While Spark and Flink can handle both batch and stream workloads, Flink has emerged as the leading stream processor due to high throughput and exactly-once processing semantics.

Technology

@diego_pacheco
❏ Cat's Father
❏ Head of Software Architecture
❏ Agile Coach
❏ SOA/Microservices Expert
❏ DevOps Practitioner
❏ Speaker
❏ Author
diegopacheco
http://diego-pacheco.blogspot.com.br/
About me...
https://diegopacheco.github.io/

What's hot

Git hooks For PHP Developers

Umut IŞIK

Developing faster than ever (Liferay DEVCON 2017)

Sébastien Le Marchand

Il message-passing per più di dieci anni è stata la soluzione di riferimento per affrontare le sfide e l'implementazione di un sistema distribuito: problemi di rete, forte accoppiamento fra nodi, eterogeneità delle applicazioni (diversi linguaggi di programmazione). Questo talk tratterà le soluzioni per costruire un'architettura orientata ai servizi (SOA) ed effettuare message-passing in maniera performante ed affidabile; ripercorreremo insieme i concetti e le tecnologie principali che sono alla base dei sistemi distribuiti.

Modern Messaging for Distributed Systems

Andrea Rabbaglietti

Are you in the project which needs ten thousand auto-scaled docker containers on Kubernetes in a multi-regional AWS deployment? Right.. we also have never done that. Nevertheless, you still might need DevOps 'magic' to deliver those boring everyday IT projects on time and budget. I will tell our journey with DevOps at a small IT shop and share tools & practices we have tried and which of them were useful and which were total overkill. Ilgvars is founder & developer at FinoTech. He's been working in IT field for more than 15 years and still kind of enjoys it. Things he likes even more: cooking & eating, few sports activities and finally beer.

“Practical DevOps by a small team of devs” by Ilgvars Jēcis from FinoTech at...

DevClub_lv

The swiss knife of a word press developer

Ivelina Dimova

The JAMStack (Javascript, APIs, Markup).

Michael Brooks

DevOps is a large part of a company of any size. In the 9+ years that I have been a professional developer I have always taken an interest in DevOps and have been the "server person" for most of the teams I have been a part of. I would like to teach others how easy it is to implement modern tools to make their everyday development and development processes better. I will cover a range of topics from "Stop using WAMP/MAMP and start using Vagrant", "version control isn't renaming files", "Automate common tasks with shell scripts / command line PHP apps" and "From Vagrant to Production".

DevOps For Small Teams

Joe Ferguson

Abstract: Still using Maven as a build tool in your projects? If so, are you using it because you think it is a great tool, or just because you’ve always done it this way? Within 15 minutes we’ll share with you the reasons that made us ditch Maven and fall in love with our build tool again. Be aware though, after you’ve visited this quickie there is a big chance you’ll find yourself migrating all your projects tonight. Luckily, this is easier than you might think. Talk (quickie) of 15 minutes on 8th November 2018 at J-Fall 2018, the annual Dutch Java User Group (NLJUG) conference.

Why you should embrace Gradle and ditch Maven

GijsLeussink

How to survive continuous innovation - Sebastien Goasguen - DevOpsDays Tel Av...

DevOpsDays Tel Aviv

Grunt js and WordPress

WP Australia

The learning curve for REST API security is severe and unforgiving. Specifications promise infinite flexibility, habitually give old concepts new names, and almost seem designed to deliberately confuse. With an aggressive distaste for fancy terminology, this session delves into OAuth 2.0 with and without JWT for user identity; AWS-style security for B2B with API keys; and OAuth 2.0 Proof of Possession, which merges both into two-factor bliss. Using a baseline microservice architecture, the presentation compares them, with a heavy focus on the wire, showing actual HTTP messages and analyzing their impact on load and security. Starting with basic authentication and a brief intro to hashing and signing, this is the perfect session to align the whole team.

Chasing the RESTful Trinity - Client CLI and Documentation

Roberto Cortez

The Shape of Speed

Oliver Tse

Javascript toolkit

Marcos Vinícius

A major part of our lives is working safely with production - yet few organizations today are designing production to enable higher quality and end to end verification of the code we write and deploy. In this talk, we build on the foundation of great microservice architectures to include the first class design of testability as one of the most important artifacts that high velocity and high-quality teams should consider. In particular, we’ll explore what it’s like to build quality software with no development, QA, or staging environments. We'll include a deep dive into “verification in production” and what it really takes to build software that can safely be tested continuously in production. Let’s build developer happiness by *knowing* production is correct. Given at QCon New York June 2017

Production - Designing for Testability

Michael Bryzek

Github developing stack

Vicente Bolea

There is something about JavaScript - Choose Forum 2014

jbandi

Grunt understanding

Khalid Khan

Madison PHP 2015 - DevOps For Small Teams

Joe Ferguson

Website Speed :: Fox Valley Computing Professionals, September 2014

Straight North

The Web We Weave :: How Web Developers Think About Performance

Oliver Tse

What's hot (20)

Git hooks For PHP Developers

Developing faster than ever (Liferay DEVCON 2017)

Modern Messaging for Distributed Systems

“Practical DevOps by a small team of devs” by Ilgvars Jēcis from FinoTech at...

The swiss knife of a word press developer

The JAMStack (Javascript, APIs, Markup).

DevOps For Small Teams

Why you should embrace Gradle and ditch Maven

How to survive continuous innovation - Sebastien Goasguen - DevOpsDays Tel Av...

Grunt js and WordPress

Chasing the RESTful Trinity - Client CLI and Documentation

The Shape of Speed

Javascript toolkit

Production - Designing for Testability

Github developing stack

There is something about JavaScript - Choose Forum 2014

Grunt understanding

Madison PHP 2015 - DevOps For Small Teams

Website Speed :: Fox Valley Computing Professionals, September 2014

The Web We Weave :: How Web Developers Think About Performance

Recently uploaded

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place. Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects. Here’s what you’ll gain: - Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows. - Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy. - Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency. - Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity. We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic. Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.

Essentials of Automations: Optimizing FME Workflows with Parameters

Safe Software

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Julian Hyde

Assuring Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.

"Impact of front-end architecture on development cost", Viktor Turskyi

Fwdays

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Product School

Speed Wins: From Kafka to APIs in Minutes

confluent

Join us as we dive into the latest updates to the UiPath Orchestrator API, including new limits and features for 2024. Discover how these changes can enhance your automation projects and streamline your workflows. 📚 Overview of UiPath Orchestrator API 🔧 Recent changes to API limits 🛠️ How to adapt to new limits 📋 Best practices for using the Orchestrator API efficiently ❓ Q&A session

Exploring UiPath Orchestrator API: updates and limits in 2024 🚀

DianaGray10

IESVE for Early Stage Design and Planning

IES VE

Designing Great Products: The Power of Design and Leadership by Chief Designe...

Product School

Ever caught yourself nodding along when someone mentions "delivering value" in Agile, but secretly wondering what the heck they actually mean? You're not alone! Join us for an eye-opening session where we'll strip away the buzzwords and dive into the heart of Agile—value delivery. But what is "value"? Is it a mythical unicorn in the world of software development, or is there more to this overused term? This isn't going to be a sit-and-get lecture. We're talking about a face-to-face, interactive meetup where YOU play a crucial role. Come along to: Define It: What does "value" really mean? We’ll build a definition that’s not just words, but a compass for your Agile journey. Contextualise It: Discover what value means specifically to you, your team, your company, and your industry. Because one size does not fit all. Deliver It: Share strategies and gather new ones for uncovering and delivering true value—no more shooting in the dark!

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

David Michel

ODC, Data Fabric and Architecture User Group

CatarinaPereira64715

Bits & Pixels using AI for Good.........

Alison B. Lowndes

Knowledge engineering: from people to machines and back

Elena Simperl

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Product School

Discover the essentials of performance testing in the IT sector with our concise guide. Learn about various testing types such as load, stress, endurance, spike, scalability, and volume testing. Understand key performance metrics like response time, throughput, CPU and memory utilization, and error rate. Explore top tools like Apache JMeter, LoadRunner, Gatling, Neoload, and BlazeMeter. Gain insights into best practices for defining objectives, creating realistic scenarios, automating tests, and optimizing performance to ensure user satisfaction, reliability, scalability, and cost efficiency. Ideal for developers, QA engineers, and IT professionals. Visit Expeed Software for more information. https://expeed.com/

In-Depth Performance Testing Guide for IT Professionals

Expeed Software

Key Trends Shaping the Future of Infrastructure.pdf

Cheryl Hung

Intrigued by why some of the world's largest companies (Netflix, Google, Cisco, Twitter, Uber etc) are using gRPC? In this demo based talk we delve into the world of gRPC in .Net, what it does and why we should use it. We compare the interface with both Rest and graphQL. We will show you how to implement grpc server-side in .net and in the web. Finally, I will show you how the tooling helps you deliver powerful interfaces and interact with them quickly and simply.

Demystifying gRPC in .Net by John Staveley

John Staveley

As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other? Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

Tobias Schneck

Recently uploaded (20)

When stars align: studies in data quality, knowledge graphs, and machine lear...

Essentials of Automations: Optimizing FME Workflows with Parameters

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)

Assuring Contact Center Experiences for Your Customers With ThousandEyes

"Impact of front-end architecture on development cost", Viktor Turskyi

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Speed Wins: From Kafka to APIs in Minutes

Exploring UiPath Orchestrator API: updates and limits in 2024 🚀

IESVE for Early Stage Design and Planning

Designing Great Products: The Power of Design and Leadership by Chief Designe...

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx

ODC, Data Fabric and Architecture User Group

Bits & Pixels using AI for Good.........

Knowledge engineering: from people to machines and back

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

In-Depth Performance Testing Guide for IT Professionals

Key Trends Shaping the Future of Infrastructure.pdf

Demystifying gRPC in .Net by John Staveley

Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024

Making sense of streaming

1. Making Sense of Streaming Diego Pacheco

2. @diego_pacheco ❏ Cat's Father ❏ Head of Software Architecture ❏ Agile Coach ❏ SOA/Microservices Expert ❏ DevOps Practitioner ❏ Speaker ❏ Author diegopacheco http://diego-pacheco.blogspot.com.br/ About me... https://diegopacheco.github.io/

3. What the HECK is Big Data...

4. Streaming is Action @ Scale!

5. Numbers...

6. Hadoop ~ 2008

7. Storm ~ 2011

8. Lambda Architecture

9. Kappa Architecture

10. Spark ~ 2014

11. Flink ~ 2011

12. Flink ~ 2011

13. Pattern: Kafka + Stream + Cassandra

14. Spark vs Flink