Webinar: Queues with RabbitMQ - Lorna Mitchell

•

1 like•646 views

Queues are a great addition to any application that has some tasks that need processing asynchronously. This could be sending a confirmation email, resizing an avatar, or recalculating a running total of some kind; in all those cases it would be cool to send the response back to the user and then sort out that task later. This session looks at how to use a RabbitMQ job queue in your application. It also looks at how to design elegant and robust long-running workers that will consume the jobs from the queue and process them. This session is ideal for technical leads, developers and architects alike.

Technology

Queues With
RabbitMQ

Lorna Mitchell, IBM

Introducing Queues
Use queues to:
• asynchronously process tasks in your (existing) application
• e.g. sending email, processing uploads
• provide loose coupling points
• e.g. delegate heavy tasks, split ownership/technology
• enable parts of the system to scale appropriately
• e.g. an event in one system causes a thousand other
actions
@lornajane

Queue Tools
A selection of queue-ish tools
• RabbitMQ* http://www.rabbitmq.com/
• Gearman http://gearman.org/
• Beanstalkd http://kr.github.io/beanstalkd/
• Kafka https://kafka.apache.org/
• Redis https://redis.io/

* RabbitMQ is used in today's examples
@lornajane

Management Plugin
https://www.rabbitmq.com/management.html
@lornajane

Getting To Know RabbitMQ
Some vocabulary
• broker: the RabbitMQ instance
• exchange: where to send the messages to
• queue: where messages wait to be processed
• binding key: rules for which messages go into this queue
• message: the data to be processed
• routing key: message route information
• consumer: worker script to processes the messages
@lornajane

RabbitMQ Exchanges
Exchanges are the routing logic of RabbitMQ

Messages go to exchanges and the exchanges put them into
the correct queues for storage
@lornajane

Types Of Exchange
Direct: a given routing key puts messages onto the matching
queue(s)

Topic: queues are bound by key, and messages are routed to
as many queues as their routing key matches

Fanout: messages go to all queues bound to this exchange
@lornajane

Default Exchange
There is a default exchange in RabbitMQ

Its name is '' and routing is on queue name
@lornajane

RabbitMQ Queues
Queues can:
• have wildcards in binding keys
• be durable (messages have their own durability)
• have a maximum length
• be configured with a "dead letter exchange"

If producer needs feedback, create another queue to put
status updates into
@lornajane

Example App: Webhooks
(code: https://github.com/ibm-watson-data-lab/guestbook )

A simple guestbook application is extended to allow webhook
notifications of new comments.
@lornajane

Creating Workers
Workers are disposable!

• if things go wrong, exit
• separate tool to monitor/restart as needed
• beware long-running process hazards
• everything processed "at least once" (but maybe more than
once, and in any order...)
@lornajane

Independent Workers
For best results:
• awesome, aggregated logging
• monitoring: queue size, worker uptime, processing time
• miniminum viable dependencies
@lornajane

Completed Messages
• acknowledge when messages are processed successfully
• can acknowledge failure
• reject the message
• optionally: requeue
@lornajane

Dead Letter Queues
• reject without requeue
• exceed TTL
• queue length exceeded
@lornajane

Retries
Implement your own logic to handle retries

Create a new message with:
• all the existing message contents
• plus some metadata such as retry count or backoff time
@lornajane

Feedback Mechanisms
Rabbit is fire-and-forget; work is delegated

Common pattern: return queue to put updates into for the
original producer then to consume.
@lornajane

Queues With RabbitMQ

Queues are awesome for scalability and robustness

RabbitMQ is open source, lightweight and fast

Queues help us meet the requirements for modern
applications
@lornajane

Thanks!
Resources:
• RabbitMQ: https://www.rabbitmq.com/
• Resilience: https://lrnja.net/rabbitfail
• Blog: https://lornajane.net
• Code: https://github.com/ibm-watson-data-lab/guestbook

• @lornajane
• lorna.mitchell@uk.ibm.com
@lornajane

Isaac Newton, the father of modern software engineering, called it “Standing on the shoulders of giants”. Modern development is exciting as it gets easier and easier, partly because of the wealth of resources available at our fingertips. One category of these resources are libraries, SDKs, and frameworks. This talk will be a guide into the considerations that go into building a library for both iOS/Swift and Java/Android. We will be taking cues from both my personal experience, as well as from studying how the leaders in the field do it.

Microservices: 5 things I wish I'd known - Vincent Kok - Codemotion Amsterdam...

Codemotion

Microservices are hot! A lot of companies are experimenting with this architectural pattern that greatly benefits the software development process. When adopting new patterns we always encounter that moment where we think 'if only I knew this three months ago'. This talk will be a sneak peak into the world of microservices at Atlassian and reveal what we've learned about microservices: how to arrange, configure and build your code efficiently; deployment and testing; and how to operate effectively in this environment. In this talk you will learn how to immediately apply five simple strategies.

Webinar - Big Data: Let's SMACK - Jorg Schad

Codemotion

For many use cases such as fraud detection or reacting on sensor data the response times of traditional batch processing are simply to slow. In order to be able to react to such events close to real-time, we need to go beyond the classical batch processing and utilize stream processing systems such as Apache Spark Streaming, Apache Flink, or Apache Storm. But these systems are not sufficient by itself. One common example for such fast data pipelines is the SMACK stack using Apache Spark, Mesos, Kafka, Akka, Cassandra, Kafka.

Mario Cartia - SMACK is the new LAMP! - Codemotion Milan 2017

Codemotion

SMACK è l'acronimo di Spark, Mesos, Akka, Cassandra e Kafka. Il titolo del talk "provocatoriamente" confronta lo stack di tecnologie per lo sviluppo di applicazioni Reactive con quello più comunemente utilizzato nell'ambito dello sviluppo web. Durante il talk verranno illustrati i concetti di base della Reactive programming, le differenze concettuali introdotte da questo paradigma rispetto all'approccio "classico" della programmazione web ed alcuni casi di successo legati all'utilizzo di queste tecnologie.

How to build an HA container orchestrator infrastructure for production – Giu...

Codemotion

Serverless in production, an experience report (codemotion milan)

Yan Cui

AWS Lambda has changed the way we deploy and run software, but the serverless paradigm has created new challenges to old problems: How do you test a cloud-hosted function locally? How do you monitor them? What about logging and config management? And how do we start migrating from existing architectures? Yan Cui shares solutions to these challenges, drawing on his experience running Lambda in production and migrating from an existing monolithic architecture.

Automated Scaling of Microservice Stacks for JavaEE Applications

Jelastic Multi-Cloud PaaS

PyCon India 2012: Celery Talk

Piyush Kumar

Lambda is the next stage in the evolution of the AWS platform. It allows you to build reactive, event-driven systems that are easy to deploy, update and scale. Amazon manages all the undifferentiated heavy-lifting for you so you can focus on delivering value to your customers with even greater speed and cost efficiency. Join Yan in this talk as we take a deep dive through AWS Lambda and the Serverless framework. We'll see how to start building reactive systems using AWS Lambda, Kinesis and API Gateway, without having to manage any servers. And, you only pay for your services when they are used. We'll discuss lessons learned, best practices and current limitations with AWS Lambda. We'll also get to know the Serverless framework, which helps automate both deployment and versioning so that you can better focus on the things that matter to your customers.

Exactly-once Semantics in Apache Kafka

confluent

Apache Kafka's rise in popularity as a streaming platform has demanded a revisit of its traditional at-least-once message delivery semantics. In this talk, we present the recent additions to Kafka to achieve exactly-once semantics (EoS) including support for idempotence and transactions in the Kafka clients. The main focus will be the specific semantics that Kafka distributed transactions enable and the underlying mechanics which allow them to scale efficiently.

Beyond Fault Tolerance with Actor Programming

Fabio Tiriticco

Actor Programming is a software building approach that lets you can go beyond fault tolerance and achieve Resilience, which is the capacity of a system to self-heal and spring back into a fresh shape. First I'll introduce the difference between Reactive Programming and Reactive Systems, and then we'll go over a couple of implementation examples using Scala and Akka. The coupled GitHub repository with the code is here: https://github.com/ticofab/ActorDemo

Docker Cambridge: Serverless Functions Made Simple with OpenFaaS

Alex Ellis

JUST EAT: Tools we use to enable our culture

Peter Mounce

Building Micro-Services with Scala

Yardena Meymann

"Microservices" is one of the hottest buzzwords and, as usual, everyone wants them, but few know how to build them. In this talk we will offer our interpretation of microservice architecture, and show how we are implementing these ideas: using Scala, Akka, sbt and Docker, we modularized Akka applications, Spark jobs and Play servers. In the talk we will discuss design trade-offs and challenges that we faced in the process, and how we have overcome them. The focus is not on particular features of Scala language or a library, but on building modern applications using the Typesafe stack and other open-source tools.

Spark Summit Europe: Building a REST Job Server for interactive Spark as a se...

gethue

Livy is a new open source Spark REST Server for submitting and interacting with your Spark jobs from anywhere. Livy is conceptually based on the incredibly popular IPython/Jupyter, but implemented to better integrate into the Hadoop ecosystem with multi users. Spark can now be offered as a service to anyone in a simple way: Spark shells in Python or Scala can be ran by Livy in the cluster while the end user is manipulating them at his own convenience through a REST api. Regular non-interactive applications can also be submitted. The output of the jobs can be introspected and returned in a tabular format, which makes it visualizable in charts. Livy can point to a unique Spark cluster and create several contexts by users. With YARN impersonation, jobs will be executed with the actual permissions of the users submitting them. Livy also enables the development of Spark Notebook applications. Those are ideal for quickly doing interactive Spark visualizations and collaboration from a Web browser! This talk is technical and details the architecture and design decisions taken for developing this server, as well as its internals. It also describes the alternatives we tried and the challenges that were faced. The capabilities of Livy will then be lived demo in Hue’s Notebook Application through a real life scenario. https://spark-summit.org/eu-2015/events/building-a-rest-job-server-for-interactive-spark-as-a-service/

Building a Raspberry Pi cluster with Kubernetes, OpenFaaS and .NET

Alex Ellis

How to Build Your First Web App in Go

All Things Open

Dev-Friendly Ops

Josh Schramm

Build reactive systems on lambda

Yan Cui

Webinar: Giovanni Laquidara - Creare applicazioni VR

Codemotion

Webinar: Mario Cartia - Facciamo il Punto su Presente e Futuro dei framework ...

Codemotion

Poco più di 10 anni fa nasceva il framework opensource Hadoop rivoluzionando il mondo dei Big Data. Nato come implementazione opensource di due tecnologie di Google, MapReduce e GFS, metteva a disposizione di tutti strumenti per l'analisi di grandi quantità di dati che fino ad allora erano stati appannaggio del colosso di Mountain View. Qualche anno dopo presso l'AMPLab dell'Università di Berkeley nasceva un altro framework che avrebbe rivoluzionato il settore: Spark. Durante il webinar verrà analizzato lo "stato dell'arte" di questi due prodotti oggi nonché le features che verranno introdotte a breve disponibili attualmente in modalità "sperimentale" nelle versioni in fase di sviluppo.

What's hot

OpenFaaS JeffConf 2017 - Milan

Alex Ellis

Easily create dashboards to manage your databases with OVH

OVHcloud

Mini-Training: Netflix Simian Army

Betclic Everest Group Tech Team

Testing at Stream-Scale

All Things Open

Continuous Delivery and Infrastructure as Code

Sascha Möllering

Celery

Yipit

CNCF Intro & Demo - OpenFaaS framework

Alex Ellis

Zero to Serverless - OpenFaaS at the Open Source Summit

Alex Ellis

Celery workshop

Eswar Vandanapu

The future of paas is serverless

Yan Cui

Exactly-once Semantics in Apache Kafka

confluent

Beyond Fault Tolerance with Actor Programming

Fabio Tiriticco

Docker Cambridge: Serverless Functions Made Simple with OpenFaaS

Alex Ellis

JUST EAT: Tools we use to enable our culture

Peter Mounce

Building Micro-Services with Scala

Yardena Meymann

Spark Summit Europe: Building a REST Job Server for interactive Spark as a se...

gethue

Building a Raspberry Pi cluster with Kubernetes, OpenFaaS and .NET

Alex Ellis

How to Build Your First Web App in Go

All Things Open

Dev-Friendly Ops

Josh Schramm

Build reactive systems on lambda

Yan Cui

What's hot (20)

OpenFaaS JeffConf 2017 - Milan

Easily create dashboards to manage your databases with OVH

Mini-Training: Netflix Simian Army

Testing at Stream-Scale

Continuous Delivery and Infrastructure as Code

Celery

CNCF Intro & Demo - OpenFaaS framework

Zero to Serverless - OpenFaaS at the Open Source Summit

Celery workshop

The future of paas is serverless

Exactly-once Semantics in Apache Kafka

Beyond Fault Tolerance with Actor Programming

Docker Cambridge: Serverless Functions Made Simple with OpenFaaS

JUST EAT: Tools we use to enable our culture

Building Micro-Services with Scala

Spark Summit Europe: Building a REST Job Server for interactive Spark as a se...

Building a Raspberry Pi cluster with Kubernetes, OpenFaaS and .NET

How to Build Your First Web App in Go

Dev-Friendly Ops

Build reactive systems on lambda

Viewers also liked

Webinar: Giovanni Laquidara - Creare applicazioni VR

Codemotion

Webinar: Mario Cartia - Facciamo il Punto su Presente e Futuro dei framework ...

Codemotion

Webinar - Matteo Manchi: Dal web al nativo: Introduzione a React Native

Codemotion

Capturing the human body motion with inertial sensors, the Open Source way - ...

Codemotion

Motion capture devices have historically been prohibitively expensive, and their use was restricted to military and scientific institutions, and some audiovisual companies. In the last years MEMS based inertial sensors have experienced significant development on their precision while decreasing size and cost. In combination with the appearance and spread of some computational inexpensive sensor fusion algorithms and fast microcomputers, motion capture at domestic level is now possible. In this talk we’ll review the essential techniques to it by focusing on a system developed by the author.

Microservices Without Servers - James Thomas - Codemotion Amsterdam 2017

Codemotion

Servers kill your productivity. Rather than building software, you end up maintaining computers. Wasn't the "cloud" supposed to fix this? It sounded so promising until we realised it was just renting VMs in a remote datacenter. We couldn't escape "servers". Until now! In this session, we'll learn how to build microservices without servers, using modern serverless platforms. We’ll look at common challenges (and solutions) to building applications using serverless stacks. Exploring serverless design patterns will give developers the knowledge to build architectures using these new platforms.

Functional programming is not just lambdas! - David Sferruzza - Codemotion Am...

Codemotion

Functional programming is a trending topic, especially in web development. People often try to explain its essence with too theoretical concepts (monads) or too superficial points of view (it's just lambda functions). I will help you understand and feel what it really is and why it matters in the real world. We will see fundamental functional programming concepts, and apply them with JavaScript to get better code. Writing good code means spending less time to fix bugs or add features!

Machine learning from a software engineer's perspective - Marijn van Zelst - ...

Codemotion

Lot's of software engineers seem to avoid the field of machine learning because it seems hard. In this talk I want to give developers an intuition of what machine learning is using visual examples and without using mathematical formulas. I want to show that machine learning will make things possible that cannot be achieved using traditional procedural programming. I will identify high level components of a supervised machine learning algorithm: vectors, feature spaces, neural networks and labels.

Downtime is not an option - day 2 operations - Jörg Schad

Codemotion

Composable architectures The Lego of IT - Alessandro David

Codemotion

Viewers also liked (9)

Webinar: Giovanni Laquidara - Creare applicazioni VR

Webinar: Mario Cartia - Facciamo il Punto su Presente e Futuro dei framework ...

Webinar - Matteo Manchi: Dal web al nativo: Introduzione a React Native

Capturing the human body motion with inertial sensors, the Open Source way - ...

Microservices Without Servers - James Thomas - Codemotion Amsterdam 2017

Functional programming is not just lambdas! - David Sferruzza - Codemotion Am...

Machine learning from a software engineer's perspective - Marijn van Zelst - ...

Downtime is not an option - day 2 operations - Jörg Schad

Composable architectures The Lego of IT - Alessandro David

Similar to Webinar: Queues with RabbitMQ - Lorna Mitchell

Parallel and Asynchronous Programming - ITProDevConnections 2012 (English)

Panagiotis Kanavos

Life in a Queue - Using Message Queue with django

Tareque Hossain

Brief introduction on message queue and how its relevant in web applications How to tell if your web application could benefit from message queue Common example of tasks that could benefit from message queues Choosing a broker/protocol What broker/protocol PBS Education chose and why Message queue solution architecture Brief introduction on celery/carrot Writing a message queue task using celery How to invoke a message queue taks What happens when you invoke a task (walk through architecture) How to write tasks efficiently What are the things that are good to know when writing tasks (things we experienced at PBS Education)

Parallel and Asynchronous Programming - ITProDevConnections 2012 (Greek)

Panagiotis Kanavos

How we use Twisted in Launchpad

Michael Hudson-Doyle

John adams talk cloudy

John Adams

Rooster Tech Talk

Chris Johnson

Follow the White Rabbit - Message Queues with PHP

Eric Rodriguez (Hiring in Lex)

An introduction to message queues with PHP. We'll focus on RabbitMQ and how to leverage queuing scenarios in your applications. The talk will cover the main concepts of RabbitMQ server and AMQP protocol and show how to use it in PHP. The RabbitMqBundle for Symfony2 will be presented and we'll see how easy you can start to use message queuing in minutes. Presented at Symfony User Group Belgium: http://www.meetup.com/Symfony-User-Group-Belgium/events/169953362/

Virtual Flink Forward 2020: How Streaming Helps Your Staging Environment and ...

Flink Forward

In this session, we will look at how Apache Flink can be used to stream anonymized API request and response data from a production environment to make sure staging environments are up-to-date and reflect the most recent features (and bugs) that comprise a service. The talk will also examine how to deal with issues of data retention, throttling, and persistence, finishing with recommendations for how to use these sandbox environments to rapidly prototype and test new features and fixes.

Building production spark streaming applications

Joey Echeverria

Designing, implementing, and testing an Apache Spark Streaming application is necessary to deploy to production but is not sufficient for long term management and monitoring. Simply learning the Spark Streaming APIs only gets you part of the way there. In this talk, I’ll be focusing on everything that happens after you’ve implemented your application in the context of a real-time alerting system for IT operational data.

Cooking a rabbit pieTomas Doran

Building Awesome APIs with Lumen

Kit Brennan

Real time system_performance_monTomas Doran

Being Ready for Apache Kafka - Apache: Big Data Europe 2015

Michael Noll

These are the slides of my Kafka talk at Apache: Big Data Europe in Budapest, Hungary. Enjoy! --Michael Apache Kafka is a high-throughput distributed messaging system that has become a mission-critical infrastructure component for modern data platforms. Kafka is used across a wide range of industries by thousands of companies such as Twitter, Netflix, Cisco, PayPal, and many others. After a brief introduction to Kafka this talk will provide an update on the growth and status of the Kafka project community. Rest of the talk will focus on walking the audience through what's required to put Kafka in production. We’ll give an overview of the current ecosystem of Kafka, including: client libraries for creating your own apps; operational tools; peripheral components required for running Kafka in production and for integration with other systems like Hadoop. We will cover the upcoming project roadmap, which adds key features to make Kafka even more convenient to use and more robust in production.

Apache Kafka 0.8 basic training - Verisign

Michael Noll

Apache Kafka 0.8 basic training (120 slides) covering: 1. Introducing Kafka: history, Kafka at LinkedIn, Kafka adoption in the industry, why Kafka 2. Kafka core concepts: topics, partitions, replicas, producers, consumers, brokers 3. Operating Kafka: architecture, hardware specs, deploying, monitoring, P&S tuning 4. Developing Kafka apps: writing to Kafka, reading from Kafka, testing, serialization, compression, example apps 5. Playing with Kafka using Wirbelsturm Audience: developers, operations, architects Created by Michael G. Noll, Data Architect, Verisign, https://www.verisigninc.com/ Verisign is a global leader in domain names and internet security. Tools mentioned: - Wirbelsturm (https://github.com/miguno/wirbelsturm) - kafka-storm-starter (https://github.com/miguno/kafka-storm-starter) Blog post at: http://www.michael-noll.com/blog/2014/08/18/apache-kafka-training-deck-and-tutorial/ Many thanks to the LinkedIn Engineering team (the creators of Kafka) and the Apache Kafka open source community!

Lessons Learned: Using Spark and Microservices

Alexis Seigneurin

3.2 Streaming and Messaging

振东刘

Akka in Production - ScalaDays 2015

Evan Chan

Everyone in the Scala world is using or looking into using Akka for low-latency, scalable, distributed or concurrent systems. I'd like to share my story of developing and productionizing multiple Akka apps, including low-latency ingestion and real-time processing systems, and Spark-based applications. When does one use actors vs futures? Can we use Akka with, or in place of, Storm? How did we set up instrumentation and monitoring in production? How does one use VisualVM to debug Akka apps in production? What happens if the mailbox gets full? What is our Akka stack like? I will share best practices for building Akka and Scala apps, pitfalls and things we'd like to avoid, and a vision of where we would like to go for ideal Akka monitoring, instrumentation, and debugging facilities. Plus backpressure and at-least-once processing.

Realtime traffic analyser

Alex Moskvin

Performance optimization - JavaScriptFilip Mares

Message Queues in Ruby - An Overview

Pradeep Elankumaran

Similar to Webinar: Queues with RabbitMQ - Lorna Mitchell (20)

Parallel and Asynchronous Programming - ITProDevConnections 2012 (English)

Life in a Queue - Using Message Queue with django

Parallel and Asynchronous Programming - ITProDevConnections 2012 (Greek)

How we use Twisted in Launchpad

John adams talk cloudy

Rooster Tech Talk

Follow the White Rabbit - Message Queues with PHP

Virtual Flink Forward 2020: How Streaming Helps Your Staging Environment and ...

Building production spark streaming applications

Cooking a rabbit pie

Building Awesome APIs with Lumen

Real time system_performance_mon

Being Ready for Apache Kafka - Apache: Big Data Europe 2015

Apache Kafka 0.8 basic training - Verisign

Lessons Learned: Using Spark and Microservices

3.2 Streaming and Messaging

Akka in Production - ScalaDays 2015

Realtime traffic analyser

Performance optimization - JavaScript

Message Queues in Ruby - An Overview

More from Codemotion

Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...

Codemotion

Pompili - From hero to_zero: The FatalNoise neverending story

Codemotion

Pastore - Commodore 65 - La storia

Codemotion

Pennisi - Essere Richard Altwasser

Codemotion

Rivivere l'ebbrezza di progettare un vecchio computer o una consolle da bar è oggi possibile sfruttando le FPGA, ovvero logiche programmabili che consentono a chiunque di progettare il proprio hardware o di ricrearne uno del passato. In questa sessione si racconta come dal reverse engineering dell'hardware di vecchie glorie come il Commodore 64 e lo ZX Spectrum sia stato possibile farle rivivere attraverso tecnologie oggi alla portata di tutti.

Michel Schudel - Let's build a blockchain... in 40 minutes! - Codemotion Amst...

Codemotion

There's a lot of talk about blockchain, but how does the technology behind it actually work? For developers, getting some hands-on experience is the fastest way to get familiair with new technologies. So let's build a blockchain, then! In this session, we're going to build one in plain old Java, and have it working in 40 minutes. We'll cover key concepts of a blockchain: transactions, blocks, mining, proof-of-work, and reaching consensus in the blockchain network. After this session, you'll have a better understanding of core aspects of blockchain technology.

Richard Süselbeck - Building your own ride share app - Codemotion Amsterdam 2019

Codemotion

When was the last time you were truly lost? Thanks to the maps and location technology in our phones, a whole generation has now grown up in a world where getting lost is truly a thing of the past. Location technology goes far beyond maps in the palm of our hand, however. In this talk, we will explore how a ridesharing app works. How do we discover our destination?How do we find the closest driver? How do we display this information on a map? How do we find the best route?To answer these questions,we will be learning about a variety of location APIs, including Maps, Positioning, Geocoding etc.

Eward Driehuis - What we learned from 20.000 attacks - Codemotion Amsterdam 2019

Codemotion

Eward Driehuis, SecureLink's research chief, will guide you through the bumpy ride we call the cyber threat landscape. As the industry has over a decade of experience of dealing with increasingly sophisticated attacks, you might be surprised to hear more attacks slip through the cracks than ever. From analyzing 20.000 of them in 2018, backed by a quarter of a million security events and over ten trillion data points, Eward will outline why this happens, how attacks are changing, and why it doesn't matter how neatly or securely you code.

Francesco Baldassarri - Deliver Data at Scale - Codemotion Amsterdam 2019 -

Codemotion

IoT revolution is ended. Thanks to hardware improvement, building an intelligent ecosystem is easier than never before for both startups and large-scale enterprises. The real challenge is now to connect, process, store and analyze data: in the cloud, but also, at the edge. We’ll give a quick look on frameworks that aggregate dispersed devices data into a single global optimized system allowing to improve operational efficiency, to predict maintenance, to track asset in real-time, to secure cloud-connected devices and much more.

Martin Förtsch, Thomas Endres - Stereoscopic Style Transfer AI - Codemotion A...

Codemotion

What if Virtual Reality glasses could transform your environment into a three-dimensional work of art in realtime in the style of a painting from Van Gogh? One of the many interesting developments in the field of Deep Learning is the so called "Style Transfer". It describes a possibility to create a patchwork (or pastiche) from two images. While one of these images defines the the artistic style of the result picture, the other one is used for extracting the image content. A team from TNG Technology Consulting managed to build an AI showcase using OpenCV and Tensorflow to realize such goggles.

Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...

Codemotion

Blockchain (and Cryptocurrency) is an evolution of 20-year old research from scientists like Chaum, Lamport, and Castro & Liskov. Due to the current hype, it's hard to distinguish beneficial aspects of the technology from a desire for a "silver bullet" for device security, verifiable logistics, or "saving democracy". The problem: blockchain introduces new security challenges - and blind adoption without understanding reduces overall security. In this talk, Melanie Rieback and Klaus Kursawe explain the pitfalls and limits of blockchain, so you can avoid making your applications LESS secure.

Angelo van der Sijpt - How well do you know your network stack? - Codemotion ...

Codemotion

Networking is a core part of computing in the digital world we inhabit. But, how well do you know how it works? Do you understand all the moving parts of the OSI stack inside your computer, and how the network is actually put together? How can this ever work? This guided safari of layers, standards, protocols, and happenstance will bring us close to the copper wire, and up through the layers of CDMA/CD, ARP, routing and HTTP. We will make a few excursions through patchworks that still work forty years later, and cleverly designed mechanisms that show that simplicity is the only way to last.

Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...

Codemotion

Performance tests are not only an important instrument for understanding a system and its runtime environment. It is also essential in order to check stability and scalability – non-functional requirements that might be decisive for success. But won't my cloud hosting service scale for me as long as I can afford it? Yes, but… It only operates and scales resources. It won't automatically make your system fast, stable and scalable. This talk shows how such and comparable questions can be clarified with performance tests and how DevOps teams benefit from regular test practise.

Sascha Wolter - Conversational AI Demystified - Codemotion Amsterdam 2019

Codemotion

Sascha will demonstrate the opportunities and challenges of Conversational AI learned from the practice. Both Technology and User Experience will be covered introducing a process finding micro-moments, writing happy paths, gathering intents, designing the conversational flow, and finally publishing on almost all channels including Voice Services and Chatbots. Valuable for enterprises, developers, and designers. All live on stage in just minutes and with almost no code.

Michele Tonutti - Scaling is caring - Codemotion Amsterdam 2019

Codemotion

A key challenge we face at Pacmed is quickly calibrating and deploying our tools for clinical decision support in different hospitals, where data formats may vary greatly. Using Intensive Care Units as a case study, I’ll delve into our scalable Python pipeline, which leverages Pandas’ split-apply-combine approach to perform complex feature engineering and automatic quality checks on large time-varying data, e.g. vital signs. I’ll show how we use the resulting flexible and interpretable dataframes to quickly (re)train our models to predict mortality, discharge, and medical complications.

Pat Hermens - From 100 to 1,000+ deployments a day - Codemotion Amsterdam 2019

Codemotion

Coolblue is a proud Dutch company, with a large internal development department; one that truly takes CI/CD to heart. Empowerment through automation is at the heart of these development teams, and with more than 1000 deployments a day, we think it's working out quite well. In this session, Pat Hermens (a Development Managers) will step you through what enables us to move so quickly, which tools we use, and most importantly, the mindset that is required to enable development teams to deliver at such a rapid pace.

James Birnie - Using Many Worlds of Compute Power with Quantum - Codemotion A...

Codemotion

Quantum computers can use all of the possible pathways generated by quantum decisions to solve problems that will forever remain intractable to classical compute power. As the mega players vie for quantum supremacy and Rigetti announces its $1M "quantum advantage" prize, we live in exciting times. IBM-Q and Microsoft Q# are two ways you can learn to program quantum computers so that you're ready when the quantum revolution comes. I'll demonstrate some quantum solutions to problems that will forever be out of reach of classical, including organic chemistry and large number factorisation.

Don Goodman-Wilson - Chinese food, motor scooters, and open source developmen...

Codemotion

Chinese food exploded across America in the early 20th century, rapidly adapting to local tastes while also spreading like wildfire. How was it able to spread so fast? The GY6 is a family of scooter engines that has achieved near total ubiquity in Europe. It is reliable and cheap to manufacture, and it's made in factories across China. How are these factories able to remain afloat? Chinese-American food and the GY6 are both riveting studies in product-market fit, and both are the product of a distributed open source-like development model. What lessons can we learn for open source software?

Pieter Omvlee - The story behind Sketch - Codemotion Amsterdam 2019

Codemotion

Dave Farley - Taking Back “Software Engineering” - Codemotion Amsterdam 2019

Codemotion

Would you fly in a plane designed by a craftsman or would you prefer your aircraft to be designed by engineers? We are learning that science and empiricism works in software development, maybe now is the time to redefine what “Software Engineering” really means. Software isn't bridge-building, it is not car or aircraft development either, but then neither is Chemical Engineering. Engineering is different in different disciplines. Maybe it is time for us to begin thinking about retrieving the term "Software Engineering" maybe it is time to define what our "Engineering" discipline should be.

Joshua Hoffman - Should the CTO be Coding? - Codemotion Amsterdam 2019

Codemotion

What is the job of a CTO and how does it change as a startup grows in size and scale? As a CTO, where should you spend your focus? As an engineer aspiring to be a CTO, what skills should you pursue? In this inspiring and personal talk, I describe my journey from early Red Hat engineer to CTO at Bloomon. I will share my view on what it means to be a CTO, and ultimately answer the question: Should the CTO be coding?

More from Codemotion (20)