End of 2016, Dailymotion revamped the whole company, in that slide, we will explain you how we have used the DevOps mindset as an enabler to scale up our engineering team and our architecture.
EclipseCon France 2014 presentation on Papyrus for Real-Time Embedded applications.
It presents how Papyrus, an open-source software and system modeling tool, can address the needs of the real-time embedded software developer.
File has been updated with up-to-date information post-conference.
By using terraform, creating the network components, kubernetes cluster, CI/CD pipeline and a working application without touching anything on AWS console! Fully automated!
Papyrus for RealTime - Executable Modeling on EclipseCharles Rivet
Papyrus for RealTime is an Eclipse-based open-source UML-RT modeling tool providing complete code generation for cyber-physical systems. This deck was presented at EclipseCon North America 2016.
EclipseCon France 2014 presentation on Papyrus for Real-Time Embedded applications.
It presents how Papyrus, an open-source software and system modeling tool, can address the needs of the real-time embedded software developer.
File has been updated with up-to-date information post-conference.
By using terraform, creating the network components, kubernetes cluster, CI/CD pipeline and a working application without touching anything on AWS console! Fully automated!
Papyrus for RealTime - Executable Modeling on EclipseCharles Rivet
Papyrus for RealTime is an Eclipse-based open-source UML-RT modeling tool providing complete code generation for cyber-physical systems. This deck was presented at EclipseCon North America 2016.
Every customer-facing project aims to achieve high uptime. We, as developers, write thorough unit and integration tests to catch as many bugs as possible. We even set up staging environments to simulate production as well as we can. Yet, downtime still happens and not only once we had to revert a change.
Blue-Green deployment is a technique that allows us to ship code with high confidence, minimizing the risk of errors. We will go through the why and what of Blue-Green,as well as show how you can start doing it using Spinnaker.
BDEVOPS @ Adobe 11.04.2017
44CON 2014 - Binary Protocol Analysis with CANAPE, James Forshaw44CON
44CON 2014 - Binary Protocol Analysis with CANAPE, James Forshaw
CANAPE is an open source network proxy written in .NET. It has been developed to aid in the analysis and exploitation of unknown application network protocols using a similar use case to common HTTP proxies such as Burp or CAT.
This workshop will go through the basics of analysing an unknown application protocol with hands on training examples. By the end of the workshop candidates should be able to better understand CANAPE’s functionality and be able to apply that to other protocols they come across.
CloudCamp. Paul Hopton, @relayr_cloud - 'The WunderBar - Bootstrapping the In...Chris Purrington
Paul Hopton, @relayr_cloud - 'The WunderBar - Bootstrapping the Internet of Things
How to move beyond corporate hype, and make the Internet of Things happen (almost) now.
Papyrus-RT - Executable modeling on eclipseCharles Rivet
Presentation made at EclipseCon France.
This slide deck shows the progress made in the creation of Papyrus for Real Time, one of the first products in the Eclipse Papyrus product line.
Papyrus-RT is a modeling tool implementing the UML-RT domain specific language targeting reactive embedded and cyber-physical systems.
PapyrusRT: Modelling and Code GenerationErnesto Posse
Papyrus-RT is an open-source industrial-grade, complete modeling environment for the development of complex, software intensive, real-time, embedded, cyber-physical systems.
This presentation discusses the motivations for the project, the UML-RT language, the tool and the code generation.
This presentation was given in the Open-Source Software for Model Driven Engineering workshop (OSS4MDE'15) collocated with the MODELS'15 conference, Ottawa, Canada, September 29, 2015.
Setting up an ONAP development environment is not easy. Development tools and practices are not collected in a single place. This project pretends to collect and standardize that process.
Atmosphere 2018: Jukka Forsgren - SPINNAKER AND MULTI-CLOUD CI/CDPROIDEA
I will demo and talk about the new emerging open source continuous deployment tool called Spinnaker, and how it can be used in multi-cloud and Kubernetes CI/CD pipelines. This tool is gaining popularity fast and providing a valid alternative to Jenkins especially when it comes to the CD part. It has some heavyweights behind it, for example Netflix (who originally developed it), Waze, and Google who is taking part in its development. Spinnaker also natively pushes immutable architecture with integrated image baking and deploying those images into target accounts.
Presentation written by Sean Cohen and Steve Gordon at Red Hat covering the highlights of the OpenStack Liberty release.
Presented to the Atlanta OpenStack Meetup on October 15th.
Deploy your apps using Google Cloud service, App Engine. It is server-less service for deploying apps. You don't need worry about hardware, installation, operation and maintenance. You only focus with your business and application.
Kubernetes and Cloud Native Meetup - March, 2019CloudOps2005
This year's first round of Kubernetes and Cloud Native meetups in Eastern Canada began with an update of the CNCF by Ayrat Khayretdinov, CNCF Ambassador and Solutions Architect at CloudOps. He explained the status of various projects and highlights from KubeCon + CloudNativeCon. To learn the basics of cloud native application modernization, sign up for one of our hands-on, three-day workshops on Docker and Kubernetes at https://www.cloudops.com/workshops/#DockerK8s
This material was used during the ONAP DDF + OPNFV Plugfest 2019 in Paris to explain the different components used during the development of the MultiCloud Kubernetes plugin
Transformacion e innovacion digital Meetup - Application Modernization and Mi...José Román Martín Gil
Slides from my talk at Transformation Digital Meetup in Valencia about Application Modernization and Migration, a journey to the cloud.
Event: https://www.meetup.com/es-ES/transformacion-e-innovacion-digital/events/251042790/
Every customer-facing project aims to achieve high uptime. We, as developers, write thorough unit and integration tests to catch as many bugs as possible. We even set up staging environments to simulate production as well as we can. Yet, downtime still happens and not only once we had to revert a change.
Blue-Green deployment is a technique that allows us to ship code with high confidence, minimizing the risk of errors. We will go through the why and what of Blue-Green,as well as show how you can start doing it using Spinnaker.
BDEVOPS @ Adobe 11.04.2017
44CON 2014 - Binary Protocol Analysis with CANAPE, James Forshaw44CON
44CON 2014 - Binary Protocol Analysis with CANAPE, James Forshaw
CANAPE is an open source network proxy written in .NET. It has been developed to aid in the analysis and exploitation of unknown application network protocols using a similar use case to common HTTP proxies such as Burp or CAT.
This workshop will go through the basics of analysing an unknown application protocol with hands on training examples. By the end of the workshop candidates should be able to better understand CANAPE’s functionality and be able to apply that to other protocols they come across.
CloudCamp. Paul Hopton, @relayr_cloud - 'The WunderBar - Bootstrapping the In...Chris Purrington
Paul Hopton, @relayr_cloud - 'The WunderBar - Bootstrapping the Internet of Things
How to move beyond corporate hype, and make the Internet of Things happen (almost) now.
Papyrus-RT - Executable modeling on eclipseCharles Rivet
Presentation made at EclipseCon France.
This slide deck shows the progress made in the creation of Papyrus for Real Time, one of the first products in the Eclipse Papyrus product line.
Papyrus-RT is a modeling tool implementing the UML-RT domain specific language targeting reactive embedded and cyber-physical systems.
PapyrusRT: Modelling and Code GenerationErnesto Posse
Papyrus-RT is an open-source industrial-grade, complete modeling environment for the development of complex, software intensive, real-time, embedded, cyber-physical systems.
This presentation discusses the motivations for the project, the UML-RT language, the tool and the code generation.
This presentation was given in the Open-Source Software for Model Driven Engineering workshop (OSS4MDE'15) collocated with the MODELS'15 conference, Ottawa, Canada, September 29, 2015.
Setting up an ONAP development environment is not easy. Development tools and practices are not collected in a single place. This project pretends to collect and standardize that process.
Atmosphere 2018: Jukka Forsgren - SPINNAKER AND MULTI-CLOUD CI/CDPROIDEA
I will demo and talk about the new emerging open source continuous deployment tool called Spinnaker, and how it can be used in multi-cloud and Kubernetes CI/CD pipelines. This tool is gaining popularity fast and providing a valid alternative to Jenkins especially when it comes to the CD part. It has some heavyweights behind it, for example Netflix (who originally developed it), Waze, and Google who is taking part in its development. Spinnaker also natively pushes immutable architecture with integrated image baking and deploying those images into target accounts.
Presentation written by Sean Cohen and Steve Gordon at Red Hat covering the highlights of the OpenStack Liberty release.
Presented to the Atlanta OpenStack Meetup on October 15th.
Deploy your apps using Google Cloud service, App Engine. It is server-less service for deploying apps. You don't need worry about hardware, installation, operation and maintenance. You only focus with your business and application.
Kubernetes and Cloud Native Meetup - March, 2019CloudOps2005
This year's first round of Kubernetes and Cloud Native meetups in Eastern Canada began with an update of the CNCF by Ayrat Khayretdinov, CNCF Ambassador and Solutions Architect at CloudOps. He explained the status of various projects and highlights from KubeCon + CloudNativeCon. To learn the basics of cloud native application modernization, sign up for one of our hands-on, three-day workshops on Docker and Kubernetes at https://www.cloudops.com/workshops/#DockerK8s
This material was used during the ONAP DDF + OPNFV Plugfest 2019 in Paris to explain the different components used during the development of the MultiCloud Kubernetes plugin
Transformacion e innovacion digital Meetup - Application Modernization and Mi...José Román Martín Gil
Slides from my talk at Transformation Digital Meetup in Valencia about Application Modernization and Migration, a journey to the cloud.
Event: https://www.meetup.com/es-ES/transformacion-e-innovacion-digital/events/251042790/
The Story of SNCF Connect - biggest Flutter app in Europe (@FlutterHeroes 2023)François
From 10 years native to the biggest Flutter app in Europe: the story of SNCF Connect with Flutter.
Slide deck from the conference at Flutter Heroes 2023 (Turin, Italy).
Presented by François Nollen, Adrien Body and Valentin Michalak for SNCF Connect & Tech.
Replay available: https://www.youtube.com/watch?v=CbfRLjxg_rw
OSDC 2018 | Three years running containers with Kubernetes in Production by T...NETWAYS
The talk gives a state of the art update of experiences with deploying applications in Kubernetes on scale. If in clouds or on premises, Kubernetes took over the leading role as a container operating system. The central paradigm of stateless containers connected to storage and services is the core of Kubernetes. However, it can be extended to distributed databases, Machine Learning, Windows VMs in Kubernetes. All these applications have been considered as edge cases a few years ago, however, are going more and more mainstream today.
Awareness presentation on the integration of Network Operations into DevOps and using tools like Ansible and UCS director to automate network operations.
DevOps on Steroids Featuring Red Hat & Alantiss - Pop-up Loft Tel AvivAmazon Web Services
Session 1:"Git, Bitbucket, Jira, Chef an ALM DevOps demo by ManageWare"
In this session we will explore the method and tools for an end-to-end workflow, from Story to code Deployment.
Session 2: ""DevOps with Red Hat OpenShift and our offering on AWS""
Integrating the workflow between Development & Operations in an agile way through automation
OSDC 2013 | The OpenNebula Cloud Platform for Datacenter Virtualization by Co...NETWAYS
The presentation will describe the OpenNebula project for data center virtualization, with a focus on how to deploy OpenNebula-based private clouds and its unique features to tune and adapt it to any technology in the cloud and virtualization ecosystem.
The target audience are developers and IT administrators interested in deploying a private cloud solution, or in the integration of OpenNebula with other projects. The talk will be useful for both people with experience or without prior knowledge of OpenNebula, as it will start by introducing the project and its main features, along with a quick demonstration. Although this is not a hands-on tutorial, by the end of the presentation attendees will have a comprehensive idea of the integration and customization capabilities of OpenNebula in different areas, like user authentication, virtualization, storage, networking, etc.
OpenNebulaConf2019 - Welcome and Project Update - Ignacio M. Llorente, Rubén ...OpenNebula Project
We've made our way into the world of open cloud — where each organization can find the right cloud for its unique needs. A single cloud management platform cannot be all things to all people. There will be a cloud space with several offerings focused on different environments and/or industries. The OpenNebula commitment to the open cloud is at the very base of its mission — to become the simplest cloud enabling platform — and its purpose — to bring simplicity to the private and hybrid enterprise cloud. OpenNebula exists to help companies build simple, cost-effective, reliable, open enterprise clouds on existing IT infrastructure. The OpenNebula Conference will be a great opportunity to communicate and share our vision and commitment, to look back at how the project has grown in the last 9 years, and to shed some insight into what to expect from the project in the near future.
Scaling Gilt: from Monolithic Ruby Application to Distributed Scala Micro-Ser...C4Media
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/10fVilQ.
Yoni Goldberg describes some of the technological innovations that have helped Gilt to reach its current size, and highlight some of the core challenges that the company's engineering team continues to face. He also discusses what every tech team needs to consider and address before heading down the path of building a first-class micro-services architecture. Filmed at qconnewyork.com.
Since joining Gilt at 2010 as a platform engineer, Yoni Goldberg has been leading a variety of personalization efforts and other customer-facing initiatives--including the Gilt Insider loyalty program, the post-purchase experience, and SEO/optimization efforts. Prior to joining Gilt, Yoni worked at Google, where he wrote his master's thesis on Fusion Tables.
First steps into developing an application as a suite of small services, and analysis of tools and architecture approaches to be used.
Topics covered:
1) What is a micro service architecture
2)Advantages in code procedures, team dynamics and scaling
3) How container services such as docker assist in its implementation
4) How to deploy code in a micro services architecture
5) Container Management tools and resource efficiency (mesos, kubernetes, aws container service)
6) Scaling up
By PeoplePerHour team
presented by CTO Spyros Lambrinidis & Senior DevOps Panagiotis Moustafellos @ Docker Athens Meetup 18/02/2015
OpenNebulaConf2018 - Welcome and Project Update - Ignacio M. Llorente, Rubén ...OpenNebula Project
We’re moving into a world of open cloud — where each organization can find the right cloud for its unique needs. A single cloud management platform can not be all things to all people, there will be a cloud space with several offerings focused on different environments and/or industries. The OpenNebula commitment to the open cloud flows directly out of its mission — to become the simplest cloud enabling platform — and its purpose — to bring simplicity to the private and hybrid enterprise cloud. OpenNebula exists to help companies build simple, cost-effective, reliable, open enterprise clouds on existing IT infrastructure. The OpenNebula Conference will be a great opportunity to remind our vision, vision and commitment, to look back at how the project has grown in the last decade, and to give a peek at what to expect from the project in the near future.
Similar to How we scale up our architecture and organization at Dailymotion (20)
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
How we scale up our architecture and organization at Dailymotion
1. from a french monolith to a worldwide platform: a human story
2. Stan Chollet
Chapter lead Core API
Tribe Scale @ Dailymotion
https://stan.life
Président Association Orléans Tech
Formateur Kubernetes & GraphQL
3. 3
3billion
video views per month
300million
unique visitors per month
150million
videos in our catalogue
Dailymotion, one of the
leading video
destination platforms in
the world
4. OUR MISSION
4
transforming our video platform into a
global destination for must-see videos.
Building the best “go-to”
experience where users can get
their daily dose of must-see
videos, and partners can leverage
the latest tools to grow and
monetise their audience.
5. FROM MONOLITH TO SOA
5
Our road to micro-service architecture SOA
• monolith LAMP Stack
• hosted on bare-metal
• mono-datacenter (PARIS)
• REST API
• fullstack website
• geo-distributed
• apps run in container (docker)
• orchestrated on top of Kubernetes
• multiple languages (mainly Python / Golang)
• GraphQL API
• fully API Centric
TO
6. GRAPHQL - AN ENABLER FOR OUR FRONTEND AND OUR BACKEND
6
FROM. TO.
Monolith PHP
Website
HTML
REST API
GraphQL
svc 1
python
svc 2
golang
svc 3
java
7. GRAPHQL - AN ENABLER FOR OUR FRONTEND AND OUR BACKEND
7
TRIBES ? SQUADS ?
Tribe
Squad Squad
Chapter
Squad
Chapter
Tribe
Squad Squa
Chapter
Chapter
Squad
8. GRAPHQL - AN ENABLER FOR OUR FRONTEND AND OUR BACKEND
8
SOA AS AN ORGANIZATIONAL ENABLER.
SOA (geo-distributed) architecture
GraphQL
Data
service
User
service
Partner
service
Monolith (mono-datacenter)
Website
HTML
REST API
ownership product enabler ownership product tribes ownership mixed
9. FIRST STEP
9
• Built & managed by one team (2 people)
• Deployed in 3 regions on AWS
• Orchestrated on top of kubernetes
• Apps deployed with custom bash scripts
• Good application monitoring
• Poor infrastructure monitoring
FROM SEPTEMBER 2016 TO JANUARY 2017.
GraphQL
REST
Legacy PHP
Search
python
Kubernetes on AWS
FOUNDATIONS•
10. SECOND STEP
10
TIME TO
SCALE•
FROM JANUARY 2017 TO JUNE 2017.
People
• from 2 to ~30 people.
• from 1 to 5 teams
Services
• from 1 to ~15 services.
• from 1 to ~10 languages / technologies
Release
• from an average of 1 deployment per
day to more than 10
11. HUMAN FIRST
• Hired more than 30 people over a couple a months
• Organised training sessions for newcomers
• Optimised and reviewed our on-boarding process
• Optimised the way to work on an SOA stack
• Evangelised (GraphQL + Infrastructure)
FROM 2 TO ~30 PEOPLE.
12. • Only one dependency on the developer's laptop: docker
• Simplify the technical on-boarding process
• Simplify the project switching over our 500+
repositories
• Use generic tasks name to launch code quality checks
• Let developers use the technologies they want
12
make style
make test
make test-unit
make test-functional
make test-integration
make complexity
make run
13. FROM AWS TO GCP
13
• Worldwide network (subnets can be routed from one region to another)
• Ingress anycast IP, easy to setup
• A hosted Kubernetes managed service with cool features such as node autoscaling
• Connection to Dailymotion’s private network in Paris
• Currently deployed in 3 regions across the world (~80 nodes)
FROM 1 SERVICE TO 10 SERVICES.
14. NEW HIGHLY SCALABLE HYBRID ARCHITECTURE
14
Geo-Distributed
for high performance everywhere in the world
Hybrid Infra
on Premise together with Google Cloud
Auto-scaling
adapts to the audience
Google Cloud POP
On Premise POP
CDN
15. GIVE ROOT ACCESS TO DEVELOPERS 😎
15
• Implement continuous deployment
(except production which needs human approval)
• Let developers deploy by themselves
• Delegate deployment workflow to developers through Jenkinsfile
(Pipeline).
• Enforce common interfaces, minimum code quality, deployment
guidelines built by the devops team
FROM 1 DEPLOYMENT PER DAYTO MORE THAN 10.
16. WE ARE LEARNING FROM OUR MISTAKES
16
STEP #1:
First we deployed our applications sequentially, region by region using bash scripts
STEP #2:
We wanted to manage our cluster from a single API endpoint : Federation
Some API objects were missing in the Federation → mixed deployment methods : some
objects in the Federation and others deployed region by region.
STEP #3 (déjà-vu):
Now, we’re deploying our applications sequentially region by region using Helm
FROM 1 DEPLOYMENT PER DAYTO MORE THAN 10.
17. CHARTS EVERYWHERE !
17
• Manage dependencies between our applications.
• Deploy a complete stack with a single command.
• Help us to manage different environments/regions within a chart.
• Easy to rollback: each deployment has a unique revision id
• Ongoing : Provision a staging environment per pull request
FROM 1 DEPLOYMENT PER DAYTO MORE THAN 10.
18. FROM SLA 99,999% TO 99,9999999999999999999999999999999999%
18
• APM with Open Tracing Specification
• Monitoring / Alerting
• Logging Specification for each service
• Feature Flipping, Progressive rollout, Experimentation (A/B)
HOW WE OPERATE OUR PLATFORM?
19. WE ARE NOT ROBOTS
19
BUILD. Software Engineer
• Write code
• Build applications which aren’t easy to operate
SHIP. Release Engineer
• Package & deploy applications
RUN. System Engineer
• Operate infrastructure & app
• Unable to fix applications by themselves
FROM SOFTWARE / SYSTEM ENGINEER TO PRODUCTION ENGINEER.
BUILD / SHIP / RUN.
Production Engineer
• Can build applications
• Package & deploy applications
• Operate application in production
• Build their applications with “RUN” mindset
• Build tools for software engineers
TO
20. helm upgrade —install westeros —reuse-values —set imageTag=30610c5 dailymotion/westeros-gbased-raulicache
BOOM !
WHAT: Bad parameter applied on helm command
• 3 clusters emptied (~ 1 300 containers)
• All our products were unusable
AND: We were down during 19 minutes
• ~10 minutes to be notified
• ~7 minutes to understand
• ~2 minutes to recover from scratch the entire architecture
NOW: Grow up
• Wrap destructive commands
• Improve monitoring
21. INFINITE AND BEYOND
21
• Hybrid architecture (on premises)
• Stateful use cases: manage volume provisioning in the same way
we orchestrate applications
• Performance improvements (Service mesh)
• Security: user authentication and auditing, secrets encryption.
• Open Source our GraphQL Engine (Python, performance oriented)
AND NOW ?