The document discusses the technologies used by Ghostmonitor, an e-commerce analytics company with 11 employees established in May 2015. It focuses on their use of microservices and related patterns. Key points include that they develop their single application as a suite of small independent services, use technologies like Docker, Kubernetes, and Redis, and have over 50 services running in 550+ containers to power their analytics platform. They discuss their approaches to local development, testing, deployment, monitoring, and other challenges of building with a microservices architecture.
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)Red Hat Developers
Red Hat JBoss Fuse integration services delivers cloud-based integration based on OpenShift by Red Hat to deliver continuous delivery of tested, production-ready integration solutions. Utilizing a drag and drop, code-free UI and combining that with the integration power of Apache Camel, Fuse integration services is the next generation iPaaS. In this session, we'll walk you through why iPaaS is important, the current Fuse integration services roadmap, and the innovation happening in open source community projects to make this a reality.
Traditional static middleware servers do not fit a cloud-native, micro service model. Rapid container provisioning, software defined networking, and scaling policies now demand spinning up discrete infrastructure services on demand. Chris will present a next generation integration and application hosting environment that will free DevOps from expensive static deployments and glacial refresh cycles. He will describe:
* Why traditional middleware servers must evolve into dynamic infrastructure services
* What Cloud-native, microservices friendly architectural design patterns force teams to rethink integration and application hosting platforms
* What reference architecture ensures successful microservices projects
* How teams can establish DevOps workflows that support on-demand infrastructure services
WSO2Con USA 2015: End-to-end Microservice Architecture with WSO2 Identity Ser...WSO2
During the first half of 2015, iJET Labs used WSO2 Identity Server and API Gateway to help deliver its next generation products. Using WSO2 middleware, iJET now offers secure Federated access to RESTful APIs backed by a scalable microservices architecture. During the course of this journey, iJET Labs worked with WSO2 to extend open source products to meet our unique needs. In this session, we will talk about
WSO2 API Gateway and Identity Server integration
Federated SSO using WSO2 Identity Server
Microservices
Security
AWS deployment automation
Kubernetes is much more than a runtime platform for Docker containers. Through its API not only can you create custom clients, but you can also extend Kubernetes. Those custom Controllers are called Operators and work with application-specific custom resource definitions.
Not only can you write those Kubernetes operators in Go, but you can also do this in Java. Within this talk, you will be guided through setting up and your first explorations of the Kubernetes API within a plain Java program. We explore the concepts of resource listeners, programmatic creation of deployments and services and how this can be used for your custom requirements.
IPaaS 2.0: Fuse Integration Services (Robert Davies & Keith Babo)Red Hat Developers
Red Hat JBoss Fuse integration services delivers cloud-based integration based on OpenShift by Red Hat to deliver continuous delivery of tested, production-ready integration solutions. Utilizing a drag and drop, code-free UI and combining that with the integration power of Apache Camel, Fuse integration services is the next generation iPaaS. In this session, we'll walk you through why iPaaS is important, the current Fuse integration services roadmap, and the innovation happening in open source community projects to make this a reality.
Traditional static middleware servers do not fit a cloud-native, micro service model. Rapid container provisioning, software defined networking, and scaling policies now demand spinning up discrete infrastructure services on demand. Chris will present a next generation integration and application hosting environment that will free DevOps from expensive static deployments and glacial refresh cycles. He will describe:
* Why traditional middleware servers must evolve into dynamic infrastructure services
* What Cloud-native, microservices friendly architectural design patterns force teams to rethink integration and application hosting platforms
* What reference architecture ensures successful microservices projects
* How teams can establish DevOps workflows that support on-demand infrastructure services
WSO2Con USA 2015: End-to-end Microservice Architecture with WSO2 Identity Ser...WSO2
During the first half of 2015, iJET Labs used WSO2 Identity Server and API Gateway to help deliver its next generation products. Using WSO2 middleware, iJET now offers secure Federated access to RESTful APIs backed by a scalable microservices architecture. During the course of this journey, iJET Labs worked with WSO2 to extend open source products to meet our unique needs. In this session, we will talk about
WSO2 API Gateway and Identity Server integration
Federated SSO using WSO2 Identity Server
Microservices
Security
AWS deployment automation
Kubernetes is much more than a runtime platform for Docker containers. Through its API not only can you create custom clients, but you can also extend Kubernetes. Those custom Controllers are called Operators and work with application-specific custom resource definitions.
Not only can you write those Kubernetes operators in Go, but you can also do this in Java. Within this talk, you will be guided through setting up and your first explorations of the Kubernetes API within a plain Java program. We explore the concepts of resource listeners, programmatic creation of deployments and services and how this can be used for your custom requirements.
شرح عن الفروقات بين التصميمات البرمجيه
#Monoliths vs Microservices
#Definitions (Monolithic, Microservices)
#Benefits
#Challenges
When to use
My advice
المصادر
https://medium.com/koderlabs/introduction-to-monolithic-architecture-and-microservices-architecture-b211a5955c63
https://articles.microservices.com/monolithic-vs-microservices-architecture-5c4848858f59
https://microservices.io/patterns/monolithic.html
https://microservices.io/patterns/microservices.html
https://www.youtube.com/watch?v=u_LnubpBDCA
https://www.youtube.com/watch?v=KV3j3MZTXgk
https://www.youtube.com/watch?v=6-vNG33En88&t=852s
https://programmerfriend.com/monolith-vs-microservices/
https://www.hys-enterprise.com/blog/why-and-how-netflix-amazon-and-uber-migrated-to-microservices-learn-from-their-experience/
My YouTube:
https://www.youtube.com/channel/UCE2cLj1ZlV7EUKkCJ3UiZKg
My Twitter account
https://twitter.com/ahmad_ezzeir
My Facebook page:
https://www.facebook.com/Arabic-DevOps-Ahmad-Ezzeir-100543094861932
slideshare:
https://www.slideshare.net/ahmadezzeir/git-locally-git-rmrevertreset
linkedin
https://www.linkedin.com/in/ahmad-ezzeir/
My Twitter account
https://twitter.com/ahmad_ezzeir
My Facebook page:
https://www.facebook.com/Arabic-DevOps-Ahmad-Ezzeir-100543094861932
linkedin
https://www.linkedin.com/in/ahmad-ezzeir/
Modernizing the Legacy - How Dish is Adapting its SOA Services for a Cloud Fi...VMware Tanzu
SpringOne Platform 2016
Speakers: Rob Bennett; Director, Development, Dish Networks; Chandra Nemalipuri; Principal Software Engineer, Dish Networks; Lax Rastogi; Senior Manager, Dish Networks
Like many companies, Dish has a large number of SOA services that have been built using previous generations of technology. In this session we will discuss the challenges faced in converting legacy services to cloud native applications and the different approaches we considered for resolving the conflicts. We will then dive deeper into the approach that we chose to modernize our services and put us on a track towards a microservices based architecture running on Cloud Foundry.
The Hardest Part of Microservices: Calling Your ServicesChristian Posta
When building microservices, you must solve for a number of critical functions, but the process can be incredibly complex and expensive to maintain. Christian Posta offers an overview of Envoy Proxy and Istio.io Service Mesh, explaining how they solve application networking problems more elegantly by pushing these concerns down to the infrastructure layer and demonstrating how it all works.
Docker right now provides great value in the enterprise but the value proposition is more about developer productivity than scale-out.
Docker benefits include resource management, environment management, continuous delivery, developer and operations collaboration, and hybrid workloads.
Take care in its introduction. Consider Docker as just part of an overall toolkit and you don't need to go "full stack" to gain value.
In this session, Steef-Jan Wiggers and Eldert Grootenboer will provide you an overview over all the different extensibility points and will demonstrate a few of them in demo’s.
Brendon Foxen (Channel 4) - Speeding up Software Delivery at Channel 4Outlyer
A look into why Channel 4 needed to speed up the software delivery pipeline and how a Micro services architecture, a CI/CD approach and Docker is helping to make that happen.
Video: Coming Soon
Join DevOps Exchange London here: http://www.meetup.com/DevOps-Exchange-London
Follow DOXLON on twitter http://www.twitter.com/doxlon
شرح عن الفروقات بين التصميمات البرمجيه
#Monoliths vs Microservices
#Definitions (Monolithic, Microservices)
#Benefits
#Challenges
When to use
My advice
المصادر
https://medium.com/koderlabs/introduction-to-monolithic-architecture-and-microservices-architecture-b211a5955c63
https://articles.microservices.com/monolithic-vs-microservices-architecture-5c4848858f59
https://microservices.io/patterns/monolithic.html
https://microservices.io/patterns/microservices.html
https://www.youtube.com/watch?v=u_LnubpBDCA
https://www.youtube.com/watch?v=KV3j3MZTXgk
https://www.youtube.com/watch?v=6-vNG33En88&t=852s
https://programmerfriend.com/monolith-vs-microservices/
https://www.hys-enterprise.com/blog/why-and-how-netflix-amazon-and-uber-migrated-to-microservices-learn-from-their-experience/
My YouTube:
https://www.youtube.com/channel/UCE2cLj1ZlV7EUKkCJ3UiZKg
My Twitter account
https://twitter.com/ahmad_ezzeir
My Facebook page:
https://www.facebook.com/Arabic-DevOps-Ahmad-Ezzeir-100543094861932
slideshare:
https://www.slideshare.net/ahmadezzeir/git-locally-git-rmrevertreset
linkedin
https://www.linkedin.com/in/ahmad-ezzeir/
My Twitter account
https://twitter.com/ahmad_ezzeir
My Facebook page:
https://www.facebook.com/Arabic-DevOps-Ahmad-Ezzeir-100543094861932
linkedin
https://www.linkedin.com/in/ahmad-ezzeir/
Modernizing the Legacy - How Dish is Adapting its SOA Services for a Cloud Fi...VMware Tanzu
SpringOne Platform 2016
Speakers: Rob Bennett; Director, Development, Dish Networks; Chandra Nemalipuri; Principal Software Engineer, Dish Networks; Lax Rastogi; Senior Manager, Dish Networks
Like many companies, Dish has a large number of SOA services that have been built using previous generations of technology. In this session we will discuss the challenges faced in converting legacy services to cloud native applications and the different approaches we considered for resolving the conflicts. We will then dive deeper into the approach that we chose to modernize our services and put us on a track towards a microservices based architecture running on Cloud Foundry.
The Hardest Part of Microservices: Calling Your ServicesChristian Posta
When building microservices, you must solve for a number of critical functions, but the process can be incredibly complex and expensive to maintain. Christian Posta offers an overview of Envoy Proxy and Istio.io Service Mesh, explaining how they solve application networking problems more elegantly by pushing these concerns down to the infrastructure layer and demonstrating how it all works.
Docker right now provides great value in the enterprise but the value proposition is more about developer productivity than scale-out.
Docker benefits include resource management, environment management, continuous delivery, developer and operations collaboration, and hybrid workloads.
Take care in its introduction. Consider Docker as just part of an overall toolkit and you don't need to go "full stack" to gain value.
In this session, Steef-Jan Wiggers and Eldert Grootenboer will provide you an overview over all the different extensibility points and will demonstrate a few of them in demo’s.
Brendon Foxen (Channel 4) - Speeding up Software Delivery at Channel 4Outlyer
A look into why Channel 4 needed to speed up the software delivery pipeline and how a Micro services architecture, a CI/CD approach and Docker is helping to make that happen.
Video: Coming Soon
Join DevOps Exchange London here: http://www.meetup.com/DevOps-Exchange-London
Follow DOXLON on twitter http://www.twitter.com/doxlon
Bluemix provides developers with multiple open-source compute options to run their apps, chief among them Cloud Foundry, the world’s leading platform-as-a-service (PaaS) offering. Cloud Foundry enables teams to practice continuous delivery by supporting the full software development lifecycle, from dev to deployment. One of the key advantages of the platform is the ability it gives developers to easily configure and start using a MongoDB datastore for their application. In this lightning talk, Bluemix developer advocate Jake Peyser will go over Cloud Foundry and best practices for data storage when using the platform. He will then take attendees through a live demo where he will show users how to quickly configure a MongoDB instance in Bluemix and connect it to an application.
Want to integrate MongoDB into your Cloud Foundry App? Learn exactly how to do that with Bluemix Developer Advocate Jake Peyser! Follow him @Jakepeyser.
This slide deck was originally used for a Lightning Talk on integrating MongoDB into a Cloud Foundry application at MongoDB World 2015. It contains an overview of Cloud Foundry, as well as an explanation of where the MongoDB service fits into the technology stack.
[WSO2 Summit Americas 2020] Creating Smart Endpoints Using Integration Micros...WSO2
As microservices-based applications are inherently distributed, the integration of microservices is becoming one of the hardest things when realizing microservices architecture. Rather than using a conventional centralized ESB for integrating services, microservices are integrated based on the smart-endpoints terminology, where all the smarts live at the endpoints while they are interconnected via a lightweight messaging infrastructure. These smart endpoints are often built as integration microservices on top of cloud-native integration technologies.
In this deck, Kasun will explore some key integration patterns for building integration microservices.
Open Source Approach to Design and Deployment of Microservices-based VNFOpen Networking Summit
Prem Sankar's presentation from the 2017 Open Networking Summit.
Microservice is gaining increased adoption in the Telco NFV world. It is key to understand the design and deployment methodologies involved in developing Microservice based VNF. This talk provides an opensource practitioner approach to building and deploying a Microservice based VNF and includes the following: - Design patterns, workflow models - Design models for VNF placement, capacity management, scale-in/out and resiliency - Deployment considerations that includes handing of scale and fault tolerant VNF using well known Opensource tools
This slide deck was presented in OpenNetworkingSummit 2017. The theme is building microservices based VNF using Opensource tools and ecosystem. This also covers Design patterns that are relevant to the VNF context. Overview of COE project in Opendaylight and deployment scenarios it addresses.
Are you jumping on the microservices bandwagon? When and when not to adopt micro services architecture? If you must, what are the considerations? This slidedeck will help answer a few of those questions...
A developer can now build out Cloud Native applications using our patterns-first approach. You simply select the type of building block you’d like to create followed by which services you’d like to incorporate into your application (i.e., Cloudant database, WatsonConversation, Push Notifications).
For enterprises trying to stay ahead of the game, having a robust and fast application development program can make or break their market presence. The challenge for developers, however, is to build responsive, devise-agnostic applications in days, not months.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
5. ”
…the microservice architectural style is an approach
to developing a single application as a suite of small
services, each running in its own process and
communicating with lightweight mechanisms, often
an HTTP resource API.
- Martin Fowler
6. 12factor application
1. Codebase
2. Dependencies
3. Config
4. Backing services
5. Build release run
6. Process
7. Port binding
8. Concurrency
9. Disposability
10.Dev/Prod parity
11.Logs
12.Admin process
7. Microservices patterns
•Service communication HTTP/AMQP or
asynchronous protocols
•API gateway
•Service discovery
•Multiple service instances per host
•Serverless deployment
•Database per Service
•Event-driven architecture
•Database triggers
Udvozlok mindenkit!
Papp David vagyok a ghostmonitornal dolgozom mint Chief Architect. A feladatom az ,hogy a fejlesztes, infrastruktura es minden ami DevOps az hozzam tartozik a cegnel. 1 evvel ezelott is eloadtam ezen a koferencian koszonom ,hogy ujra itt lehetek akkor meg eleg kezdok voltunk a microservices dolgokban akkor meg csak arrol szolt a dolog ,hogy mit tervezunk most megnezzuk mit tanultunk es hova juttotunk. Az eloadas egy altalanos kepet fog festeni milyen problemakkal kell megkuzdeni ,hogy eljussunk egy mukodo rendszerhez es mi milyen problemakkal kuzdotunk meg es ezeket ,hogyan oldotuk fel.
Kik is vagyunk mi?
A cegunk 2015 majusaban jott letre de az otlet mar ennel korabban felvetotod az alaptiokban. A jelenlegi csapatletszam korulbelul 11 fobol all. Ezeknek a nagyresze fejleszot. A cegunket tenktive mi egy e-commerce analytics platform vagyunk.
Az eredeti problema amiert letre jott maga a ceg az a kosar elhagyas(cart abdonment)
Aki foglalkozik webshoppal az mindneki szembesul ezzal a problemaval ,hogy elkezdi az ugyfel vasarlast majd nem fejezi be. Ez a szam egy webshop eseten korulbelul 70-30%-ban oszlik meg. Szoval az emberek 70% el hagyja a kosarat mi ezen probalunk segiteni.
Milyen technologiakat hasznalunk a cegnel?
AWS – Amazon WebServices
Docker
NodeJS
Codeship
Terraform
CoreOS
MongoDB
Elasticsearch
Az elaodasom temaja ,hogy microservices uzemeltetoi szemmel… de eloszor is tisztazzuk ,hogy mi az a microservices es amugy miert jo ez az egesz azon kivul ,hogy jol megszivatjuk magunkat …. ,hogy tul bonyolitunk egy jo nagy rendszer sok kicsivel.
Hagyj idot elolvasni.
Egy martin flower nevu uriember szepen osszefoglalta nekunk ,hogy mi is az a microservices. Ez egy olyan architecturalis megoldas ami kulon allo applickaciokat kapcsol osszes egy egyszeru protokkolon keresztul pl. HTTP.
Most ,hogy mar ismerjuk korulbelul a microservices fogalmat ismerkedjunk meg azzal a 12 alap szabalyal amit erdemes figyelembe venni az alkalmazas es az infrastruktura megtervezesenel. Ez a 12 szabaly segiteni fog nekunk abban ,hogy egy jol atgondolt rendszert kapjunk. 12 szabaly esetleg 12 megoldando problemat is hasznalhatnank erre.
Gyorsan vegyunk parat vegig:
Codebase -> Legyen egy kozponti taroloba tarolva a kodjaink. Ezt annyira nem nehez megugorni.
Tudjunk depedenciakat kezelni -> Compose, NPM, GEM …stb
Environmentbol konfigrulhatoak legyenek – kulonbozo konfig valtoztatasok -> Hogy ez miert jo? Hat mert nem kell ujra deployolni az alkalmazast eleg csak oket szep sorba ujra inditani.
Backing services -> Itt arra gondolt a kolto ,hogy mivel nagyon sok hater szerveziunk van meg kell oldanunk azt a problemat ,hogy tudjanak beszelgetni ezaltal szuksegunk lesz valamilyen service discovery megoldasra. Elosztott rendszerekrol leven szo ez nem olyan elhanyagolhato problema de konnyen meg lehet csinalni.
Itt igazabol arrol van szo ,hogy allitsunk elo egy olyan artifactot vagy imaget amit futtatni fogunk. A mi esetunkben ez a Docker image jelenti.
Torekedjunk arra ,hogy minden szervizunk stateless legyen.
Ez igazabol nem annyira fontos egy fix porton figyeljen a service de ezt a legtobb rendszer lekezelni nekunk.
Gondoljunk az esetleges skalazasi problemakra.
Eldobhatosag -> Ne taroljunk adatokat magaba a szervezikbe erre hasznaljunk valami object storage megoldast vagy barmilyen persistance storageot. Fontos ,hogy a szervizek indulas es lealassa gyors legyen. Pl. ha nagyobb request szam beesik a rendszerbe es a rendszernek reagalnia kell akkor nem mindegy mennyi ido alatt indul el egy szerviz. Ez a mi esetunkben egy viszonylag fontos dolog. Ugyan is mi nem ismerjuk az ugyfeleink latogatotsagat es ha nagyobb request szam beesik akkor arra nekunk reagalni kell a rendszer leallasa/kieses nelkul.
A staging/prod kornyezetunk egyezzen meg igy elkerulhetok legyen esetleges az integracios hibak. Erre nekunk az a megoldasunk ,hogy Kubernetes egy clusterben fut mint ket kornyezet de ugyelve arra ,hogy a staging esetleges hiba eseten ne tudja bedonteni az eles oldalt igy eroforras korlatozas van bevezetve minden servicre. De az eles rendszerben ugyan ugy termeszetesen magasabb ertekkel.
A logolas hat ez az a masik nagy problema amivel szembe kell nezni ha egy microservices architecturat kezdunk el epiteni. Ugyan is a logokat nem csak el kell tudnunk nyelni hanem ezeket fel is kell tudni dolgozni es mondjuk riasztasokat kell csinalni. Ugyan is a monitorozo rendszerrel egyutt nagyon jol ki tudjuk egesziteni a problemak feltarasat esetleg elore tudunk jelezni dolgokat.
Kulonitsuk el az admin feladatokat az eredeti alkalmazastol. Adatbazis takartitas…analitkak
Maga a microservices ugy-e egy elmelet egy megkozelites ezert nincs semmien szabvany ami leirana mikent is kellene egy ilyen rendszert megepiteni ezert is szep ez az egesz.
Par ember viszont meg fogalmazott par hasznos patternt ami jo ha betartunk kulonben szivni fogtok mint mi .
A szervizeink valamilyen aszinkron protokolon keresztul beszlegessnek pl. HTTP vagy AMQP vagy valamilyen aszinkron protokollon keresztul.
Hasznaljunk API gateway-t amin keresztul bejonnek a keresek es tudjuk esetlegesen korlatozni a bejovo forgalmat es a megfelelo szervizhet iranyitani.
Service discovery errol meg lesz ezt kicsit kesobb kifejetem.
Tobb service fusson egy darab instancesen ezt kell tul magyarazni sokkal hatekonyabban ki tudjuk hasznalni az eroforrasokat.
Serverless deployment ez ugy-e az uj trend mint pl. AWS lambda ,hogy feltoltunk egy zippet amibe minden be van csomagolva es fut valahol mi meg orulunk neki.
Errol is lesz szo hamarosan.
Event-driven architecture es Database triggers itt nem olyan adatbazis triggerekre kell gondolni mint ami mysql-be. Nemtudom ki mennyire ismermi a dynamodb-t abba van lehetoseg olyanra, hogy ha egy esemeny bekovetkezik pl. az adatbazisban modosult egy ertek ami ugy-e egy update akkor az generaljon egy eventet. Igy egy esemeny vezerelt arhitecturat tudunk epiteni.
Egy service csak egy tablat irhat.
Mivel elosztott rendszerekrol beszelunk eleg nagy problema lenne ,hogy ha minden egyes szerviz ami szeretne hozza ferni az adatbazishoz elkezdne irni. Ugyan is akkor minden szerviznek ismerni kellene az adatbazis semat.
Gondolom eltudjatok kepzelni mi tortenne ha 1-nel tobb helyen kellene ugyan azt adatbazis semat modositani. Igy viszont az osszes szerviz egy REST API-t lat amit hivogat hat es ha szuksege van egy vagy tobb adatra akkor azt elkeri egy masik servicetol. Ez a model biztositja nekunk azt is ,hogy nem kell ugyan azt az adatbazis motort hasznalni minden szerviz alatt hanem ez viszonylag konnyen tudjuk modositani egyik nap mongodb van mogotte a masik nap meg mar mondjuk egy mysql.
El is erkeztunk az eloszott rendszerek masik problemajahoz meg pedig ,hogy talaljunk meg egymast mikozben minden dinamikus es minden valtozik semminek sincs fix ip cime vagy fix portja. Ugy-e amikor a rendszerunk elejen vagyunk ebbe a problemaba bele se gondolunk amig mondjuk van 5-6 szerveziunk de amikor ez a szam elkezd drasztikusan novekedni akkor ez a problema hamar szembe fog velunk jonni. Az se szerencses ha ezeket beegetjunk szepen a konfigokba mert akkor egy esetleges valtoztatas eseten a szerviz osszes depedenciajat ujra kellene deployolni nem lenne tul szerencseses.
!!!Plusz kep bejon…
Erre nyujt nekunk megoldast a service discovery minden service ami elindul az beirja magat a service registry-be amikor pedig leall akkor a rendszer kiveszi ot a kiszolgalasbol. A service discoverynek ket legismertebb modja az DNS es az ENVIRONMENT alapu. Mi mind a kettot hasznaljunk amennyiben pl. DNS nem megy ez esetben fallbackelunk ENVIRONMENT-re.
Felsoroltam eloszor ezt a 4 problemat ami egybol szembejott amikor ki talatuk ,hogy mi szeretnenk ilyen szep elosztott rendszer epiteni…hat igen ezt tok jo milyen jo lesz ez nekunk igen de mivel is valositsuk meg ezeket.
Eloszor is szuksegunk volt valami frameworkre az elozo rendszer eleg vegyes volt tartalmazott Sails meg Express-t vegyesen valahogy ezt az alaptotott fel kellet oldani le lovom a poet a valasztas nem ezekre esett.
Ott volt a lokalis fejlesztes problema ez maig nem teljesen megoldott problema viszont mar van ra egy viszonylag jol bevalt megoldasunk.
Hat valahogy tesztleni is kellene azokat a franya szerveziket es nem csak mockolt adatokkal. Igen am de itt is szembe jott par problema…
Ezekbol a kodobol kellene csinalni valami Docker image-t is fabrikalni lovagoljuk mar meg a trendeket.
Jojon a kovetkezo 4-s csoport…
Ohh…hat ha mar le buildeltuk a docker imageket akkor azokat valahogy ki is kellene deplyolni valami infrastrukturara
Ha mar van deployerunk akkor valamin futtasuk mar ezeket az imageket….
Igen…igen futnak csucsszuper meg is kellene nezni mit csinalnak ezeket a frana szervezinek biztos nem mukodik mindig jol…hat nem…
Tok jo ,hogy monitorozzuk oket de azert jo lenne tudnunk mi tortenik a lelkukbe es valami informaciot kikopnek az eljutna hozzank valami strukturalt formaba.
Nezzuk is az elso jatekost a framework problema. Alapvetoen hive vagyunk annak az elvnek ,hogy nekunk problemakat kell megoldani es nem programozunk kell viszont van amikor kompromisuzmokat kell kotni. A Koa framework egy eleg minimal dolog de azert ez mellet viszonylag sok mindent nyujt nekunk. Nem er fel mondjuk egy Express vagy Sails kepessegeivel viszont biztositja nekunk azt ,hogy a szervezinek ne legyenek tulsagosan eroforras igenyesek. Az atlag szervziek 128MB rammal nagyon szep elfutkaroznak.
Szuksegunk volt egy kozponti LIB-re amit minden szerviz behuzz es tud hasznalni. Szukseg volt arra ,hogy modularis legyen nem minden szerviznek van szuksege minden funkciora pl. egy API-nek nincs szuksege MongoDB-re de szuksege van Service Discoveryre es mondjuk REDIS-re. Viszont ha kesobb szukseg van barmilyen extra dologra akkor csak be kapcsoljuk az adott szervizre es mar lehet is hasznalni.
Most ,hogy mar van frameworkunk meg alkalmazasunk ezt akkor csomagoljuk mar be ,hogy hordozhato image-be ami a mi esetunkbe a Docker lett. A lokalis fejleszteshez szukseg van Docker-compose vagy dockerre ugyan is a kulonbozo dependeciakat valahogy futtatnunk kell magunknal de maga a fejlesztes nem dockeren belul csinaluk azt siman a megszokott modon tortenik. A dolog annyiban bonyolodik ,hogy ha a olyan szervizt fejlesztunk aminek sok a dependiciaja akkor akar 10-15 docker is futhat a gepukon ami azert a regebbi gepeknek eleg megterhelo lehet. Viszont mivel van environment alapu service-discoverynek ezert ezeket ugyan beallitjuk lokalba es megfogja talalni lokalisan futtatot szerviz a dockerben futokat.
A Jet a codeship nevezetu CI-nak a termeke amivel localba lehet futtatni ugyan azokat a dolgokat mint fent. Igy localba is ki tudjuk probalni mi fog torteni CI kornyezetben.
Minikube es Kube-Solo pedig abban segit ha van valami kubernetes specifikus problemank vagy big picture akarjuk latni localba akkor ezeket futtatjuk es ugyan azt vegre tudjuk hajtani mint fent az eles vagy staging rendszerbe.
A rendszerben egyarant megtalalhato unit test – e2e test – integration test. Mi is figyeljuk a codecoveraget meg a code complexity-t erre a Codeclimate nevu termeket hasznaljuk.
Szeretjuk a valosaghu teszteket ami az jelenti amennyire lehet nem mockolt teszteket hasznalunk hanem van lehetosegunk teszteles es buildeles kozben behivni mas szerveziket igy tudjuk tesztelni kozel azt az allaptot mint ami eles vagy staging kornyezetben megtortenik.
Itt a buildelesnel fontos meg emeliteni meg egy fontos problemat ugyna is a dockernek van egy ”kis” hibaja ami nem is olyan kicsit es eleg gyorsan szembe fog velunk meg pedig nem tud kezelni inditasi sorrendeket szoval egy mas dependciai nem sorrendben fognak elindulni hanem ossze vissza szoval erre a problemara jobb ,hogy fel keszulunk. Mindenkinek a sajat rendszerben erre megoldast kell talalnia.
A codeship services fileban tudjuk meghatarozni ,hogy az adott buildhez milyen szervizeket huzunk be a buildelesnel. Ebben az esetben mongodb de itt adhatjuk meg az elobb emlitett tobbi backend szervizt amit szuksegunk van.
Codeship-steps
Itt adahatjuk meg azokat a parancsokat amiket a kodon vegre kell hajtania buildeles kozben. A commandokat tudjuk kulonbozo branchrek korlatozni.
Nezzuk gyorsan at a fajlt.
Az elso sor servce: app ez minden build eseten lefut. Az npm install azert fut itt le mert ezek a buildek meg nem artifactory alapuakra ugyan is nodejsnel is szukseg volt a sok package miatt amire szukseg lett. Az atlag szerviz meret az 100-200MB volt az artifactes build utan pedig 37MB-t. Beszeltunk a 12 szabalynal ez a 9-es szabalyhoz tartozik ugyan is amikor le kell tolteni a szerverek minden egyes buildnel mondjuk 100-200MB-t ha ez leviszuk a 37MB mar is drasztikusan csokkeni fog a deployolasi ido.
A push nekunk a deplyolas a commit_id tag-re epul igy sokkal konnyebben tudunk rollbackelni.
Ez egy altalunk kifejlesztett sajat deployer. A feladata az ,hogy a codeship webhookokat feldolgozva elo allitatsa a template fileokat vagy veghez vigyen egy rolling-updatet. A kettot kozott annyi a kulonbseg mig az egyik eseten maga a template valtozik a masik eseten csak uj imaget rakunk ki a rendszerbe.
A deployer maga ugy mukodik ,hogy van egy API resze az megkapja a hookot ezt feldolgozva berakja egy queue-ba ami jelen esetben REDIS majd innen a workerek kiveszik es feldolgozak az adott feladatot. Ha vegezetke akkor kuldenek rola egy notification-t slack channelbe.
Rollback -> jelenleg ez meg terv de hamarossan megvalositasra kerul. Ennek van egy beepitett modja ,hogy ha a deployolas kozbe hiba csuszik pl. nem tud elindulni a szerviz akkor a deploy automatikusan megszakad.
Template management
Kezeli a kulonbozo szervizek eroforrasat
A szervizek verzio kezelest
Environment valtozok
AWS kulcsok
MONGO_URI
REDIS_URI