Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

1

Share

Download to read offline

From Legacy to a Microservices Architecture

Download to read offline

Case of study about using Python, ZCA, Pyramid, Angular 4 and Deis Paas with Kubernets in AWS

Related Books

Free with a 30 day trial from Scribd

See all

From Legacy to a Microservices Architecture

  1. 1. From Legacy To a Microservices Architecture Python, ZCA, Pyramid, Angular 4 & Deis
  2. 2. About me Ruda Porto Filgueiras Software Architect at Briefy
 http://briefy.co From Brazil, based in Berlin Twitter @rudaporto
 ruda@briefy.co
  3. 3. The Company
  4. 4. Briefy Team
  5. 5. Creating visual content at scale INPUT & IMPORT We define a briefing together with our Customers. After this we receive a listing with all items to shoot Hotels, Flats, Restaurants, etc. Receive Orders ASSIGNMENTS We connect Creatives assigning them to Orders in the location are available. Creative schedules the shoot with the third-party contact and execute it. Assign Creative VALIDATE & PACKAGING Creative upload assets and we validate and do QA to make sure we are following the briefing and specific requirements. We deliver the content to Customer: SFTP, GDrive, API. QA & Delivery
  6. 6. Visual content matter
  7. 7. Embrace the change
  8. 8. Overview
  9. 9. A solution based on three pillars API CENTRIC To serve distinct use cases using the same codebase, Briefy built its solution around REST APIs.
 Web, Mobile and customer integration were the planned consumers. MICROSERVICES Single responsibility and distinct life cycles for parts of the solution pointed Briefy to use a micro services architecture. EVENTS BASED Most of the communication between the services could be done asynchronously, so an event-based — with messaging — solution was implemented.
  10. 10. Code quality and reuse as principles TESTING Codebase testing with
 Continuous Integration and
 Continuous deployment CODE QUALITY Style guide and
 documentation for
 the entire codebase DEVOPS Use existing toolset to minimise devops work. CODE REUSE Shared packages/libraries
 Build upon existing OSS codebase
  11. 11. Tech stack INFRA Storage: GDrive & S3
 Messages: SQS
 Cluster: 
 Kubernetes 1.7.8
 DEIS 2.18 API Python 3.6.3
 Zope Component Architecture
 Pyramid 1.9 / Cornice
 SQLAlchemy 1.2
 Postgresql 9.6 DEV TOOLS Repo: GitHub
 CI: TravisCI
 Logging: Logstash
 Reporting: Sentry / NewRelic / Slack WEB FRONTEND Angular 4
 Typescript
 Generic views
 Generic Listings
 Generic CTAs
  12. 12. Infrastructure
  13. 13. AWS THREE ENVIRONMENTS Live: eu-central-1
 Staging: us-east-1
 Development: eu-west-1 SERVICES Message routed using SQS
 S3 for static data & assets
 Kubernets Cluster on EC2
 Dies Workflow
  14. 14. Kops AUTOMATE Kubernets cluster creation with a single command line, including bastion and nat
 High availability, multiple SDN options MAINTAIN Cluster upgrade and change
 Export format templates to cloud formation and terraform
  15. 15. Kubernets KUBERNETES OVER EC2 One master, n minions DEIS ON TOP OF KUBERNETES Workflow solution similar to Heroku PaaS EC2 WITH AUTO SCALING T2 and M4 instances.
  16. 16. Deis Workflow DOCKER Automatic build and save images SIMPLE git push deis master TWELVE-FACTOR And also support legacy if necessary
  17. 17. Devops MONITORING WITH NEWRELIC Servers and applications REPORTS ON SLACK / EMAIL Deploy, errors and alerts posted on tech channel AUTOMATION WITH ANSIBLE AWS and Cloudflare integration
  18. 18. Code
  19. 19. Shared codebase BRIEFY.COMMON Events / Queue
 Base Models / Mixins
 Logging
 Workflow BRIEFY.WS REST API blueprints
 Validation
 Error handling BRIEFY.GDRIVE Common GDrive actions
 Create, Copy, Metadata, List, Download BRIEFY.S3 Common S3 actions
 Create, Copy, Metadata, List, Download
  20. 20. briefy.common QUEUE WORKER Queue worker to consume SQS messages and dispatch WORKFLOW Workflow engine to create solid and secure business processes EVENTS Event firing to internal component architecture and to SQS MODELS Base mixins and sqlalchemy models with versioning, local roles support and polymorphic types CACHE Cache object serialization on redis with dogpile.cache
  21. 21. briefy.ws FILTER Sort and filter listings automatically based on model fields RESOURCES Base classes to exposure models as API endpoints: REST, workflow transitions, versioning EVENTS Object lifecycle events firing SCHEMA Automatic colander schema generation for validation and deserialization SECURITY Resource factory to dynamically compute ACLs based on model workflow and local roles
  22. 22. Productivity tools CHECKSTYLE Linting rules and code style guide COOKIECUTTERS Templates to create new packages and services DOCKER IMAGES Python 3.6.3
 Plone 5.0.8
 Nginx / Varnish
  23. 23. Services
  24. 24. Core services MS.LAURE Automatic quality assurance and renaming of assets LEICA Workflow and scheduling solution.
 API-based CHOREOGRAPHER Events handling and routing. ROLLEIFLEX Authorisation and user management service. ALEXANDRIA Digital Assets management solution
 API-based
  25. 25. Support services CMS Content management backend for Briefy website.
 Using Plone CMS. THUMBOR Image server handling automatic cropping, resizing and image transforms BRIEFY.SLACK Service handling Slack notifications. BRIEFY.MAIL Service handling Mail notifications using Mandrill as the provider HOTELS CRM CRM solution to deal with Hotels acquisition for the City Packages initiative. LEADS Service handling leads and quotes coming from briefy.co website. SITEMAP Service that generates a sitemap.xml for CMS and static content.
 SEO helper API GATEWAY Gateway and router handling public accesses to Briefy API
  26. 26. Leica: from briefing to assets
  27. 27. The core of Briefy WORKFLOW & PERMISSIONS Every model have its own workflow HUNDREDS OF EVENTS Workflow transition, task execution, object creation and update. DATA MODELS Customer, Project, Order, Assignment, Creative and User profiles Being the scheduling and workflow solution for visual content creation, Leica is the core of Briefy operations.

  28. 28. Elements of Leica BRIEFY.FRONTEND Web application developed with Angular4, connects to Leica API.
 This app has 3 distinct users: Customer, Creative and Briefy team member, with distinct permissions. User Interface BRIEFY.LEICA Domain models, permissions, business logic and API endpoints.
 Codebase is well documented and 90% code coverage. API LEICA TASKS Scheduled tasks execute workflow transitions, generate reports and post KPIs.
 Share the same codebase as briefy.leica Tasks
  29. 29. Domain Models CUSTOMER Company requesting visual content creation. PROJECT A grouping of orders from the same customer.
 Usually attached to one contract (or agreement) ORDER A content creation request. When it is not confirmed, it is called Lead.
 Customer oriented. ASSIGNMENT A content creation execution schedule.
 Creative oriented. JOB POOL Groups available Orders to be offered to a group of pre-selected Creatives. CREATIVE A visual content creator - Photographer, videographer — profile. CUSTOMER USER Profile of a user attached to a Customer. BRIEFY USER Profile of a Briefy team member user.
  30. 30. Managing visual content creation at scale cost is complex Confidential Managing Visual Content Creation Is Complex
  31. 31. From Events To Actions
  32. 32. Every action, one event (1) ARIANA Assign a creative to an Order ORDER.WORKFLOW.ASSIGN 2017-07-27 15:46:10.03267 120
 Times
  33. 33. Every action, one event (2) OPHELIE Validates a Creative PROFESSIONAL.WORKFLOW.VALIDATE 2017-07-31 15:55:11.436015 213
 Times
  34. 34. Events trigger new actions ASSIGN. IS SCHEDULED Assignment is scheduled on Leica, generating an event EVENT IS ROUTED Actions are registered to this event, i.e.: Send email to Creative EMAIL SENT TO CREATIVE Using Mandrill, a message is sent to the Creative An event on Leica will be sent to Choreographer, there it will be routed to other micro services
 to execute new actions.
 This flow is asynchronous and all messages are sent using SQS queues. BRIEFY.LEICA BRIEFY.CHOREOGRAPHER BRIEFY.MAIL
  35. 35. Thank you! @rudaporto
 
 ruda@briefy.co
 +49 176 28607522 Rudá Porto Filgueiras
  • AlbertoRodrigues7

    Oct. 20, 2017

Case of study about using Python, ZCA, Pyramid, Angular 4 and Deis Paas with Kubernets in AWS

Views

Total views

599

On Slideshare

0

From embeds

0

Number of embeds

118

Actions

Downloads

7

Shares

0

Comments

0

Likes

1

×