Successfully reported this slideshow.
Your SlideShare is downloading. ×

There's no magic... until you talk about databases

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Upcoming SlideShare
ShowUs: Improved DoIt
ShowUs: Improved DoIt
Loading in …3
×

Check these out next

1 of 39 Ad

More Related Content

Recently uploaded (20)

Advertisement

There's no magic... until you talk about databases

  1. 1. Norbert Hartl ESUG 2022 There's no magic... ... until you talk about databases
  2. 2. "An important thing you need to know about a rule is 
 when you should break it" (Norbert Hartl, ESUG 2022)
  3. 3. Recap: ESUG 2018
  4. 4. tech stack
  5. 5. It has grown …
  6. 6. It has grown small
  7. 7. Three things that can kill a project
  8. 8. 1. complexity Three things that can kill a project
  9. 9. 1. complexity 2. complexity Three things that can kill a project
  10. 10. 1. complexity 2. complexity 3. javascript Three things that can kill a project
  11. 11. tech stack
  12. 12. tech stack - elasticsearch
  13. 13. Events and aggregation Everyone likes dashboards
  14. 14. Events and aggregation Everyone likes dashboards • avoid
  15. 15. Events and aggregation Everyone likes dashboards • avoid • postpone
  16. 16. tech stack - micro services
  17. 17. tech stack - orchestration
  18. 18. Docker swarm & Kubernetes There is only one advize
  19. 19. Docker swarm & Kubernetes There is only one advize • Don't
  20. 20. tech stack - monitoring
  21. 21. „If you have a service that is not monitored you don’t have a
  22. 22. tech stack - containers
  23. 23. tech stack - orchestration
  24. 24. Ansible cheat sheet - name: Deploy apptivegrid API community.docker.docker_container: name: "apptive-api-{{ item.0+1 }}" image: apptivegrid-api: {{ apptivegrid_api_version }} ports: - "{{internal_ip}}:{{ item.1 }}:3600" volumes: - /mnt/apptive_store:/data/apptive with_indexed_items: "{{ apptive_api_ports }}" backend apptivegrid-api-backend balance leastconn {% for apihost in groups[‚api-group'] %} {% for port in hostvars[apihost].apptive_api_ports %} server api-…-{{port}} {{ .…internal_ip }}: {{ port }} check {% endfor %} {% endfor %} [api-group] apptive1 apptive2 apptive3 hostname: apptive1 internal_ip: 10.1.2.5 apptive_api_ports: - 3600 - 3601 - hosts: api-group 
 roles: - apptivegrid-api inventory play host_vars apptivegrid-api role haproxy role
  25. 25. tech stack - load balancer
  26. 26. tech stack
  27. 27. tech stack - database
  28. 28. tech stack - database
  29. 29. wait...
  30. 30. tech stack
  31. 31. Mongo DB The good parts • simple document storage • provides database cluster • supposed to be web scale • we have voyage for it
  32. 32. Mongo DB The not-so-good parts • JSON supports 6 data types • BSON supports a few more • transactions are not part of mongo talk • single writer vs. sharding • query DSLs are a drag
  33. 33. Soil What it needs to be an OO database? • ACID transaction (with MVCC) • Regional fi le locking (row-level locking) • serialization/materialization • A b-tree implementation for indexing • 100% smalltalk
  34. 34. How do we scale that? Escaping the single machine • Files are local on a machine • Opening databases per request is expensive • File locking enables multi-image usage • How to scale to more than one machine? P DB
  35. 35. Distribute the database Escape step #1 • Reduce con fl ict potential • Partition the model • Each user has its own database on disk (4kb) P P P P P P P P DB DB DB
  36. 36. Distribute the database Escape step #2 • Use a distributed fi lesystem (GlusterFS) • Enables multi machine setup • File locking across the network is not reliable P P P P P P P P P P P P P P P P P P P P P P P P replicatio replicatio DB DB DB DB DB DB DB DB DB
  37. 37. Distribute the database Escape step #3 • stateless service • URI contains partition criteria ( /api/users/74827492/...) • stick on path,word(3,/) if { path_beg /api/users/ } • each request to the same database goes to the same image
  38. 38. P P P P P P P P P P P P P P P P P P P P P P P P hapro xy haproxy I'm a cloud shape • 123 -> ... Request /api/users/123/... replication replication DB DB DB DB DB DB DB DB DB Request /api/users/456/... hapro xy • 123 -> ...
  39. 39. Escape summary The complete plan • Persistence approaches are application speci c • Architecture can provide performance/scalability • Writing local fi les does not need to be a blocker • Pinning writes to one place solves a lot

×