Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Data Abstraction for Large Web Applications

4,097 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Data Abstraction for Large Web Applications

  1. 1. Data Abstraction forLarge Web Applications By Brandon Savage
  2. 2. Who Am I?• Software developer at Mozilla working on Socorro• Author of the PHP Playbook• Former frequent blogger on PHP topics• Private pilot in my spare time
  3. 3. Data Abstraction For LARGE Web Applications
  4. 4. No magic bullets
  5. 5. Once upon a time...In a galaxy far far away...
  6. 6. Eventually the web grew up. And grew larger.
  7. 7. Most webapps still start asthough they’ll always use a database.
  8. 8. We need to change our thinking.
  9. 9. Socorro
  10. 10. Socorro Data Sources• Postgres• REST API (Middleware)• Elastic Search• Hbase• Bugzilla REST API• Memcache
  11. 11. A database-centric model just doesn’t work anymore.
  12. 12. Solving the problem• Separate the use of data from the retrieval of data.• Think in terms of actions.• Build our applications to be storage agnostic.• Use the correct data storage medium.
  13. 13. #1 Separate the use of data from the retrieval of data
  14. 14. <?phpclass MainPage_Controller {/* ... */public function do_something(){ /* ... */ $sql = ‘SELECT * FROM database”; $results = $this->execute($sql); return $this->executeView(‘index’, array(‘results’ => $results)); }}
  15. 15. <?phpclass Data_Model {/* ... */public function get_some_data() { $sql = ‘SELECT * FROM database”; $results = $this->execute($sql); /** process results **/ return $processedResults }}
  16. 16. Processing the data is a separate layer.
  17. 17. <?phpclass Data_Model {/* ... */ public function getSomeData() { $data = $this->adapter->queryData();! /** process data here **/! return $processedData; }}class Data_Model_Adapter extends MySQL_Adapter implements Adapter{ public function queryData() { $sql = ‘SELECT * FROM table’; /** turn into common format **/ return $commonFormatData; }}
  18. 18. Swapping out data sources becomes very simple.
  19. 19. A cautionary tale
  20. 20. Move to middleware in Socorro
  21. 21. Make life easier on yourself: do it right the first time!
  22. 22. #2 Think in terms of actions.
  23. 23. Actions move beyond SELECT,INSERT, UPDATE and DELETE.
  24. 24. Domain Modeling:“What are you modeling?”
  25. 25. What do I want? What do I need?What does this data represent?
  26. 26. Django Models: One model per table.All methods relate to SQL. That sucks.
  27. 27. <?phpabstract class User_Model {public function loadUser();public function authenticateUser();public function showUserPhones();}
  28. 28. #3 Build our applications to be storage agnostic
  29. 29. Use a standard data format
  30. 30. stdClass()
  31. 31. Create custom objects for typehinting or additional methods
  32. 32. Avoid expecting built-ins like PDOStatement and MongoCursor outside retrieval layer
  33. 33. #4 Use the correct storage medium.
  34. 34. Example: memcache isn’t for long-term storage.
  35. 35. Example: MongoDB is not for relational data storage.
  36. 36. Relational data goes in relational databases!
  37. 37. Choose the correct NoSQL database for your needs.
  38. 38. Availability, reliability, and consistency. Pick two.
  39. 39. Consider data storage that isn’t a database at all.
  40. 40. Alternative data options• Elastic Search• Redis• S3• The File System (Yes! It still exists!)
  41. 41. Fix it now or fix it later.But you will have to fix it.
  42. 42. Question time

×