Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Optimizing the Performance-Related Configurations of Object-Relational Mapping Frameworks Using a Multi-Objective Genetic Algorithm

430 views

Published on

ICPE 2016 presentation

Published in: Science
  • Be the first to comment

Optimizing the Performance-Related Configurations of Object-Relational Mapping Frameworks Using a Multi-Objective Genetic Algorithm

  1. 1. Optimizing the Performance-RelatedOptimizing the Performance-Related Configurations of ORM FrameworksConfigurations of ORM Frameworks Ravjot Singh, Cor-Paul Bezemer,Weiyi Shang,Ahmed E. Hassan Using a Multi-Objective Genetic AlgorithmUsing a Multi-Objective Genetic Algorithm
  2. 2. 2 Configuration is DIFFICULTConfiguration is DIFFICULT
  3. 3. 3 27% of customer issues are27% of customer issues are due to misconfigurationdue to misconfiguration
  4. 4. 4 20% of misconfigurations cause20% of misconfigurations cause performance degradationsperformance degradations
  5. 5. 5 A popular online store estimates a $1.6A popular online store estimates a $1.6 billion loss for a 1-second slowdownbillion loss for a 1-second slowdown
  6. 6. 6 Databases are at the coreDatabases are at the core of many large systemsof many large systems
  7. 7. 7 Software developers knowSoftware developers know how to think in objects...how to think in objects...
  8. 8. 8 But have less experienceBut have less experience with databaseswith databases
  9. 9. 9 Object-relational mappingObject-relational mapping (ORM) helps developers(ORM) helps developers CODE
  10. 10. 10 Code without ORM isCode without ORM is tedioustedious public class Person { // ... public String getName(){ sql = “SELECT name FROM … WHERE …”; ResultSet rs = stmt.executeQuery(sql); while(rs.next()) return rs.getString(“name”); } }
  11. 11. 11 Code with ORM is cleanCode with ORM is clean @Entity public class Person { @Id Integer getId() { ... } public String getName(){ return this.name; } }
  12. 12. 12 Several popular ORMSeveral popular ORM frameworks existframeworks exist
  13. 13. 13 ORM has many configurationORM has many configuration optionsoptions
  14. 14. 14 ORM configuration is…ORM configuration is… guess what?guess what?
  15. 15. 15 ORM configuration is…ORM configuration is… guess what? Difficult!guess what? Difficult!
  16. 16. 16 The impact of one ORMThe impact of one ORM configuration optionconfiguration option hibernate.max_fetch_depth = {0|1|2|3|...}
  17. 17. 17 The impact of one ORMThe impact of one ORM configuration optionconfiguration option hibernate.max_fetch_depth = {0|1|2|3|...} Optimal configuration depends on what you need!
  18. 18. 18 Should we care about ORMShould we care about ORM performance configuration?performance configuration? Analyze 11 boolean configuration options hence 211 = 2048 configurations
  19. 19. 19 We compare the 'default' withWe compare the 'default' with the optimal configurationthe optimal configuration Default configuration System Tests Run tests Default execution time
  20. 20. 20 We compare the 'default' withWe compare the 'default' with the optimal configurationthe optimal configuration Default configuration System Tests Run tests Optimal configuration System Tests Default execution time Optimal execution time Run tests
  21. 21. 21 We compare the 'default' withWe compare the 'default' with the optimal configurationthe optimal configuration Default configuration System Tests Run tests Optimal configuration System Tests Compare Default execution time Optimal execution time Run tests
  22. 22. 22 Yes, we should care about ORMYes, we should care about ORM performance configuration!performance configuration! 89% 96% of the test cases are significantly slower using default!
  23. 23. 23 How can we guide ORMHow can we guide ORM performance configuration?performance configuration?
  24. 24. 24 Optimize ORM configurationOptimize ORM configuration by trial-and-errorby trial-and-error Randomly select configuration A
  25. 25. 25 Optimize ORM configurationOptimize ORM configuration by trial-and-errorby trial-and-error Evaluate configuration A on some workload W Randomly select configuration A
  26. 26. 26 Optimize ORM configurationOptimize ORM configuration by trial-and-errorby trial-and-error Evaluate configuration A on some workload W Compare execution time for configuration A with execution time for current best configuration Randomly select configuration A
  27. 27. 27 Optimize ORM configurationOptimize ORM configuration by trial-and-errorby trial-and-error Evaluate configuration A on some workload W Compare execution time for configuration A with execution time for current best configuration Randomly select configuration A Use best configuration
  28. 28. 28 Optimize ORM configurationOptimize ORM configuration by trial-and-errorby trial-and-error Evaluate configuration A on some workload W Compare execution time for configuration A with execution time for current best configuration Randomly select configuration A Use best configuration
  29. 29. 29 An evolutionary/genetic algorithmAn evolutionary/genetic algorithm guides this trial-and-error processguides this trial-and-error process
  30. 30. 30 Genetic algorithm conceptGenetic algorithm concept Optimize population based on one or more objectives We start from the 'default' configuration as supplied by the developer
  31. 31. 31 Genetic algorithm conceptGenetic algorithm concept 1 1 0 0 0 0 0 0 1 0 0 1 0 0 0 1+
  32. 32. 32 Genetic algorithm conceptGenetic algorithm concept 1 1 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 1 0 0 0 0 0 1 +
  33. 33. 33 Genetic algorithm conceptGenetic algorithm concept 1 1 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 1 0 0 0 0 0 1 1 1 0 0 0 1 0 1 +
  34. 34. 34 Genetic algorithm conceptGenetic algorithm concept 1 1 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 1 0 0 0 1 0 1 Execution time = 50s Execution time = 60s Execution time = 30s
  35. 35. 35 Genetic algorithm conceptGenetic algorithm concept 1 1 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 1 0 0 0 1 0 1 Execution time = 50s Execution time = 60s Execution time = 30s
  36. 36. 36 Optimizing ORMOptimizing ORM performance configurationperformance configuration Objectives: CPU Memory Execution time
  37. 37. 37 Optimizing ORMOptimizing ORM performance configurationperformance configuration Objectives: CPU Memory Execution time Calculate Δ with default configuration
  38. 38. 38 Evaluation of our approachEvaluation of our approach 1. Closeness of configurations found by our approach to the optimal configuration 2. Speed with which we can find sub- optimal configurations
  39. 39. 39 ExperimentsExperiments Default configuration Tests Repeat 100 times Optimal configuration
  40. 40. 40 Closeness of configurationsCloseness of configurations to the optimalto the optimal We rank all existing configurations based on dominance Configuration A dominates configuration B if: 1. B is not better than A for all objectives 2.A is better than B for at least one objective
  41. 41. 41 Ranking of configurationsRanking of configurations Configuration CPU usage Memory usage Execution time Default 0 0 0 A +50% +50% 0 B -80% -20% 0 C 0 -80% -20%
  42. 42. 42 Ranking of configurationsRanking of configurations Configuration CPU usage Memory usage Execution time Default 0 0 0 A +50% +50% 0 B -80% -20% 0 C 0 -80% -20% Rank 1: B and C Rank 2: default Rank 3:A
  43. 43. 43 The genetic algorithm findsThe genetic algorithm finds configurations close to the best rankconfigurations close to the best rank BEST WORST
  44. 44. 44 Speed with whichSpeed with which configurations are foundconfigurations are found Depends on: - Workload - Application - Data
  45. 45. 45 The genetic algorithm findsThe genetic algorithm finds configurations fast (~5-20 minutes)configurations fast (~5-20 minutes)
  46. 46. 46 Yes, we should care about ORMYes, we should care about ORM performance configuration!performance configuration! 89% 96% of the test cases are significantly slower using default!
  47. 47. 47
  48. 48. 48 How can we guide ORMHow can we guide ORM performance configuration?performance configuration?
  49. 49. 49
  50. 50. 50 The genetic algorithm findsThe genetic algorithm finds configurations close to the optimal rankconfigurations close to the optimal rank
  51. 51. 51
  52. 52. 52 The genetic algorithm findsThe genetic algorithm finds configurations fast (~5-20 minutes)configurations fast (~5-20 minutes)
  53. 53. 53
  54. 54. 54 Cor-Paul Bezemer bezemer@cs.queensu.ca http://sailhome.cs.queensu.ca/~corpaul/
  55. 55. 55
  56. 56. 56 Studied configuration optionsStudied configuration options order_updates jdbc.batch_size order_inserts connection.release_mode default_batch_fetch_size jdbc.batch_versioned_data max_fetch_depth id.new_generator_mappings jdbc.fetch_size bytecode.use_reflection_optimizer cache.use_second_level_cache
  57. 57. 57 Yes, we should care about ORMYes, we should care about ORM performance configuration!performance configuration!

×