Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Sprint 49 review

335 views

Published on

Sprint Statistics (Oleg Barenboim)
Community Update (Carol Chen)
Classic UI (Dan Clarizio)
Service UI (Chris Kacerguis)
Providers (Adam Grare)
Automate (Greg McCullough)
Platform (Gregg Tanzillo)
API (Alberto Bellotti)
Performance (Dennis Metzger)
Quality Engineering (Dave Johnson)
Discussion

Published in: Software
  • Be the first to comment

  • Be the first to like this

Sprint 49 review

  1. 1. ManageIQ Sprint 49 Review - Sprint End November 14, 2016 November 16, 2016
  2. 2. Overview ● Sprint Statistics (Oleg Barenboim) ● Community Update (Carol Chen) ● Classic UI (Dan Clarizio) ● Service UI (Chris Kacerguis) ● Providers (Adam Grare) ● Automate (Greg McCullough) ● Platform (Gregg Tanzillo) ● API (Alberto Bellotti) ● Performance (Dennis Metzger) ● Quality Engineering (Dave Johnson) ● Discussion
  3. 3. Sprint Statistics (O. Barenboim) 354 Pull Requests Merged for ManageIQ/manageiq
  4. 4. Sprint Statistics - Euwe Backports (O. Barenboim) 162 Pull Requests Backported to Euwe
  5. 5. PR Breakdown by Feature Category (O. Barenboim) * Note that some PRs have more than one category. Providers
  6. 6. Sprint Comparison (O. Barenboim) Darga-2 DargaBeta Darga-3 DargaGA DargaRC Darga-4 EuweBeta EuweRC
  7. 7. All Repo Stats (O. Barenboim) Repo Closed manageiq 354 integration_tests 118 manageiq-ui-service 35 manageiq.org 14 manageiq-providers-amazon 12 azure-armest 9 manageiq-api-client 7 manageiq-pods 7 manageiq-providers-lenovo 7 manageiq-providers-azure 6 manageiq-appliance-build 5 wrapanapi 5 guides 4 miq_bot 3 manageiq-appliance 2 manageiq-gems-pending 2 ui-components 2 Total 592
  8. 8. Community Update (Carol Chen) ● 82% more pageviews than previous 4-week period ● 48% pageviews are to /docs/ pages (compared to 28% in previous period)
  9. 9. Community Update ● Recent events ○ KubeCon (Nov 8-9, Seattle) - Jerome Marc with ManageIQ demo ○ RubyConf (Nov 10-12, Cincinnati) - Chris Arcand presented “Deletion Driven Development” ● Upcoming events ○ DevConf.CZ (Jan 27-29, 2017, Brno) ○ FOSDEM (Feb 4-5, 2017, Brussels) ○ If you have talks/activities related to ManageIQ, please email carchen@redhat.com to coordinate ManageIQ presence and promotion ● Voting for F-release name ○ http://talk.manageiq.org/t/voting-on-name-of-f-release/1881 ○ Past releases: B (13 votes), C (16 votes), D (27 votes) ○ E-release naming garnered 53 votes in total ○ F-release currently at 70 votes, with 3 more days to go. Vote vote vote! ● Metrics (past 6 months) ○ GitHub: stars 33%⬆, contributors 36%⬆, forks 37%⬆ ○ Talk Forum: users 24%⬆, posts 30%⬆ ○ Social media channels: 12-30%⬆
  10. 10. Classic UI (Dan Clarizio) ● PRs merged (165) ○ Bugs (115) ○ Enhancements (16) ○ Tech Debt / Refactoring (22/11) ○ Enhancements Backported to EUWE (9) ● Bug Fixes ○ Charting/Topology/Toolbar buttons cleanup ○ Regressions from VM tree changes ○ I18n updates ○ Lots of EUWE fixes
  11. 11. Classic UI ● Tech Debt / Refactoring ○ Removal of old chart and timeline code ○ De-duplication PRs ○ Toolbars - Too many (11) to list, 25 out of 31 completed! ● Enhancements ○ Add/remove Hosts to/from Host Aggregates ○ Advanced search added to Containers ○ Containers - Label based Auto-Tagging
  12. 12. Classic UI Add/remove Hosts to/from Host Aggregates
  13. 13. Classic UI Advanced search added to Containers
  14. 14. Classic UI Label based Auto-Tagging
  15. 15. Service UI (Chris Kacerguis) ● Numerous bug fixes (~35 PRs, 18 bug fixes) ● Lots of technical debt resolved ● Added more Unit Tests (59% test coverage) ● Added yarn for dependency management ● Removed bower ● Added Power Operations in for VMs
  16. 16. Providers (Adam Grare)
  17. 17. Hawkular (Adam Grare) Lead with 19 PRs merged, 18 Bugs fixed Added a way to overwrite an existing deployment
  18. 18. RHV (Adam Grare) Remove full refresh when provisioning a new VM ○ Provision to a large env reduced from 17min to 1min ○ Uses the improved refresh for new VM from last sprint VM Live Migration added Added API to import VMs from VMware to RHEV
  19. 19. OpenStack (Adam Grare) Added support for new Gnocchi metrics service Backend support for creating floating IPs using task queue
  20. 20. Amazon (Adam Grare) Cloud Manager refresh moved to new DTO (Data Transfer Object) method
  21. 21. Azure (Adam Grare) Has a new home at https://github.com/ManageIQ/manageiq-providers-azure
  22. 22. Lenovo (Adam Grare) Phase 1 of physical server refresh
  23. 23. VMware vCloud (Adam Grare) Credentials for vCloud Events can be added in the UI
  24. 24. Pluggable Providers (Adam Grare) ● supports :clone ● supports :suspend ● supports :reset
  25. 25. Automate (Greg McCullough) Service Models ● Expose ems_events for vm_or_template Notifications ● Added Tenant/Global notification audiences Service ● Expose service power state
  26. 26. Automate (Greg McCullough) Orchestration ● Show Azure orchestration stack failure ○ Scan through its operations and delegate the error message to the stack ● Set zone when delivering a service template provision task ● Link a VM to the service through the containing orchestration stack
  27. 27. Automate (Greg McCullough) Fixes ● Automate schedule ○ key/pair attributes fixed to match Simulation ● MiqGroup ○ filter relationship corrected
  28. 28. Automate (Greg McCullough) Fixes ● Git Repository ○ Support repositories without the domain directory ■ SimpleDomain (Repository) ● SimpleDomain (Directory) (old style) ○ __domain__.yaml ■ SimpleDomain (Repository) ● __domain__.yaml ○ Refresh was not updating the contents and new references. ■ New branches and tags were not getting fetched ■ New content was not getting fetched and imported
  29. 29. Platform (Gregg Tanzillo) Enhancements / Bug Fixes ● Centralized Administration ● Chargeback ● Appliance ● PostgreSQL HA
  30. 30. Platform (Gregg Tanzillo) Centralized Administration ● Service Template Provisioning ○ ServiceOrder (shopping cart) creation on remote region ● VM Reconfigure ○ Proper routing to remote regions ○ Create separate request per region
  31. 31. Platform (Gregg Tanzillo) Chargeback ● Reinstated daily rates ● Simplified rates editor ○ Disabled editing of variable rate for a fixed rate tier
  32. 32. Platform (Gregg Tanzillo) Appliance ● Increased default worker memory thresholds ○ Metrics processor worker: 400 to 600 MB ○ Priority worker: 400 to 600 MB ○ Generic worker: 400 to 500 MB ○ Schedule worker: 300 to 500 MB. ● Configurable number of saved logfile rotations
  33. 33. Platform (Gregg Tanzillo) PostgreSQL High Availability ● Raise EVM event on failover ○ “db_failover_executed”
  34. 34. API (Alberto Bellotti) ● New Provider action import_vm which adds the ability to import VMs ○ Currently supporting importing VmWare VMs in RHEV ○ POST /api/providers/:id { “action” : “import_vm”, “resource” : { “source” : { “href” : “http://localhost:3000/api/vms/:source_vm_id” }, “target” : { “name” : “<new_vm_name>”, “cluster” : { “href” : “http://localhost:3000/api/clusters/:cluster_id” }, “data_store” : { “href” : “http://localhost:3000/api/data_stores/:data_store_id” }, “sparse” : true } } }
  35. 35. API (Alberto Bellotti) ● Enhanced /api/policies to support CRUD actions POST /api/policies { "name" : "sample policy", "description" : "sample policy description", "towhat" : "ManageIQ::Providers::Redhat::InfraManager", "conditions_ids" : [ 2, 3 ], "policy_contents" : [{ "event_id" : 2, "actions": [ {"action_id" : 1, "opts" : { "qualifier" : "failure" } } ] }] } POST /api/policies/:id - single resource action edit POST /api/policies - action edit for Bulk edits POST /api/policies/:id - single resource action delete DELETE /api/policies/:id - single resource DELETE POST /api/policies - action delete for Bulk deletes
  36. 36. API (Alberto Bellotti) ● Enhanced /api/conditions to support CRUD actions POST /api/conditions { "name" : "sample condition", "description" : "sample condition description", "expression" : { "=" : { "field" : "ContainerImage-architecture", "value" : "dsa" } }, "towhat" : "ExtManagementSystem", "modifier" : "allow" } POST /api/conditions/:id - single resource action edit POST /api/conditions - action edit for Bulk edits POST /api/conditions/:id - single resource action delete POST /api/conditions - action delete for Bulk deletes
  37. 37. API (Alberto Bellotti) ● New collection /api/actions with CRUD actions Querying actions GET /api/actions GET /api/actions/:id POST /api/actions - action query for Bulk queries Creation actions via: POST /api/actions { "name" : "sample action", "description" : "sample action description", "action_type” : “custom_automation”, "options” : { “ae_message” : “<message>”, “ae_request” : “<request>”, “ae_hash” : { “key” : “value” } }, } POST /api/actions/:id - single resource action edit POST /api/actions - action edit for Bulk edits POST /api/actions/:id - single resource action delete POST /api/actions - action delete for Bulk deletes
  38. 38. API (Alberto Bellotti) ● Added ability to approve and deny /api/requests ○ When requests are not auto-approved, we now support the ability to “approve” and “deny” requests. POST /api/requests/:id { “action” : “approve”, “resource” : { “reason” : “Reason for Request Approval” } } POST /api/requests/:id { “action” : “deny”, “resource” : { “reason” : “Reason for Request Denial” } } POST /api/requests - action “approve” for bulk requests POST /api/requests - action “deny” for bulk requests
  39. 39. API (Alberto Bellotti) ● Id’s and other attributes now returned with Service Dialog contents. GET /api/service_dialogs/2 { "href": "http://localhost:3000/api/service_dialogs/2", "id": 2, ... "content": [ { "id": 2, "description": "Simple Dialog", ... "dialog_tabs": [ { "id": 4, "description": "Host Info", ... "dialog_groups": [ { "id": 7, "description": "Sample Box", … }
  40. 40. API (Alberto Bellotti) ● Added ability to copy Service Dialogs ○ This is made available via the copy action on /api/service_dialogs/:id resources. With this action, the Service Dialog is copied to a new Service Dialog with the label “Copy of <current_dialog_label>” unless one is specified POST /api/service_dialogs/:id { “action” : “copy”, “resource” : { “label” : “copied_service_dialog” } } POST /api/service_dialogs/:id - action copy to copy a single resource POST /api/service_dialogs - action copy to copy resources in bulk
  41. 41. API (Alberto Bellotti) ● Added ability to delete Service Requests ○ POST /api/service_requests/:id - action delete ○ DELETE /api/service_requests/:id ○ POST /api/service_requests - action delete for Bulk deletes
  42. 42. Performance (Dennis Metzger) Team was focused cross-team help manageiq-performance rubygem (Nick) ● Initial focus is on UI performance metrics ● Fundamental component for automated performance regression testing Notable PRs (Keenan) ● Moved to Ancestry 2.2.1 ● Removed an O(n^2) from save inventory ● Reduced time to load desired Resource Pools ● That Targeted Refresh thing (well refresh in general)
  43. 43. Performance (manageiq-performance) Collect & Report on performance metrics for MIQ ● single configurable and pluggable middleware for gathering data ● accompanying cli interface for benchmarking, reporting and analyzing ● improve automation of performance metrics Designed for QE, Perf Team, Developers, Support ● Unify toolchain so that everyone only has to learn one Borrows heavily from existing tools/frameworks: ● rack-mini-profiler ● cfme-performance ● perf_utils (@kbrock)
  44. 44. Performance (manageiq-performance - middleware) Activated by a header (WITH_PERFORMANCE_MONITORING) Pluggable ● Request timers and info ● ActiveRecord data ● stackprof profiling (if installed) Configurable ● Multiple storage backends ● "Browser Mode" (activate via url param / "always on") ● Configure/enable plugins
  45. 45. Performance (manageiq-performance) ● Simplify testing of app changes ● Automate gathering and reporting of metrics miqperf benchmark $ bundle exec miqperf benchmark /service/explorer/s-1000000000001 --count 5 --> getting csrf_token... --> logging in... --> making GET request: /service/explorer/s-1000000000001 --> making GET request: /service/explorer/s-1000000000001 --> making GET request: /service/explorer/s-1000000000001 --> making GET request: /service/explorer/s-1000000000001 --> making GET request: /service/explorer/s-1000000000001 $ bundle exec miqperf benchmark --requestfile ...
  46. 46. Performance (manageiq-performance - CLI) Requestfile GET => /auth_key_pair_cloud GET => /auth_key_pair_cloud/index GET => /auth_key_pair_cloud/show GET => /auth_key_pair_cloud/show_list POST => /auth_key_pair_cloud/show POST => /auth_key_pair_cloud/show_list GET => /availability_zone GET => /availability_zone/index GET => /availability_zone/show GET => /availability_zone/show_list POST => /availability_zone/show POST => /availability_zone/show_list GET => /catalog GET => /catalog/explorer GET => /catalog/show/1000000000001 POST => /catalog/explorer ... $ bin/rake manageiq_performance:build_request_file
  47. 47. Performance (manageiq-performance - CLI cont.) miqperf report $ bundle exec miqperf report --last service-explorer-s-1000000000001 | ms | queries | query (ms) | rows | | ---: | ---: | ---: | ---: | | 4166 | 203 | 286.6 | 7065 | | 2446 | 203 | 202.7 | 7065 | | 2328 | 203 | 205.5 | 7065 | | 2354 | 203 | 204.2 | 7065 | | 2590 | 203 | 208.3 | 7065 |
  48. 48. Performance (manageiq-performance - CLI cont.) miqperf analyze $ bundle exec miqperf analyze --last tmp/manageiq_performance/run_1476917567/service%explorer%s-1000000000001 ================================== Mode: wall(1000) Samples: 42877 (0.76% miss rate) GC: 6207 (14.48%) ================================== TOTAL (pct) SAMPLES (pct) FRAME 2535 (5.9%) 2535 (5.9%) Puma::Single#run 1357 (3.2%) 1338 (3.1%) block in ActiveRecord::Co... 1293 (3.0%) 1243 (2.9%) block in Sprockets::Resol... 935 (2.2%) 935 (2.2%) block in ActiveRecord::Re… ... $ bundle exec miqperf analyze --last --method "Sprockets::Resolve#dirname_matches"
  49. 49. Performance (manageiq-performance - TODO) Future improvements and enhancements ● More plugins (memory, method profiling, simple timers, etc.) ● More storage backends (statsd, ELK support, etc.) ● Better reporting ● Browser user interface (for those who dislike the command line) ● MiqQueue monitoring ● IRB profile helpers (in progress...) ● Headless browser metrics ● User feedback (wink wink)
  50. 50. Performance - ancestry (kbrock) Ancestry ilike -> like - 120ms / same rows count_or_objects - 720ms / ½ queries Using scopes to access hosts - 350ms / -520 rows Relationship arrange filters /ems_infra/?view=show_folders | ms |queries| query (ms)| rows | comments | ---:| ---:| ---:| ---:| --- | 1,917.3 | 290 | 962.9 | 1,927 | before | 707.9 | 230 | 156.7 | 235 | after | 63.1% | 21% | 83.7% | 88% |
  51. 51. Performance - vm/explorer (kbrock) vm_infra/explorer (23k vms) Finished up tree work Ancestry (filtering arranged), parents, resource pools | ms |queries | query (ms) | rows | comments | ---:| ---:| ---:| ---:| --- | 31,738.2 | 130 | 3,981.9 | 68,205 | before | 3,370.4 | 128 | 1,390.8 | 5,548 | after | 89.4% | 2% | 65.1% | 92% |
  52. 52. Quality Engineering (Dave Johnson) ○ Continuing with bug verification ○ Addressing concerns over the number of open issues and rate of incoming bugs ○ Significant effort going into bug re-triage in an effort to increase quality
  53. 53. Quality Engineering (Dave Johnson) Automation ● Seeing infinnispinny (infinite spinners) trying to find a way to catch it as it’s sporadic ● Navmazing (new navigation framework) conversion ○ 677 -> 309 removals of old force_navigate ○ More in PRs yet to be merged ● Still a fair few locator/text changes coming in
  54. 54. Quality Engineering (Dave Johnson) Integration Tests 114 PRs Merged
  55. 55. Discussion Sprint 50 Review - December 7

×