ManageIQ
Sprint 62 Review - Sprint End June 05, 2017
June 07, 2017
Meeting Start: 10:30am EST
Overview
● Sprint Statistics (Oleg Barenboim)
● Community Update (Carol Chen)
● Architecture (Jason Frey)
● Classic UI (Dan Clarizio)
● Service UI (Chris Kacerguis)
● Providers (Bronagh Sorota)
● Automate (Greg McCullough)
● Platform (Gregg Tanzillo)
● API (Alberto Bellotti)
● SmartState (Richard Oliveri)
● Documentation (Chris Budzilowicz)
● Discussion
All Repo Stats - Top 10
(O. Barenboim)
Top Repositories # Closed
ManageIQ/manageiq 132
ManageIQ/manageiq-ui-classic 80
ManageIQ/integration_tests 66
ManageIQ/manageiq-gems-pending 29
ManageIQ/manageiq-docs 25
ManageIQ/manageiq-ui-service 20
ManageIQ/manageiq-pods 13
ManageIQ/manageiq-providers-openstack 12
ManageIQ/vmware_web_service 11
ManageIQ/guides 10
Total of 493 closed across ALL ManageIQ Organization
Pull Requests from: 2017-05-22 00:00:00 UTC to: 2017-06-05 00:00:00 UTC
New repositories this sprint:
activerecord-id_regions
container-httpd
manageiq-network_discovery
manageiq-providers-ansible_tower
manageiq-providers-foreman
manageiq-providers-google
manageiq-providers-scvmm
manageiq-smartstate
manageiq-web_service
Merged PR Type Distribution across
ALL ManageIQ repositories
PRs Merged per Sprint across ManageIQ Repositories
(O. Barenboim)
Repository Health Report
June 06, 2017
Repository Files GPA Issues % Coverage
manageiq-ui-classic 2,455 1.92 9379 63%
manageiq 1903 2.78 1558 NA
manageiq-ui-service 311 4.00 0 60%
manageiq-content 252 2.15 248 63%
manageiq-providers-openstack 151 2.40 220 61%
manageiq-providers-amazon 111 2.30 152 77%
manageiq-providers-vmware 88 2.21 106 61%
manageiq-providers-ovirt 79 2.68 62 78%
manageiq-providers-azure 68 2.83 38 66%
manageiq-providers-hawkular 45 .2.48 12 58%
manageiq-providers-kubernetes 40 2.43 30 85%
manageiq-providers-lenovo 31 4.00 4 76%
manageiq-providers-openshift 27 3.00 3 76%
● Releases
○ Fine-2: http://manageiq.org/blog/2017/06/Announcing-Fine-2/
● Last Week in ManageIQ
○ http://manageiq.org/blog/2017/05/gearing-up-for-Gaprindashvili-and-beyond/ by
Laura Galis
● Upcoming Events
○ Ruby User Group meetup, June 8 in Vienna, Austria - David presenting
https://www.meetup.com/vienna-rb/events/239689512/
○ AnsibleFest, June 22 in London, UK - ManageIQ demo pod (Sergio, Carol)
https://www.ansible.com/ansiblefest/london
● Past event recap
○ OpenExpo 2017: https://twitter.com/i/moments/872194268747751424
Community Update
(Carol Chen)
Architecture
(Jason Frey)
● Running on Kubernetes
○ Apache httpd podification (new repo)
○ Not Privileged
○ Authentication
○ Podified workers
● Dependency minimization
○ To reduce memory footprint
○ To aid in transition to running on Kubernetes
○ manageiq-gems-pending repo being emptied
○ Extracting each worker to separate repo
Architecture
(Jason Frey)
● MiqQueue Investigation
○ Biggest users are Provider Workers
■ Reviewing design to avoid MiqQueue altogether
○ Multiple Use Cases
■ Message Bus (ActiveMQ)
■ Background Job Queue (Sidekiq)
Architecture
(Jason Frey)
● Provider Investigation
○ Metrics
■ Reviewing TSDB technologies
■ Reviewing algorithms to avoid MiqQueue
○ Inventory
■ Collector/Persister Split
■ Graph Refresh
■ Streaming? (Kafka)
Classic UI
(Dan Clarizio)
● PRs merged (67)
○ Enhancements (8)
○ Tech Debt / Refactoring (16)
○ Bugs (38)
Classic UI
Enhancements
● Pluggable menus - allows declaration of menu items in external plugins
● Added quick and advanced search to Physical Servers list view
Classic UI
Responsive textarea selectors
Old
New
Service UI
(Chris Kacerguis)
Stories by Type (20 total):
- Bugs = 9
- Chores = 9
- Features = 1
Stories by section:
- Authentication: 4
- Branding: 1
- Service Details: 3
- Global / Housekeeping: 6
- Testing: 6
Service UI
(Chris Kacerguis)
- New Feature: Rewritten Skinning
- Images are just loaded from the /images dir (instead of compressed)
- No code rebuild is needed when changing the logos
- Webpack handles the setting of the URL for production
- Bug Fixes
- Focus areas: Authentication system, skinning, custom buttons, and sessions
- Chores
- Continued our push with unit testing (several new files now have 100% coverage)
- Dependency updates
Providers
(Bronagh Sorota)
Providers PR Distribution across
Repositories
Provider PRs
kubernetes/openshift 13
openstack 14
ovirt/rhevm 9
amazon 2
azure 3
vmware 4
ansible_tower 8
lenovo/physical infra 4
Providers - Core
● Four provider repos have been split out with dedicated
gitter channels:
○ manageiq-providers-google
○ manageiq-providers-scvmm
○ manageiq-providers-foreman
○ manageiq-providers-ansible_tower
Providers - Core
● Targeted refresh within Graph refresh has been simplified
● Send batches of inventory to the DB without having to
keep it all in memory
● Concurrent save strategy, more work to follow
Automate
(Greg McCullough)
PRs Merged: 19
4 - Enhancements, 10 - Bugs, 4 - TechDebt, 1 - Refactoring
● Ansible Playbook - Catalog Item
○ Add Enhanced Debug level support (Back-end)
● Add automate methods for VM import between Infra providers
○ Allow migration of VMs from VMware infra provider to RHV provider
○ Support for driver ISO selection
Platform
(Gregg Tanzillo)
PRs Merged: 40
3- Enhancements, 17 - Bugs, 11 - TechDebt, 7 - Refactoring, 2 - Performance
● RBAC
○ Fixed descendant matching for Cloud Networks
● Chargeback
○ Removed total columns that do not make sense
○ (allocated + used)
● Added timeout for active tasks
○ For cases when the worker servicing the task is killed
○ Status set to Error after 6 hours of inactivity
Platform
● Fixed report import to properly handle json originating from API
● Performance
○ Optimized ChargeableField.seed to improve server startup speed
○ Memoize root tenant to improve loading available columns in report editor
● Appliance CLI
○ Support for configuring dedicated DB appliance (rsevilla87)
○ Support for setting time zone (aljesusg)
○ Server start/stop/status (aljesusg)
API
(Alberto Bellotti)
API v2.4.0 Documentation
(Fine Release)
is now available:
http://manageiq.org/docs/api
API
● Added SQL store capability to the Token Manager (Tim W.)
○ Used to only support memcache
○ Was problematic with API workers behind a load balancer
○ New SQL store is automatically configured based on the global session store setting
API
● Added ability to query cloud subnet (Gilles D.)
○ Queries of cloud_subnets as primary collection:
■ GET /api/cloud_subnets
■ GET /api/cloud_subnets/:id
○ Supporting bulk queries of cloud subnets:
■ POST /api/cloud_subnets - action “query”
○ Queries of the cloud_subnets subcollection of providers:
■ GET /api/providers/:id/cloud_subnets
■ GET /api/providers/:id/cloud_subnets/:cloud_subnet_id
API
● Added ability to set the miq_server on VMs (Jillian T.)
○ Provided via the set_miq_server action on resources
○ Supports server specification by href or id reference.
■ POST /api/vms/:id
{
“action” : “set_miq_server”,
“resource” : {
“miq_server” : { “href” : “/api/servers/11” }
}
}
○ Supporting bulk action of cloud subnets:
■ POST /api/vms - action “query”
SmartState
(Richard Oliveri)
SmartState for Amazon
● MIQ appliance side code merged - request via SQS.
○ Repositories:
■ manageiq
■ manageiq-providers-amazon
■ amazon_ssa_support
● amazon_ssa_support updated to use the new
manageiq-smartstate gem.
● PRs merged (20)
○ Enhancements 16
○ Bug fixes 4
● Highlights
○ #390 Added a step to update the database schema to the HA update instructions.
○ #385 Added instructions for obtaining self-signed certificates for Hawkular.
○ #379 Added instructions for configuring trust for self-signed SSL certificates.
○ #378 Updated the memory requirements for ManageIQ.
○ #375 Updated content on the SSUI to match the latest UI and available actions.
○ #370 Updated the structure of the Managing Providers guide.
Documentation
(C. Budzilowicz)
Discussion
Sprint 63 Review - June 21
Meeting Start: 10:30am EST

Sprint 62

  • 1.
    ManageIQ Sprint 62 Review- Sprint End June 05, 2017 June 07, 2017 Meeting Start: 10:30am EST
  • 2.
    Overview ● Sprint Statistics(Oleg Barenboim) ● Community Update (Carol Chen) ● Architecture (Jason Frey) ● Classic UI (Dan Clarizio) ● Service UI (Chris Kacerguis) ● Providers (Bronagh Sorota) ● Automate (Greg McCullough) ● Platform (Gregg Tanzillo) ● API (Alberto Bellotti) ● SmartState (Richard Oliveri) ● Documentation (Chris Budzilowicz) ● Discussion
  • 3.
    All Repo Stats- Top 10 (O. Barenboim) Top Repositories # Closed ManageIQ/manageiq 132 ManageIQ/manageiq-ui-classic 80 ManageIQ/integration_tests 66 ManageIQ/manageiq-gems-pending 29 ManageIQ/manageiq-docs 25 ManageIQ/manageiq-ui-service 20 ManageIQ/manageiq-pods 13 ManageIQ/manageiq-providers-openstack 12 ManageIQ/vmware_web_service 11 ManageIQ/guides 10 Total of 493 closed across ALL ManageIQ Organization Pull Requests from: 2017-05-22 00:00:00 UTC to: 2017-06-05 00:00:00 UTC New repositories this sprint: activerecord-id_regions container-httpd manageiq-network_discovery manageiq-providers-ansible_tower manageiq-providers-foreman manageiq-providers-google manageiq-providers-scvmm manageiq-smartstate manageiq-web_service
  • 4.
    Merged PR TypeDistribution across ALL ManageIQ repositories
  • 5.
    PRs Merged perSprint across ManageIQ Repositories (O. Barenboim)
  • 6.
    Repository Health Report June06, 2017 Repository Files GPA Issues % Coverage manageiq-ui-classic 2,455 1.92 9379 63% manageiq 1903 2.78 1558 NA manageiq-ui-service 311 4.00 0 60% manageiq-content 252 2.15 248 63% manageiq-providers-openstack 151 2.40 220 61% manageiq-providers-amazon 111 2.30 152 77% manageiq-providers-vmware 88 2.21 106 61% manageiq-providers-ovirt 79 2.68 62 78% manageiq-providers-azure 68 2.83 38 66% manageiq-providers-hawkular 45 .2.48 12 58% manageiq-providers-kubernetes 40 2.43 30 85% manageiq-providers-lenovo 31 4.00 4 76% manageiq-providers-openshift 27 3.00 3 76%
  • 7.
    ● Releases ○ Fine-2:http://manageiq.org/blog/2017/06/Announcing-Fine-2/ ● Last Week in ManageIQ ○ http://manageiq.org/blog/2017/05/gearing-up-for-Gaprindashvili-and-beyond/ by Laura Galis ● Upcoming Events ○ Ruby User Group meetup, June 8 in Vienna, Austria - David presenting https://www.meetup.com/vienna-rb/events/239689512/ ○ AnsibleFest, June 22 in London, UK - ManageIQ demo pod (Sergio, Carol) https://www.ansible.com/ansiblefest/london ● Past event recap ○ OpenExpo 2017: https://twitter.com/i/moments/872194268747751424 Community Update (Carol Chen)
  • 8.
    Architecture (Jason Frey) ● Runningon Kubernetes ○ Apache httpd podification (new repo) ○ Not Privileged ○ Authentication ○ Podified workers ● Dependency minimization ○ To reduce memory footprint ○ To aid in transition to running on Kubernetes ○ manageiq-gems-pending repo being emptied ○ Extracting each worker to separate repo
  • 9.
    Architecture (Jason Frey) ● MiqQueueInvestigation ○ Biggest users are Provider Workers ■ Reviewing design to avoid MiqQueue altogether ○ Multiple Use Cases ■ Message Bus (ActiveMQ) ■ Background Job Queue (Sidekiq)
  • 10.
    Architecture (Jason Frey) ● ProviderInvestigation ○ Metrics ■ Reviewing TSDB technologies ■ Reviewing algorithms to avoid MiqQueue ○ Inventory ■ Collector/Persister Split ■ Graph Refresh ■ Streaming? (Kafka)
  • 11.
    Classic UI (Dan Clarizio) ●PRs merged (67) ○ Enhancements (8) ○ Tech Debt / Refactoring (16) ○ Bugs (38)
  • 12.
    Classic UI Enhancements ● Pluggablemenus - allows declaration of menu items in external plugins ● Added quick and advanced search to Physical Servers list view
  • 13.
  • 14.
    Service UI (Chris Kacerguis) Storiesby Type (20 total): - Bugs = 9 - Chores = 9 - Features = 1 Stories by section: - Authentication: 4 - Branding: 1 - Service Details: 3 - Global / Housekeeping: 6 - Testing: 6
  • 15.
    Service UI (Chris Kacerguis) -New Feature: Rewritten Skinning - Images are just loaded from the /images dir (instead of compressed) - No code rebuild is needed when changing the logos - Webpack handles the setting of the URL for production - Bug Fixes - Focus areas: Authentication system, skinning, custom buttons, and sessions - Chores - Continued our push with unit testing (several new files now have 100% coverage) - Dependency updates
  • 16.
  • 17.
    Providers PR Distributionacross Repositories Provider PRs kubernetes/openshift 13 openstack 14 ovirt/rhevm 9 amazon 2 azure 3 vmware 4 ansible_tower 8 lenovo/physical infra 4
  • 18.
    Providers - Core ●Four provider repos have been split out with dedicated gitter channels: ○ manageiq-providers-google ○ manageiq-providers-scvmm ○ manageiq-providers-foreman ○ manageiq-providers-ansible_tower
  • 19.
    Providers - Core ●Targeted refresh within Graph refresh has been simplified ● Send batches of inventory to the DB without having to keep it all in memory ● Concurrent save strategy, more work to follow
  • 20.
    Automate (Greg McCullough) PRs Merged:19 4 - Enhancements, 10 - Bugs, 4 - TechDebt, 1 - Refactoring ● Ansible Playbook - Catalog Item ○ Add Enhanced Debug level support (Back-end) ● Add automate methods for VM import between Infra providers ○ Allow migration of VMs from VMware infra provider to RHV provider ○ Support for driver ISO selection
  • 21.
    Platform (Gregg Tanzillo) PRs Merged:40 3- Enhancements, 17 - Bugs, 11 - TechDebt, 7 - Refactoring, 2 - Performance ● RBAC ○ Fixed descendant matching for Cloud Networks ● Chargeback ○ Removed total columns that do not make sense ○ (allocated + used) ● Added timeout for active tasks ○ For cases when the worker servicing the task is killed ○ Status set to Error after 6 hours of inactivity
  • 22.
    Platform ● Fixed reportimport to properly handle json originating from API ● Performance ○ Optimized ChargeableField.seed to improve server startup speed ○ Memoize root tenant to improve loading available columns in report editor ● Appliance CLI ○ Support for configuring dedicated DB appliance (rsevilla87) ○ Support for setting time zone (aljesusg) ○ Server start/stop/status (aljesusg)
  • 23.
    API (Alberto Bellotti) API v2.4.0Documentation (Fine Release) is now available: http://manageiq.org/docs/api
  • 24.
    API ● Added SQLstore capability to the Token Manager (Tim W.) ○ Used to only support memcache ○ Was problematic with API workers behind a load balancer ○ New SQL store is automatically configured based on the global session store setting
  • 25.
    API ● Added abilityto query cloud subnet (Gilles D.) ○ Queries of cloud_subnets as primary collection: ■ GET /api/cloud_subnets ■ GET /api/cloud_subnets/:id ○ Supporting bulk queries of cloud subnets: ■ POST /api/cloud_subnets - action “query” ○ Queries of the cloud_subnets subcollection of providers: ■ GET /api/providers/:id/cloud_subnets ■ GET /api/providers/:id/cloud_subnets/:cloud_subnet_id
  • 26.
    API ● Added abilityto set the miq_server on VMs (Jillian T.) ○ Provided via the set_miq_server action on resources ○ Supports server specification by href or id reference. ■ POST /api/vms/:id { “action” : “set_miq_server”, “resource” : { “miq_server” : { “href” : “/api/servers/11” } } } ○ Supporting bulk action of cloud subnets: ■ POST /api/vms - action “query”
  • 27.
    SmartState (Richard Oliveri) SmartState forAmazon ● MIQ appliance side code merged - request via SQS. ○ Repositories: ■ manageiq ■ manageiq-providers-amazon ■ amazon_ssa_support ● amazon_ssa_support updated to use the new manageiq-smartstate gem.
  • 28.
    ● PRs merged(20) ○ Enhancements 16 ○ Bug fixes 4 ● Highlights ○ #390 Added a step to update the database schema to the HA update instructions. ○ #385 Added instructions for obtaining self-signed certificates for Hawkular. ○ #379 Added instructions for configuring trust for self-signed SSL certificates. ○ #378 Updated the memory requirements for ManageIQ. ○ #375 Updated content on the SSUI to match the latest UI and available actions. ○ #370 Updated the structure of the Managing Providers guide. Documentation (C. Budzilowicz)
  • 29.
    Discussion Sprint 63 Review- June 21 Meeting Start: 10:30am EST