ManageIQ
Sprint 20 Report - Sprint End March 9, 2015
March 11, 2015
Overview
● Sprint Statistics
● Providers (G. Blomquist, J. Frey)
● Rest API (A. Bellotti)
● UI Enhancements (D. Clarizio)
● Orchestration, Retirement, Automate(G. McCullough)
● Rails 4, Technical Debt (A. Patterson)
● IPv6, STIG (J. Rafaniello)
● Fleecing (Rich O.)
● Community Update (JMW)
Sprint Statistics
● https://github.com/ManageIQ/manageiq/issues?q=milestone%3A%22Sprin
t+20+Ending+Mar+9%2C+2015%22+is%3Amerged
● 249 Pull Requests Merged!!!
○ 112 issues labeled as “bug”
○ 76 issues labeled as “enhancement”
○ 11 issues labeled as “technical debt”
○ 35 issues labeled as “refactoring”
○ 9 issues labeled as “test”
● 84 Open PRs
○ 27 issues labeled as “WIP”
Providers (G. Blomquist)
● OpenStack Infrastructure
○ Heat stacks
○ Images
○ More metrics
● Kubernetes (Containers)
○ Event catcher
Providers (J. Frey)
● Foreman integration
○ Reorganized backend tables of managers by
consolidating into ext_management_systems
○ Verification of credentials
○ Workers for EMS Refresh
○ Provisioning
■ Internal state-machine logic merged
■ Workflow refactoring
■ Researched Organization/Location support
○ Reporting may be broken
○ TODO: Scheduling of refresh
Providers (M. Draper)
Provider Plugins Update
(A Matthew Draper Production)
Providers (G. Blomquist)
● What’s next?
○ Openstack Infrastructure
■ Event catcher
■ Connecting the infrastructure to the cloud
■ Autoscaling!
○ Kubernetes
■ Expanding to OpenShift!
Rest API (A. Bellotti)
Merged PRs:
VM Management:
➢ “add_event” action #1260
➢ “scan” action #1262
Policy Management:
➢ Supporting policies and policy_profiles as subcollections #1580
➢ Querying Policy actions via /api/policy/:id/policy_actions as well as
primary collection /api/policy_actions #1608
➢ Querying Policy events via /api/policy/:id/events #1611
➢ “assign” and “unassign” policy and policy_profiles to vms, templates,
providers, hosts, resource_pools and clusters resources #1644
➢ policies and policy_profiles “resolve” for vms/templates #1255
With these merges, we have parity with the SOAP API !!
Rest API (A. Bellotti)
Merged PRs:
Foundational:
➢ Token manager enhancements #1801
Authentication tokens are now worker independent.
➢ Query performance #1881
GET 500 vms w/ name 32 seconds -> 1.4 seconds
GET 1000 vms w/ name (> 2 minutes) -> 2.4 seconds
➢ Security fix, sqlfilter parameter replaced with filter[] #1866
Additional primary collections:
➢ /api/flavors
➢ /api/security_groups
➢ /api/availability_zones #2051
UI Enhancements
● HTML 5 based console support for RHEV
and VMware VMs
● Initial UI Plugin support
● Orchestration UI updates
● HAML and I18n strings 100% completed in
views
UI Demos
HTML 5 Console Support video
Initial UI Plugin Support video
Orchestration (G. McCullough)
● Added edit/copy/remove for Orchestration
Templates
○ Backend added template validation.
● Enabled CloudFormation Provisioning
through Services
○ Default Automate state machine and methods
provided in /ManageIQ/Cloud/Orchestration/Provisioning
○ Note: Creating Service Dialogs from a template is
not supported in the UI in this build.
● Demo
Retirement Enhancement
(G. McCullough)
PR #1971 - Goal: To provide a solid out of the box retirement state machine
that can be easily modified and customized. (See PR description for more
detailed breakdown of changes)
Old logic:
● Relied heavily on internal processing
● Object immediately marked as retired
● VMs required policy assignment to access Automate.
New logic:
● All steps exposed in Automate state machine and fully customizable.
● No policy assignment.
Important changes:
● New events: request_vm_retire and request_service_retire
● New attribute “retirement_state” allows for better tracking and error
detection.
Retirement Enhancement
(G. McCullough)
Old VM retirement workflow:
1. Set retired flag
2. Raise vm_retired event
3. Raise audit event
4. Power off VM
5. Calls into Automate (Requires policy assignment)
New VM retirement workflow:
1. Raise request_vm_retire event - Calls into Automate (With no policy
assignment)
2. Start_Retirement - set retirement_state to 'retiring’
3. PreRetirement (different depending on vendor type)
4. Perform custom actions (example: Release MAC and/or IP address)
5. Remove from provider
6. Finish_retirement
a. Set retirement_state to ‘retired’
b. Raise audit event
Service Dialogs (G. McCullough)
● Added Dynamic Date Control
● Git Issue #1970: Support read-only setting for
all fields
Cloud Objects (G. McCullough)
Additional cloud objects available to automate:
● CloudObjectStoreContainer
● CloudObjectStoreObject
● CloudVolume
○ CloudVolumeAmazon
○ CloudVolumeOpenstack
○ Disk (return by attachments relationship on CloudVolume)
● VolumeSnapshot
○ CloudVolumeSnapshotAmazon
○ CloudVolumeSnapshotOpenstack
Rails 4 / Tech Debt (A. Patterson)
● Removing tech debt
○ Drop usage of ActsAsArModel
○ Refactoring views to remove dup code
○ Speed up MiqAeObject imports
● Isolating the App from Rails
● Eliminate eager loading assumptions
IPv6 (J. Rafaniello)
● Hosts and Providers will now default to use
the hostname column for communication,
not ipaddress. UI changes are still to be
done.
● Merged support for IPv6 addresses with
Rhevm/ovirt (requires ip6tables 443 open)
● Rhevm/ovirt discovery fixed to eliminate
false positives
STIG (J. Rafaniello)
● Rails server and memcached now listen on
localhost, not 0.0.0.0 in production
● Stop/start apache when ui/ws server roles
are stopped/started
● Enable SAM Account Name user type (A.
Bellotti)
Fleecing (R. Oliveri)
● Systemd Fleecing Support
○ Manages “init” processes on newer Linux
distributions.
○ Data now collected and displayed in UI.
○ UI changes still required to display additional
systemd-specific information.
○ DEMO (Mo Morsi)
Community Update (JMW)
● Tentative dates for next design summit
(online only):
○ May 7 & 8
● Development on next version of Extensions
Depot to begin “Real Soon Now”
● Automate methods document pull request:
○ https://github.com/ManageIQ/manageiq_docs/pull/1
○ To be merged into web site docs after review
● Track development on the Trello board:
○ https://trello.com/b/sogANZVl/manageiq-org-commu
nity
Questions?
Upcoming Sprint Report Schedule:
● 4/1/15
● 4/22/15
● 5/13/15
● 6/3/15
● 6/24/15

Sprint 20

  • 1.
    ManageIQ Sprint 20 Report- Sprint End March 9, 2015 March 11, 2015
  • 2.
    Overview ● Sprint Statistics ●Providers (G. Blomquist, J. Frey) ● Rest API (A. Bellotti) ● UI Enhancements (D. Clarizio) ● Orchestration, Retirement, Automate(G. McCullough) ● Rails 4, Technical Debt (A. Patterson) ● IPv6, STIG (J. Rafaniello) ● Fleecing (Rich O.) ● Community Update (JMW)
  • 3.
    Sprint Statistics ● https://github.com/ManageIQ/manageiq/issues?q=milestone%3A%22Sprin t+20+Ending+Mar+9%2C+2015%22+is%3Amerged ●249 Pull Requests Merged!!! ○ 112 issues labeled as “bug” ○ 76 issues labeled as “enhancement” ○ 11 issues labeled as “technical debt” ○ 35 issues labeled as “refactoring” ○ 9 issues labeled as “test” ● 84 Open PRs ○ 27 issues labeled as “WIP”
  • 4.
    Providers (G. Blomquist) ●OpenStack Infrastructure ○ Heat stacks ○ Images ○ More metrics ● Kubernetes (Containers) ○ Event catcher
  • 5.
    Providers (J. Frey) ●Foreman integration ○ Reorganized backend tables of managers by consolidating into ext_management_systems ○ Verification of credentials ○ Workers for EMS Refresh ○ Provisioning ■ Internal state-machine logic merged ■ Workflow refactoring ■ Researched Organization/Location support ○ Reporting may be broken ○ TODO: Scheduling of refresh
  • 6.
    Providers (M. Draper) ProviderPlugins Update (A Matthew Draper Production)
  • 7.
    Providers (G. Blomquist) ●What’s next? ○ Openstack Infrastructure ■ Event catcher ■ Connecting the infrastructure to the cloud ■ Autoscaling! ○ Kubernetes ■ Expanding to OpenShift!
  • 8.
    Rest API (A.Bellotti) Merged PRs: VM Management: ➢ “add_event” action #1260 ➢ “scan” action #1262 Policy Management: ➢ Supporting policies and policy_profiles as subcollections #1580 ➢ Querying Policy actions via /api/policy/:id/policy_actions as well as primary collection /api/policy_actions #1608 ➢ Querying Policy events via /api/policy/:id/events #1611 ➢ “assign” and “unassign” policy and policy_profiles to vms, templates, providers, hosts, resource_pools and clusters resources #1644 ➢ policies and policy_profiles “resolve” for vms/templates #1255 With these merges, we have parity with the SOAP API !!
  • 9.
    Rest API (A.Bellotti) Merged PRs: Foundational: ➢ Token manager enhancements #1801 Authentication tokens are now worker independent. ➢ Query performance #1881 GET 500 vms w/ name 32 seconds -> 1.4 seconds GET 1000 vms w/ name (> 2 minutes) -> 2.4 seconds ➢ Security fix, sqlfilter parameter replaced with filter[] #1866 Additional primary collections: ➢ /api/flavors ➢ /api/security_groups ➢ /api/availability_zones #2051
  • 10.
    UI Enhancements ● HTML5 based console support for RHEV and VMware VMs ● Initial UI Plugin support ● Orchestration UI updates ● HAML and I18n strings 100% completed in views
  • 11.
    UI Demos HTML 5Console Support video Initial UI Plugin Support video
  • 12.
    Orchestration (G. McCullough) ●Added edit/copy/remove for Orchestration Templates ○ Backend added template validation. ● Enabled CloudFormation Provisioning through Services ○ Default Automate state machine and methods provided in /ManageIQ/Cloud/Orchestration/Provisioning ○ Note: Creating Service Dialogs from a template is not supported in the UI in this build. ● Demo
  • 13.
    Retirement Enhancement (G. McCullough) PR#1971 - Goal: To provide a solid out of the box retirement state machine that can be easily modified and customized. (See PR description for more detailed breakdown of changes) Old logic: ● Relied heavily on internal processing ● Object immediately marked as retired ● VMs required policy assignment to access Automate. New logic: ● All steps exposed in Automate state machine and fully customizable. ● No policy assignment. Important changes: ● New events: request_vm_retire and request_service_retire ● New attribute “retirement_state” allows for better tracking and error detection.
  • 14.
    Retirement Enhancement (G. McCullough) OldVM retirement workflow: 1. Set retired flag 2. Raise vm_retired event 3. Raise audit event 4. Power off VM 5. Calls into Automate (Requires policy assignment) New VM retirement workflow: 1. Raise request_vm_retire event - Calls into Automate (With no policy assignment) 2. Start_Retirement - set retirement_state to 'retiring’ 3. PreRetirement (different depending on vendor type) 4. Perform custom actions (example: Release MAC and/or IP address) 5. Remove from provider 6. Finish_retirement a. Set retirement_state to ‘retired’ b. Raise audit event
  • 15.
    Service Dialogs (G.McCullough) ● Added Dynamic Date Control ● Git Issue #1970: Support read-only setting for all fields
  • 16.
    Cloud Objects (G.McCullough) Additional cloud objects available to automate: ● CloudObjectStoreContainer ● CloudObjectStoreObject ● CloudVolume ○ CloudVolumeAmazon ○ CloudVolumeOpenstack ○ Disk (return by attachments relationship on CloudVolume) ● VolumeSnapshot ○ CloudVolumeSnapshotAmazon ○ CloudVolumeSnapshotOpenstack
  • 17.
    Rails 4 /Tech Debt (A. Patterson) ● Removing tech debt ○ Drop usage of ActsAsArModel ○ Refactoring views to remove dup code ○ Speed up MiqAeObject imports ● Isolating the App from Rails ● Eliminate eager loading assumptions
  • 18.
    IPv6 (J. Rafaniello) ●Hosts and Providers will now default to use the hostname column for communication, not ipaddress. UI changes are still to be done. ● Merged support for IPv6 addresses with Rhevm/ovirt (requires ip6tables 443 open) ● Rhevm/ovirt discovery fixed to eliminate false positives
  • 19.
    STIG (J. Rafaniello) ●Rails server and memcached now listen on localhost, not 0.0.0.0 in production ● Stop/start apache when ui/ws server roles are stopped/started ● Enable SAM Account Name user type (A. Bellotti)
  • 20.
    Fleecing (R. Oliveri) ●Systemd Fleecing Support ○ Manages “init” processes on newer Linux distributions. ○ Data now collected and displayed in UI. ○ UI changes still required to display additional systemd-specific information. ○ DEMO (Mo Morsi)
  • 21.
    Community Update (JMW) ●Tentative dates for next design summit (online only): ○ May 7 & 8 ● Development on next version of Extensions Depot to begin “Real Soon Now” ● Automate methods document pull request: ○ https://github.com/ManageIQ/manageiq_docs/pull/1 ○ To be merged into web site docs after review ● Track development on the Trello board: ○ https://trello.com/b/sogANZVl/manageiq-org-commu nity
  • 22.
    Questions? Upcoming Sprint ReportSchedule: ● 4/1/15 ● 4/22/15 ● 5/13/15 ● 6/3/15 ● 6/24/15