Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

OPNFV Doctor - OpenStack最新情報セミナー 2017年7月

1,351 views

Published on

タイトル:OPNFV Doctor
講師:壬生 亮太(NEC)
アジェンダ:
- OPNFV Doctorとは
- Key Requirements as VIM
- Doctor Architecture and Typical Scenario

Published in: Technology
  • Be the first to comment

  • Be the first to like this

OPNFV Doctor - OpenStack最新情報セミナー 2017年7月

  1. 1. 1 OPNFV Doctor 壬生 亮太 NEC
  2. 2. 2 スピーカー 壬生 亮太 (みぶ りょうた) SDN/NFVソリューション事業部 Doctor プロジェクトリード 他3プロジェクトのSubmitter(コア) OPNFV アンバサダー Aodh (Alarming) コア
  3. 3. 3 Doctorがやっていること 通信事業における ユースケース・要件 要件詳細化 共同開発 インテグレーション テスト NFVリファレンスプラットフォーム Code Tool OPNFVの1プロジェクトと して、特定のトピックを このサイクで回す
  4. 4. 4 Doctorが目指している世界 IaaSプロバイダー サービスプロバイダー エンドユーザ 「VMは壊れることがあります」 「あれ動いていない?」 「このサービス使えない」 「そのVM止まりました」 「なら、スタンバイに切替」
  5. 5. 5 Doctor Project Virtualized Infrastructure Applications VIM User and Administrator Virtualized Infrastructure Manager (VIM) = OpenStack Virtual Compute Virtual Storage Virtual Network Virtualization Layer Hardware Resources App App App Doctor Scope Feature project to build fault management and maintenance framework
  6. 6. 6 Assumption of VNF (NFV Application) • Telco Applications are generally deployed in active-standby or active-active fashion App (Active) App (Standby) VM VM Machine Machine App and App Manager (VNFM) cannot detect HW failures directly App state needs to be switched when failure occurs
  7. 7. 7 Doctor Achievements • Design – Requirement Document – To-be-architecture • Implementation – Open Source Project Mapping – Gap Analysis – work in Upstream (OpenStack) • Integration and testing – Functest / Installer – Doctor CI – Performance test support – Profiler Figured out basic use cases and minimal requirements in fault management Key features are available in OpenStack Good example of how to upstream OPNFV users can test fault management scenario quickly with performance profile
  8. 8. 8 Demo @ OpenStack Summit Barcelona vEPC Failover keeping phone call session online
  9. 9. 9 Key Requirements as VIM Immediate Notification Consistent Resource State Awareness Extensible Monitoring Fault Correlation
  10. 10. 10 Doctor Architecture and Typical Scenario Monitor Notifier Manager Virtualized Infrastructure (Resource Pool) Alarm Conf. 3. Update State 2. Find Affected Application Controller Controller Controller Resource Map 1. Raw Failure Inspector 4. Notify all 5. Notify Error 0. Set Alarm 6-. Action Failure Policy Monitor Monitor
  11. 11. 11 Doctor Architecture and Typical Scenario Monitor Notifier Manager Virtualized Infrastructure (Resource Pool) Alarm Conf. 3. Update State 2. Find Affected Application Controller Controller Controller Resource Map 1. Raw Failure Inspector 4. Notify all 5. Notify Error 0. Set Alarm 6-. Action Failure Policy Monitor Monitor Consistent Resource State Awareness Immediate Notification Extensible Monitoring Fault Correlation
  12. 12. 12 Doctor OSS Map Monitor Notifier Manager Virtualized Infrastructure (Resource Pool) Alarm Conf. 3. Update State 2. Find Affected Application Controller Controller Controller Resource Map 1. Raw Failure Inspector 4. Notify all 5. Notify Error 0. Set Alarm 6-. Action Failure Policy Monitor Monitor Ceilometer /Aodh Congress e.g. Zabbix Cinder Neutron Nova
  13. 13. 13 Analyzed Gaps and Development Items Monitor Notifier Manager Virtualized Infrastructure (Resource Pool) Alarm Conf. 3. Update State 2. Find Affected Application Controller Controller Controller Resource Map 1. Raw Failure Inspector 4. Notify all 5. Notify Error 0. Set Alarm 6-. Action Failure Policy Monitor Monitor Cinder Neutron Nova State Correction e.g. Zabbix Ceilometer /Aodh Congress Event Alarm Event-driven Inspection
  14. 14. 14 Doctor Blueprints in OpenStack Project Blueprint Spec Drafter Developer Status Ceilometer /Aodh Event Alarm Evaluator Ryota Mibu (NEC) Ryota Mibu (NEC) Completed (Liberty) Nova New nova API call to mark nova-compute down Tomi Juvonen (Nokia) Roman Dobosz (Intel) Completed (Liberty) Support forcing service down Tomi Juvonen (Nokia) Carlos Goncalves (NEC) Completed (Liberty) Get valid server state Tomi Juvonen (Nokia) Tomi Juvonen (Nokia) Completed (Mitaka) Add notification for service status change Balazs Gibizer (Ericsson) Balazs Gibizer (Ericsson) Completed (Mitaka) Congress Push Type DataSource Driver Masahito Muroi (NTT) Masahito Muroi (NTT) Completed (Mitaka) Adds Doctor Driver Masahito Muroi (NTT) Masahito Muroi (NTT) Completed (Newton) Neutron Port data plane status Carlos Goncalves (NEC) Carlos Goncalves (NEC) Completed (Pike)
  15. 15. 15 Related OpenStack Trends • Admin APIs for external monitor integration • Objectify / Versioned object • Versioned object notification • Polling  Event data source (Event data enhancements) • Event Pipeline enhancement
  16. 16. 16 Ceilometer/Aodh – Event alarm sample NEW Shortcut (notification-based) EXISTING (polling-based) Manager Audit Service stats notification event CinderNeutronNova Notification- driven alarm evaluator
  17. 17. 17 Nova – Force-Down & Exposing Service State Host / Machine Hypervisor VM nova compute nova api nova conductor nova scheduler nova DBqueue External Monitoring Service vSwitch BMC EXISTING (periodic update) Force-down API [Mark Nova-Compute Down] Notifying that the nova service is no longer available Client Service disable API Evacuation API Reset Server State API Allows Nova to integrate with External Monitoring Services, and make sure Nova handles requests for the host properly service state service.update notification
  18. 18. 18 Doctor Integration http://docs.opnfv.org/en/stable-danube/submodules/doctor/docs/release/release-notes/releasenotes.html#opnfv-installer-support-matrix
  19. 19. 19 Doctor CI Job Releng CI Control Pharos / Infra team putting infra in place Functest Testing Framework TestDB Artifact Apex/Fuel Integration Doctor Feature Dev. Jenkins InstallerTester TestCase 0. Patch uploaded 2. OPNFV Deploy (Not triggered now) 3. Launch Functest container 4. Trigger Feature Specific Testing Code 7. Store Logs Testing Team Test Coordination Gerrit 1. Trigger 6. Report result 5. Run test We are now using pre- deployed OPNFV PoD
  20. 20. 20 Testing tool enhancements • Multi-VM support  Performance test VM_BASENAME=doctor_vm VM_FLAVOR=m1.tiny VM_COUNT=${VM_COUNT:-1} • Performance Profiler • User/Project Option  RBAC check DOCTOR_USER=doctor DOCTOR_PROJECT=doctor DOCTOR_ROLE=_member_ • Rewriting in Python
  21. 21. 21 Profiler Total time cost: 472(ms) ==============================================================================> |Monitor|Inspector |Controller|Notifier|Evaluator | |112 |730 |? |? |? | | | | | | | | | | | link down:0 | | | | | | | | | raw failure:112 | | | | | | | | found affected:? | | | | | | | set VM error:312 | | | | | | marked host down:842 | | | | | notified VM error:? | | | | transformed event:? | | | evaluated event:? | | fired alarm:? | received alarm:472
  22. 22. 22 Doctor Status – Fault Management Notifier MonitorController Inspector Ceilometer/ Aodh Nova Congress Neutron Cinder Done To-Be Arch. Design Gap Analysis Blueprint Coding Integration OPNFV Release Vitrage Sample Sample
  23. 23. 23 Doctor Achievements • Design – Requirement Document – To-be-architecture • Implementation – Open Source Project Mapping – Gap Analysis – work in Upstream (OpenStack) • Integration and testing – Functest / Installer – Doctor CI – Performance test support – Profiler Figured out basic use cases and minimal requirements in fault management Key features are available in OpenStack Good example of how to upstream OPNFV users can test fault management scenario quickly with performance profile
  24. 24. 24 Adopting Doctor Framework to NFVI Maintenance • Tool chain for NFVI maintenance • Exchange VNFM and VIM admin intentions via Nova Server Tag and Notification Notifier Manager 3. Update State 2. Find Affected Controller 1. Inform Maintenance Inspector 4. Notify all 5. Notify Retirement 0. Set Alarm 6. Action Ceilometer /Aodh Congress Nova Admin Virtualized Infrastructure (Resource Pool) Application 9. Perform Maintenance 7. Allow migration 8. Check App Readiness VM migrate-okVM active inactive migrate-ng retirement

×