CloudStack allows various life cycle operations for a Virtual Machine (VM). It maintains queues internally, to sync and perform all these operations. This talk briefs about how job queues are maintained in CloudStack, to execute the VM operations, followed by a demo. Suresh Anaparti is a software architect at ShapeBlue, the largest independent integrator of CloudStack technologies globally. He has over 15 years of end-to-end product development experience in Cloud Infrastructure, Telecom and Geospatial technologies. He is an active Apache CloudStack committer/contributor and is currently working with ShapeBlue. He has been working on CloudStack development for more than 5 years. ----------------------------------------- The CloudStack European User Group 2022 took place on 7th April. The day saw a virtual get together for the European CloudStack Community, hosting 265 attendees from 25 countries. The event hosted 10 sessions with from leading CloudStack experts, users and skilful engineers from the open-source world, which included: technical talks, user stories, new features and integrations presentations and more. ------------------------------------------ About CloudStack: https://cloudstack.apache.org/

1. VM Job Queues in CloudStack
- Suresh Kumar Anaparti

2. SPONSORED BY:

3. About Me!
@sureshanaparti
• Over 15 years of product development experience
• Active CloudStack Contributor since 2015
• Committer at Apache CloudStack
• Software Architect @ ShapeBlue

4. Agenda
• Introduction
• Implementation
• Debugging
• Future work
• Demo

5. Introduction
• Virtual Machine (VM)
oStates (Starting*, Running, Stopping*, Stopped, Destroyed, Migrating*, Expunging, Error)
oOperations
oTables
- vm_instance
• VM queues
oDatabase level
oTables
- sync_queue
- sync_queue_item

6. Implementation
• VM Job framework
o Sync Queue Manager
- enqueue, dequeue, purge, etc.
- sync queue table
- sync queue item table
o Async Job Manager
- Async job / VM job (sync_queue_item.content_id)
- VM job execution

7. Debugging
• Logs
omanagement sever log (/var/log/cloudstack/management/management-server.log)
2021-03-02 00:20:43,232 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-7ba29bcb)
(logid:6403b128) Queue (queue id, sync type, sync id) - (679334,VmWorkJobQueue, 80196) is reaching concurrency limit 1
• Database
oVM job tables (sync_queue, sync_queue_item)
sync_queue_item - pending jobs
- SELECT * FROM cloud.sync_queue_item
- SELECT * FROM cloud.sync_queue WHERE id IN (SELECT queue_id FROM cloud.sync_queue_item)
sync_queue.queue_size = 1
vm_instance – transitional states (Starting, Stopping, Migrating)

8. Future work
• Enhancements
oRe-prioritisation (of waiting jobs)
oAPI support (to cancel all waiting jobs, list pending jobs of a VM, etc)

9. DEMO

10. References
• VM sync improvements wiki
https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+VMSync+improvement
• Snapshot throttling wiki
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Snapshot+throttling+FS
• Source code (for Devs)
https://cloudstack.apache.org/downloads.html

11. ?

12. Thank You!