Solving Big problems with Condor - II HPC Sysadmins Meeting
•
0 likes•411 views
Solving Big problems with OS: Condor is a presentation about Condor, an open-source workload management system for computer clusters and grids. Condor provides features for heterogeneous computing, job monitoring, and fair sharing of resources. It allows users to run long-running or short jobs and supports technologies like Hadoop, MPI, and GPU computing. While flexible and powerful, Condor has some limitations like a lack of native MPI support or slot-wise preemption. However, it remains a free and popular system for managing distributed workloads.
More Related Content
Viewers also liked
Viewers also liked (19)
Similar to Solving Big problems with Condor - II HPC Sysadmins Meeting
Similar to Solving Big problems with Condor - II HPC Sysadmins Meeting (20)
More from Antonio Sanz Alcober
More from Antonio Sanz Alcober (20)
Recently uploaded
Recently uploaded (20)
Solving Big problems with Condor - II HPC Sysadmins Meeting
- 1. > Solving Big problems with OS: Condor > Antonio Sanz (ansanz@unizar.es) > 15 / Oct / 2012
- 2. 2 > Antonio Sanz > I3A System Manager > HERMES HPC cluster sysadmin > ansanz@unizar.es > @antoniosanzalc
- 3. 3
- 4. I’m no SGE guy … 4
- 5. 5 Condor – Main features
- 6. 6
- 8. Condor Basics 8 Heterogeneous computing
- 9. 9 Job Surveillance
- 10. 10 Requirements
- 11. Condor Basics 11 Fair use of resources 3. Sistemas de gestión de colas : Condor
- 12. Checkpoints 12
- 13. Condor Basics 13 Nested jobs (DAG)
- 14. 14 Easy Licensing
- 15. 15 … with Hadoop, MPI, OpenMP, GPU
- 18. 18 VM Universe
- 19. Hooks & APIs 19
- 20. Condor Basics 20 Flexibility
- 21. 21 How Condor works How Condor works
- 22. 22 Management [Hello, Dave]
- 23. 23 Compute * * * Hey!. I’m a 64K one!.
- 24. Condor Basics 24 Job list ClassAd 3. Sistemas de gestión de colas : Condor
- 25. 25 Resource list ClassAd
- 26. 26 Matchmaking
- 27. Condor Basics 27 Priority Management
- 28. 28 Data Transfer
- 29. Condor Basics 29 Job running 3. Sistemas de gestión de colas : Condor
- 30. 30 Job Monitoring
- 31. 31 Job End
- 32. 32 Example
- 33. 33 #!/bin/sh # I’m hola.sh echo Hola mundo desde `hostname` # # A Hello World .. In Condor! # # I’m hello.sub Universe = vanilla Executable = hola.sh Log = hola.log Output = hola.out Error = hola.err Queue Hello, World !!
- 34. Lanzar el cálculo 34 condor_submit 4. Condor Basics – Un cálculo fácil
- 35. Lanzar el cálculo 35 condor_q
- 36. 36 HERMES I3A HPC cluster
- 37. Condor Basics 37 1500 executing jobs, 40000 in queue … Lookin’ good
- 38. 38 Condor Tweaks
- 41. Condor Basics 41 Long Jobs
- 42. Condor Basics 42 Short Jobs
- 43. Condor Basics 43 Big Jobs
- 46. 46 Interactive Access Condor_ssh
- 48. 48 Future (always work in progress) Extra Bonus
- 49. 49 HA
- 50. Cgroups Isolation 50
- 51. Condor Basics 51 Hadoop Integration 3. Sistemas de gestión de colas : Condor
- 53. 53 Nobody’s perfect …. Condor Basics 3. Sistemas de gestión de colas : Condor
- 54. 54 No MPI + Dynamic Partitioning (yet) No MPI + Dynamic Partitioning Rellenado de trabajos HA Complicada No slot wise preemption HA tough as nails
- 55. Condor Basics 55 3. Sistemas de gestión de colas : Condor
- 56. Condor Basics 56 > Conclusiones … 3. Sistemas de gestión de colas : Condor
- 57. 57 Example
- 58. 58 Powerful. Flexible. Free. Fly like a bird with Condor Antonio Sanz Slides here: ansanz@unizar.es http://slideshare.net/ansanz @antoniosanzalc
- 59. 59 Extra Bonus
- 60. 60 I3A & Condor
- 61. 61 Alzheimer & Dementia Diagnose
- 63. 63 Rare Diseases
- 69. 69 Powerful. Flexible. Free. Fly like a bird with Condor Antonio Sanz Slides here: ansanz@unizar.es http://slideshare.net/ansanz @antoniosanzalc