This is a long talk about the main features of Condor, and what tweaks we have added at I3A.

1. > Solving Big problems with OS: Condor
> Antonio Sanz (ansanz@unizar.es)
> 15 / Oct / 2012

2. 2
> Antonio Sanz
> I3A System Manager
> HERMES HPC cluster sysadmin
> ansanz@unizar.es
> @antoniosanzalc

3. 3

4. I’m no SGE guy … 4

5. 5
Condor – Main features

6. 6

7. 7
Healthy project

8. Condor Basics 8
Heterogeneous computing

9. 9
Job Surveillance

10. 10
Requirements

11. Condor Basics 11
Fair use of resources
3. Sistemas de gestión de colas : Condor

12. Checkpoints
12

13. Condor Basics 13
Nested jobs (DAG)

14. 14
Easy
Licensing

15. 15
… with Hadoop, MPI, OpenMP, GPU

16. 16
Condor
Flocking

17. 17
Grid & Cloud
Computing

18. 18
VM Universe

19. Hooks & APIs
19

20. Condor Basics 20
Flexibility

21. 21
How Condor works
How
Condor
works

22. 22
Management
[Hello, Dave]

23. 23
Compute
*
*
* Hey!. I’m a 64K one!.

24. Condor Basics 24
Job list  ClassAd
3. Sistemas de gestión de colas : Condor

25. 25
Resource list  ClassAd

26. 26
Matchmaking

27. Condor Basics 27
Priority Management

28. 28
Data
Transfer

29. Condor Basics 29
Job running
3. Sistemas de gestión de colas : Condor

30. 30
Job Monitoring

31. 31
Job End

32. 32
Example

33. 33
#!/bin/sh
# I’m hola.sh
echo Hola mundo desde `hostname`
#
# A Hello World .. In Condor!
#
# I’m hello.sub
Universe = vanilla
Executable = hola.sh
Log = hola.log
Output = hola.out
Error = hola.err
Queue
Hello, World !!

34. Lanzar el cálculo 34
condor_submit
4. Condor Basics – Un cálculo fácil

35. Lanzar el cálculo 35
condor_q

36. 36
HERMES
I3A HPC cluster

37. Condor Basics 37
1500 executing jobs, 40000
in queue … Lookin’ good

38. 38
Condor Tweaks

39. 39
Propietary
Resources

40. 40
Dynamic Partitioning

41. Condor Basics 41
Long Jobs

42. Condor Basics 42
Short Jobs

43. Condor Basics 43
Big Jobs

44. 44
Advanced Accounting

45. 45
Dynamic Checkpointing

46. 46
Interactive Access
Condor_ssh

47. GPU Integration
Condor Basics 47

48. 48
Future (always
work in progress)
Extra Bonus

49. 49
HA

50. Cgroups Isolation
50

51. Condor Basics 51
Hadoop Integration
3. Sistemas de gestión de colas : Condor

52. 52
Green Computing

53. 53
Nobody’s perfect ….
Condor Basics
3. Sistemas de gestión de colas : Condor

54. 54
 No MPI + Dynamic
Partitioning (yet)
No MPI + Dynamic Partitioning
Rellenado de trabajos
HA Complicada
 No slot wise preemption
 HA tough as nails

55. Condor Basics 55
3. Sistemas de gestión de colas : Condor

56. Condor Basics 56
> Conclusiones
…
3. Sistemas de gestión de colas : Condor

57. 57
Example

58. 58
Powerful. Flexible. Free.
Fly like a bird with Condor
Antonio Sanz
Slides here: ansanz@unizar.es
http://slideshare.net/ansanz @antoniosanzalc

59. 59
Extra Bonus

60. 60
I3A & Condor

61. 61
Alzheimer & Dementia Diagnose

62. 62
Tissue Modelling

63. 63
Rare
Diseases

64. 64
Crash test simulations

65. 65
Heart complete sim.

66. 66
Communication
Systems

67. 67
Dynamic gaming AI

68. 68
Autonomous robots

69. 69
Powerful. Flexible. Free.
Fly like a bird with Condor
Antonio Sanz
Slides here: ansanz@unizar.es
http://slideshare.net/ansanz @antoniosanzalc