This document provides an overview of the yt astrophysics analysis and visualization toolkit. It discusses yt's goals of addressing physical rather than computational questions and getting out of the way of analysis. It also covers yt's community aspects, including the challenges of developing open source scientific software and strategies used by yt like reducing barriers to entry, open communication, and emphasizing a community of peers. Key points discussed are designing the community desired, challenges of academic rewards, and successes of yt like its development by working astrophysicists and usage on supercomputers.

1. Scaling a Code in the
Human Dimension
Matthew Turk
Columbia University

3. Barracuda

4. Barracuda
Battery

5. Barracuda
Cobra
Battery

6. Barracuda
Battery
Cobra
Quiver

7. Barracuda
Battery
Cobra
Quiver
Kangaroo

8. Barracuda
Battery
Cobra
Quiver
Kangaroo
Mob

9. Barracuda
Battery
Cobra
Quiver
Kangaroo
Mob
Ancedote

13. What is yt?

14. astro-ph/1011.3514
astro-ph/1112.4482
yt-project.org

17. There are many simulation codes.

18. There are many simulation codes,
but there is only one sky.

19. Fully-Supported
Semi-Supported
In-Progress
Enzo
FLASH
Orion
Nyx
Raw Data
Piernik
Chombo
Athena
ART
RAMSES
GDF
Cactus
Gadget
GAMER
PENCIL

20. yt is designed to address physical,
not computational,
entities and questions.

21. yt is supposed to get out of the way.

22. from yt.mods import *
pf = load("galaxy0030/galaxy0030")
p = SlicePlot(pf, 2, "Density", "c", (200, "kpc"))
p.show()

23. from yt.mods import *
pf = load("galaxy0030/galaxy0030")
p = SlicePlot(pf, 2, "Density", "c", (200, "kpc"))
p.show()
Load from disk, determine IO format, parse
parameters, set up mesh, initialize IO, and create
geometric objects.

24. from yt.mods import *
pf = load("galaxy0030/galaxy0030")
p = SlicePlot(pf, 2, "Density", "c", (200, "kpc"))
p.show()
Identify appropriate subregions of data, mask
out overlapping data, convert to CGS,
concatenate, pixelize, and return plot.

28. 10-12 g/cc

30. Low-Level Data Handling
Objects and Physical Quantities
Canned Analysis Tasks

31. (advanced)
Low-Level Data Handling
(intermediate) Objects and Physical Quantities
(beginner)
Canned Analysis Tasks

32. "Community"?

33. Traditional View of Scientific Development
"Users"
"Developers"

34. Most Scientific Development
"Users"
"Developers"

35. Community of Practice
"Devusers"

36. "Developers"
Inspection and verification
Tracking modifications
Sharing information
Doing new and interesting things

37. "Users"
Uncritical acceptance of code?

38. "Users"
"These are the people we give the code
to that don't care how it works."

40. Challenges

41. Academic Reward Structure

42. Academic Reward Structure
de facto & de jure

43. de facto & de jure
○
Utilization of developed tools
○
Respect from community
○
Project involvements
○
Invitations and opportunities to speak

44. de facto & de jure
○
Funding
○
Publications
○
Citation count
○
Influence

45. Traditional astrophysics does not
favor tool builders.

46. Chores
Documentation,
testing,
outreach,
infrastructure development.

47. Chores
Tasks not fully-aligned with reward
structure present great motivational
challenges.

48. Co-opetition
(
○
Funding
○
Publications
○
Citation count
○
Influence
)

49. The "citation economy" for community
codes is broken, and this
disproportionately impacts new and
junior contributors.

50. The "citation economy" for community
codes is broken, and this
disproportionately impacts new and
junior contributors.
(It's bad for us, but even worse
for infrastructure.)

51. How developer community
engagement, cohesion, excitement
and energy is affected by funded
improvements remains unclear.

52. Feelings of “ownership” are tricky in
academia, particularly for young
researchers.

53. Feelings of “ownership” are tricky in
academia, particularly for young
researchers.
How can progress, credit, and
intellectual priority be balanced?

54. How do the relationships between
distributed software developers
interact with academic relationships?

55. Strategies

56. Design the community you want.

57. Design the community you want.
Diversity. Tone. Enthusiasm. Congeniality.

58. Design the community you want.
This is an investment.

59. Design the community you want.
Diversity. Tone. Enthusiasm. Congeniality.

60. Growing diversity in scientific
software development is a key
challenge.

61. Technical
& Social

62. Technical
& Social

63. Reduce barrier to entry
Test on every push
Review every changeset

64. Reduce barrier to entry
Test on every push
Review every changeset
Everything comes in the box: version control, extensions,
sample data, dependencies, and tutorials.

65. CVCS
DVCS
Repo
Repository
Repo
Repo
Repo
Individuals
Repo
Repo

66. Reduce barrier to entry
Test on every push
Review every changeset
Shining Panda for unit tests & small data answer tests,
ReadTheDocs.org, and an auto-deployed ReST blog.

67. Reduce barrier to entry
Test on every push
Review every changeset
Pull requests and mentoring of new developers, through
IRC, mailing list, and code comments.

68. An upstream path...

69. Happy Application
Itch-Scratching
Pull Request Submission
Code Review & Mentoring
Participation

70. Happy Application
Itch-Scratching
Pull Request Submission
Code Review & Mentoring
Participation

71. Accept contributions of data,
scripts, images, projects

73. Communication
All project business is conducted openly.

74. Immediate

75. Immediate

76. Immediate

77. Low-Latency

78. High-Latency

79. Technical
& Social

80. Culture self-propagates.
So, it must be seeded directly.

81. The traditional "Open Source" community is
not always the right role model.

82. The traditional "Open Source" community is
not always the right role model.
Diversity. Tone. Enthusiasm. Congeniality.

83. Foster a community of peers,
not a community of elites.

84. H
R
T

85. H
Fitzpatrick & Collins-Sussman
R
T

86. Humility
R
T

87. Humility
Respect
T

88. Humility
Respect
Trust

89. Humility

90. I think there might be a bug in ...
It's like that for a good reason. Don't
touch it.

91. I think there might be a bug in ...
It behaves that way because ...

92. Respect

93. I've noticed something is acting
strangely with ...
You're probably doing it wrong.

94. I've noticed something is acting
strangely with ...
Can you tell us how you'd expect it to
act?

95. Trust

96. Letting go...

97. By emphasizing pride over
ownership, we've tried to help
projects move between people
without smothering through
control.

98. By emphasizing pride over
ownership, we've tried to help
projects move between people
without smothering through
control.
This is really difficult.

99. Successes

103. Developed by working astrophysicists.

104. !
Developed by working astrophysicists.

107. Usage on XSEDE
Nautilus
Szczepanski et al, 2012

110. Moving beyond computational astrophysics.

111. Moving beyond astrophysics.

112. Our first major litmus test, a 3.0
release with major infrastructure
overhauls, approaches.

113. "... it seems likely that significant software contributions to
existing scientific software projects are not likely to be
rewarded through the traditional reputation economy of
science. Together these factors provide a reason to expect
the over-production of independent scientific software
packages, and the underproduction of collaborative projects
in which later academics build on the work of earlier ones."
Howison & Herbsleb (2011)

114. First Workshop on Sustainable Software for Science:
Practices and Experiences
SC13: This Sunday!
wssspe.researchcomputing.org.uk

115. Thank you.

116. (Short) Bibliography
"What Makes Computational Open Source Libraries Successful?" by Bangerth & Heister
"The Art of Community" by Jono Bacon
"Producing Open Source Software" by Karl Fogel
"Team Geek" by Brian Fitzpatrick & Ben Collins-Sussman
"Organizing Simulation Code Collectives" by Mikaela Sundberg
"Scientific Software Production" by James Howison & James Herbsleb
"Your Community is your Best Feature" by Gina Trapani
"The Proof of the Pudding" by John Allsopp
"Standing Out in the Crowd" by Skud
"Why you should write buggy software" by Brian Granger

117. Tom Abel
Kenza Arraki
David Collins
Brian Crosby
Andrew Cunningham
Hilary Egan
Nathan Goldbaum
William Grey
Markus Haider
Cameron Hummels
Christian Karch
Steffen Klemer
Kacper Kowalik
Mike Kuhlen
Eve Lee
Sam Leitner
Yuan Li
Chris Malone
Josh Moloney
Chris Moody
Andrew Myers
Jill Naiman
Kaylea Nelson
Jeff Oishi
Jean-Claude Passy
Mark Richardson
Thomas Robitaille
Anna Rosen
Doug Rudd
Anthony Scopatz
Devin Silvia
Sam Skillman
Stephen Skory
Britton Smith
Geoffrey So
Casey Stark
Elizabeth Tasker
Stephanie Tonnesen
Sebastian Trujillo-Gomez
Matthew Turk
Rick Wagner
Andrew Wetzel
John Wise
John ZuHone