My keynote at the SciPy 2018 conference, on SciPy 1.0 and community building. Video: https://www.youtube.com/watch?v=oHmm3mPxg6Y&t=758s

1. SciPy 1.0 and beyond
A story of community and code

2. –Travis Oliphant
SciPy was a distribution masquerading as a library

3. SciPy — fundamental tools for scientific computing

4. numerical algorithms
data analysis
math building blocks
data structures
utils
Linear algebra
(dense & sparse)
Fourier transforms
Special functions
Av = λv
|A| =
∑
j
(−1)i+j
aijMij
y[k] =
N−1
∑
n=0
e−2πj kn
N x[n]

5. Θ(t)
dt
= ω(t)
ω(t)
dt
= − bω(t) − c sin(Θ(t))
numerical algorithms
data analysis
math building blocks
data structures
utils
Optimization
Integration
Interpolation

6. numerical algorithms
data analysis
math building blocks
data structures
utils
Clustering
Graph algorithms
Computational geometry

7. numerical algorithms
data analysis
math building blocks
data structures
utils
Sparse matrices
k-D Tree

8. numerical algorithms
data analysis
math building blocks
data structures
utils
Statistics
Signal processing
Image processing

9. numerical algorithms
data analysis
math building blocks
data structures
utils
Scientific data formats
Physical constants
• NetCDF
• Matlab
• IDL
• Matrix Market
• Fortran
• Wav
• Harwell-Boeing
• ARFF
π
ℏ
∘
C −∘
F
parsec
angstrom
units
…

10. A short history
• 2001: the first SciPy release
• 2005: transition to NumPy
• 2007: creation of SciKits
• 2008: the Documentation Marathon
• 2010: moving to a 6-monthly release cycle
• 2011: development moves to GitHub
• 2011: Python 3 support
• 2013: continuous integration with TravisCI
• 2017: SciPy 1.0 release

11. Why 1.0 now?

12. setup.py
CLASSIFIERS = """
Development Status :: 5 - Production/Stable

13. We have a governance structure

14. We have a Code of Conduct

15. We have a roadmap

16. Windows wheels, at last

17. Community

18. 100 — 120 contributors per release
* M.J. Nichol
* Juan Nunez-Iglesias
* Arno Onken +
* Nick Papior +
* Dima Pasechnik +
* Ashwin Pathak +
* Oleksandr Pavlyk +
* Stefan Peterson
* Ilhan Polat
* Andrey Portnoy +
* Ravi Kumar Prasad +
* Aman Pratik
* Eric Quintero
* Vedant Rathore +
* Tyler Reddy
* Joscha Reimer
* Philipp Rentzsch +
* Antonio Horta Ribeiro
* Ned Richards +
* Kevin Rose +
* Benoit Rostykus +
* Matt Ruffalo +
* Eli Sadoff +
* Pim Schellart
* Nico Schlömer +
* Klaus Sembritzki +
* Nikolay Shebanov +
* Jonathan Tammo Siebert
* Scott Sievert
* Max Silbiger +
* Mandeep Singh +
* Michael Stewart +
* Jonathan Sutton +
* Deep Tavker +
* Martin Thoma
* James Tocknell +
* Aleksandar Trifunovic +
* Paul van Mulbregt +
* Jacob Vanderplas
* Aditya Vijaykumar
* Pauli Virtanen
* James Webber
* Warren Weckesser
* Eric Wieser +
* Josh Wilson
* Zhiqing Xiao +
* Evgeny Zhurko
* Nikolay Zinov +
* Zé Vinícius +
A total of 121 people contributed to this release.
People with a "+" by their names contributed a patch for the first time.

19. ~15 active maintainers
$ git shortlog --grep="Merge pull request" -sn v0.19.0..upstream/master
216 Ralf Gommers
209 Pauli Virtanen
75 Evgeni Burovski
39 Josh Wilson
19 Ilhan Polat
15 Andrew Nelson
14 Eric Larson
6 Jaime Fernandez del Rio
6 Nikolay Mayorov
6 Warren Weckesser
4 Denis Laxalde
4 Tyler Reddy
3 Matt Haberland
2 Antonio Horta Ribeiro
$ python summary_pr_reviews.py v0.19.0..upstream/master # >100 PR comments
@rgommers
@pv
@larsoner
@antonior92
@ilayn
@perimosocordiae
@nmayorov
@ev-br
@person142
@andyfaff
@WarrenWeckesser
@mdhaber
@tylerjereddy
@jaimefrio
@ghost
@endolith
@lagru
@mikofski
@matthew-brett
@chrisb83
@sgubianpm
@pvanmulbregt
@josef-pkt
@eric-wieser

20. –Nadia Eghbal
Maintainers are the keystone species of software
development

22. Matthew Rocklin
Wed May 23 19:08:30 EDT 2018
Hi All,
..
My gut reaction here is that if removing masked array allows Numpy to
evolve more quickly then this excites me.
What do “we” want?

23. What do “we” want?

26. … and code

27. User-friendly interfaces
New Zealand forest bio-security
sampling locations 2017
from scipy.interpolate import griddata
row_idx = griddata(weather[['X', 'Y']].values,
weather.index.values,
latlon, method='nearest')
stations = weather.iloc[row_idx, :]
Mapping to available
weather data

28. from scipy.integrate import solve_ivp
def exponential_decay(t, y):
return -0.5 * y
def solution(t, y0):
return y0 * np.exp(-0.5*t)
TMAX = 10
y0 = 8
t_eval = np.linspace(0, TMAX)
analytical_solution = solution(t_eval, y0)
for method in ('RK45', 'RK23', 'Radau', 'BDF', 'LSODA'):
sol = solve_ivp(exponential_decay, t_span=[0, TMAX], y0=[y0],
t_eval=t_eval, method=method)
ax.plot(sol.t, analytical_solution - sol.y[0, :], label=method)
User-friendly interfaces
New interface for solving initial value problems

29. Cython bindings
cimport scipy.special.cython_special as csc
cdef:
double x = 1
double rgam
rgam = csc.gamma(x)

30. Cython bindings
from numba import jit
a = np.random.randn(30, 20)
@jit(nopython=True)
def qr(a):
return np.linalg.qr(a)
>>> %timeit np.linalg.qr(a)
47.5 µs ± 2.53 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
>>> %timeit qr(a)
10.3 µs ± 174 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

31. LowLevelCallable
footprint = np.array([[0, 1, 0],
[1, 1, 1],
[0, 1, 0]], dtype=bool)
image = misc.ascent()
@llc.jit_filter_function
def fmin(values):
"""
Return minimum of values; values are pixel values in footprint pattern.
Accelerated with Numba, via `scipy.LowLevelCallable`.
"""
result = np.inf
for v in values:
if v < result:
result = v
return result
>>> %timeit ndimage.generic_filter(image, np.min, footprint=footprint)
427 ms ± 2.15 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)
>>> %timeit ndimage.generic_filter(image, fmin, footprint=footprint)
2.81 ms ± 40.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
Credits: @jni

32. What is next for SciPy?

33. Languages
SciPy is written in:
Python, Cython, C, C++, Fortran
If Python is not fast enough, we prefer Cython right now.
What about in the future?

34. Cython Numba Pythran
Portability ++ - -
Runtime dependency ++ — ++
Maturity ++ + -
Maintenance status + ++ -
Features ++ + -
Performance + ++ ++
Ease of use - ++ +
Debugging & optimization 0 + 0
Size of binaries - ++ +

35. To grow or not to grow?
Submodules removed since 2012:
Sub(sub)modules added since 2012:
scipy.maxentropy
scipy.lib
scipy.weave
scipy.misc # removal in process
scipy.sparse.csgraph
scipy.linalg.cython_blas/lapack
scipy.special.cython_special
scipy.linalg.interpolative

36. SciPy will grow in feature completeness and quality,
not in scope

37. In the pipeline
• Support for newer LAPACK versions
• Sparse arrays (talk Hameer Abbasi today)
• Better global optimizers
• Rotation matrices (GSoC’18)

38. Beyond volunteer-only?

39. 1000+ open issues

42. @FormerPhysicist @NKrvavica @Tokixix @awakenting @axiru @cel4 @chemelnucfin @endolith @gaulinmp @hugovk @ksemb @kshitij12345 @luzpaz @mamrehn @rafalalgo
@samyak0210 @soluwalana @sudheerachary @tttthomasssss @vkk800 @wirew0rm @xoviat @yanxun827 @ybeltukov @ziejcow Aakash Jain Aaron Nielsen Abject Abraham
Escalante Adam Cox Adam Geitgey Adam Kurkiewicz Aditya Bharti Aditya Vijaykumar Adrian Kretz Akash Goel Alain Leufroy Alan McIntyre Aldrian Obaja Aleksandar
Trifunovic Ales Erjavec Alessandro Pietro Bardelli Alex Conley Alex Griffing Alex Loew Alex Reinhart Alex Rothberg Alex Seewald Alex Stewart Alexander
Eberspächer Alexander Goncearenco Alexander Grigorievskiy Alexander J. Dunlap Alexey Umnov Alexis Tabary Alistair Muldal Alvaro Sanchez-Gonzalez Aman Pratik Aman
Singh Aman Thakral Amato Kasahara Anant Prakash Anastasiia Tsyplia Anders Bech Borchersen Andras Deak Andreas Hilboll Andreas Kloeckner Andreas Kopecky Andreas
Mayer Andreas Sorge Andreea Georgescu Andrew Fowlie Andrew Nelson Andrew Schein Andrew Sczesnak Andrey Golovizin Andrey Portnoy Andrey Smirnov Andriy Gelman
André Gaul Ankit Agrawal Anne Archibald Anthony Scopatz Anton Akhmerov Antonio H Ribeiro Antonio Horta Ribeiro Antony Lee Anubhav Patel Ariel Rokem Arno Onken
Ashwin Pathak Astrofysicus Axl West Baptiste Fontaine Bastian Venthur Behzad Nouri Ben FrantzDale Ben Jude Bence Bagi Benda Xu Benjamin Rose Benjamin
Trendelkamp-Schroer Benny Malengier Benoit Rostykus Bernardo Sulzbach Bhavika Tekwani Bill Sacks Björn Dahlgren Bjørn Forsman Blair Azzopardi Blake Griffith Bob
Helmbold Bradley M. Froehle Bram Vandekerckhove Branden Rolston Brandon Beacher Brandon Carter Brandon Liu Brett M. Morris Brett R. Murphy Brian Hawthorne Brian
Newsom Brianna Laugher Brigitta Sipocz Bruno Beltran Bruno Jiménez Byron Smith CJ Carey Callum Jacob Hays Carl Sandrock Cathy Douglass Chad Baker Chad Fulton
Charles Harris Charles Masson Chris Burns Chris Jordan-Squire Chris Kerr Chris Mutel Christian Christian Brodbeck Christian Brueffer Christian Häggström
Christian Sachs Christoph Baumgarten Christoph Deil Christoph Gohlke Christoph Paulik Christoph Weidemann Christopher Kuster Christopher Lee Cimarron Mittelsteadt
Ciro Duran Santillli Clancy Rowley Clark Fitzgerald Clemens Novak Cody Collin RM Stocks Collin Tokheim Colm Ryan Corey Farwell Daan Wynen Daisuke Oyama Damian
Eads Damon McDougall Daniel B. Smith Daniel Bunting Daniel Jensen Daniel Velkov Daniel da Silva Danilo Horta Danny Hermes David Cournapeau David Ellis David
Freese David Haberthür David Hagen David Huard David M Cooke David Menéndez Hurtado David Nicholson David Warde-Farley David Wolever Deep Tavker Deepak Kumar
Gouda Denis Laxalde Derek Homeier Dezmond Goff Dieter Werthmüller Dima Pasechnik Diogo Aguiam Dirk Gorissen Divakar Roy Dmitrey Kroshko Dominic Antonacci
Dominic Else Dorota Jarecka Douglas Lessa Graciosa Dražen Lučanin Dávid Bodnár Ed Schofield Edward Richards Egor Panfilov Eli Sadoff Elliott Sales de Andrade
Emanuele Olivetti Eric Firing Eric Jones Eric Larson Eric Martin Eric Moore Eric Quintero Eric Soroos Eric Stansifer Eric Wieser Eugene Krokhalev Evan Limanto
Evgeni Burovski Evgeny Zhurko FI$H 2000 Fabian Paul Fabian Pedregosa Fabian Rost Fabrice Silva Fazlul Shahriar Felix Berkenkamp Felix Lenders Fernando Perez
Florian Weimer Florian Wilhelm Francis T. O'Donovan Francisco de la Peña Franziska Horn François Boulogne François Garillot François Magimel Frederik Rietdijk
Fredrik Wallner Fukumu Tsutsumi G Young Gabriele Farina Gael Varoquaux Garrett Reynolds Gaute Hope Gavin Parnaby Gavin Price Gaël Varoquaux Geordie McBain
George Castillo George Lewis Gerrit Ansmann Gerrit Holl Gert-Ludwig Ingold Giftlin Rajaiah Gilles Aouizerate Gilles Rochefort Giorgio Patrini Golnaz Irannejad
Graham Clenaghan Greg Caporaso Greg Dooper Gregory Allen Gregory R. Lee Grey Christoforo Guillaume Horel Guo Fei Gustav Larsson Haochen Wu Helder Cesar Helmut
Toplitzer Henrik Bengtsson Henry Lin Hervé Audren Hiroki IKEDA Ian Henriksen Ien Cheng Ilan Schnell Ilhan Polat Illia Polosukhin InSuk Joung Ion Elberdin Irvin
Probst J.J. Green J.L. Lanfranchi Jaakko Luttinen Jackie Leng Jacob Carey Jacob Silterra Jacob Stevenson Jacob Vanderplas Jacques Gaudin Jacques Kvam Jaime
Fernandez del Rio Jakob Jakobson Jakub Wilk James Gerity James T. Webber James Tocknell James Tomlinson James Webber Jamie Morton Jan Lehky Jan Schlueter Jan
Schlüter Janani Padmanabhan Janko Slavič Jarrod Millman Jason King Jean Helie Jean-François B Jean-François B. Jed Frey Jeff Armstrong Jerome Kieffer Jerry Li
Jesse Engel Jim Garrison Jim Radford Jin-Guo Liu Jinhyok Heo Joaquin Derrac Rus Jochen Garcke Joel Nothman Johann Cohen-Tanugi Johannes Ballé Johannes Kulick
Johannes Schmitz Johannes Schönberger John David Reaver John Draper John Travers Johnnie Gray Jon Haitz Legarreta Gorroño Jona Sassenhagen Jonas Hahnfeld Jonas
Rauber Jonathan Helmus Jonathan Hunt Jonathan Sutton Jonathan Tammo Siebert Jonathan Taylor Joonas Paalasmaa Jordan Heemskerk Jordi Montes Jorge Cañardo Alastuey
Joris Vankerschaver Joscha Reimer Josef Perktold Joseph Albert Joseph Fox-Rabinovitz Joseph Jon Booker Josh Lawrence Josh Lefler Josh Levy-Kramer Josh Wilson
Joshua L. Adelman Josue Melka Juan Luis Cano Rodríguez Juan M. Bello-Rivas Juan Nunez-Iglesias Juha Remes Julian Lukwata Julian Taylor Julien Lhermitte Julius
Bier Kirkegaard Jussi Leinonen Justin Lavoie Jyotirmoy Bhattacharya Jérôme Roy Jörg Dietrich Jörn Hees K.-Michael Aye Kai Kai Striega Kai-Striega KangWon Lee
Kari Schoonbee Kasper Primdal Lauritzen Kat Huang Katrin Leinweber Keith Clawson Kevin Davies Kevin Rose Klaus Sembritzki Klesk Chonkin Kolja Glogowski Konrad0
Kornel Kielczewski Kyle Oman Lam Yuen Hei Lars Buitinck Lars G Lawrence Chan Lei Ma Leo Singer Liam Damewood Lilian Besson Liming Wang Lindsey Hiltner Lorenzo
Luengo Louis Thibault Louis Tiao Loïc Estève Luca Citi Luis Pedro Coelho Luke Zoltan Kelley M.J. Nichol MYheavyGo Mads Jensen Mandeep Singh Maniteja Nandana
Manuel Reinhardt Marc Abramowitz Marc Honnorat Marcello Seri Marcos Duarte Marek Jacob Maria Knorps Mark Campanelli Mark Mikofski Mark Wiebe Markus Meister
Marti Nito Martin Manns Martin Spacek Martin Teichmann Martin Thoma Martin Ø. Christensen Martino Sorbaro Martín Gaitán Marvin Kastner Matt Dzugan Matt Haberland
Matt Hickford Matt Knox Matt Newville Matt Ruffalo Matt Terry Matteo Visconti Matthew Brett Matthias Feurer Matthias Kümmerer Matthieu Dartiailh Matthieu Melot
Matty G Matěj Kocián Max Argus Max Bolingbroke Max Linke Max Mikhaylov Max Silbiger Maximilian Singh Meet Udeshi Mher Kazandjian Michael Benfield Michael Boyle
Michael Danilov Michael Hirsch Michael James Bedford Michael Stewart Michael Wimmer Miguel de Val-Borro Mihai Capotă Mike Romberg Mike Toews Mikhail Pak Mikkel
Kristensen MinRK Naoto Mizuno Nat Wilson Nathan Bell Nathan Crock Nathan Musoke Nathan Woods Ned Richards Neil Girdhar Nelson Liu Nick Papior Nicky van Foreest
Nico Schlömer Nicola Montecchio Nicolas Del Piano Niklas Hambüchen Niklas K Nikolai Nowaczyk Nikolas Moya Nikolay Mayorov Nikolay Shebanov Nikolay Zinov Nils
Werner Noel Kippers Oleksandr (Sasha) Huziy Oleksandr Pavlyk Olga Botvinnik Olivier Grisel Orestis Floros Osvaldo Martin P. L. Lim Pablo Winant Patrick Callier
Patrick Snape Patrick Varilly Paul Ivanov Paul Nation Paul Ortyl Paul van Mulbregt Pauli Virtanen Pawel Chojnacki Peadar Coyle Pearu Peterson Pedro López-Adeva
Fernández-Layos Per Brodtkorb Perry Lee Pete Bunch Peter Cock Peter Yin Petr Baudis Phil Tooley Philip DeBoer Philipp Rentzsch Phillip Cloud Phillip J. Wolfram
Phillip Weinberg Pierre GM Pierre de Buyl Pim Schellart Piotr Uchwat Przemek Porebski Raden Muhammad Radoslaw Guzinski Rafael Rossi Rafał Byczek Ralf Gommers
Ramon Viñas Randy Heydon Raoul Bourquin Raphael Wettinger Ravi Kumar Prasad Ray Bell Regina Ongowarsito Richard Gowers Richard Tsai Rob Falck Robert Cimrman
Robert David Grant Robert Gantner Robert Kern Robert McGibbon Robert Pollak Rohit Jamuar Rohit Sivaprasad Roman Feldbauer Roman Mirochnik Roman Ring Ronan Lamy
Ronny Pfannschmidt Rupak Das Rémy Léone Sam Lewis Sam Mason Sam Tygier Sami Salonen Samuel St-Jean Santi Villalba Saurabh Agarwal Scott Sievert Scott Sinclair
Sean Gillies Sean Quinn Sebastian Berg Sebastian Gassner Sebastian Pucilowski Sebastian Pölsterl Sebastian Skoupý Sebastian Werk Sebastiano Vigna Sebastián
Vanrell Sergey B Kirpichev Sergio Oller Shauna Shinya SUZUKI Siddhartha Gandhi Simon Gibbons Skipper Seabold Sourav Singh Srikiran Stefan Otte Stefan Peterson
Stefan van der Walt Stefano Costa Stefano Martina Stephan Hoyer Stephen McQuay Steve Richardson Steven Byrnes Strahinja Lukić Sturla Molden Sumit Binnani Surhud
More Svend Vanderveken Sylvain Bellemare Syrtis Major Sytse Knypstra Søren Fuglede Jørgensen Takuya Oshima Tavi Nathanson Ted Pudlik Ted Ying Terry Jones Tetsuo
Koyama Theodore Hu Thomas A Caswell Thomas Etherington Thomas Haslwanter Thomas Hisch Thomas Keck Thomas Kluyver Thomas Pingel Thomas Robitaille Thomas Spura
Thouis (Ray) Jones Tiago M.D. Pereira Tim Cera Tim Hochberg Tim Leslie Tiziano Zito Tobias Megies Tobias Schmidt Todd Goodall Todd Jennings Tom Aldcroft Tom
Augspurger Tom Donoghue Tom Flannaghan Tom Waite Tomas Tomecek Tony S. Yu Tony Xiang Toshiki Kataoka Travis Oliphant Travis Vaught Trent Hauck Tyler Reddy Uri
Goren Utkarsh Upadhyay Uwe Schmitt Vahan Babayan Valentine Svensson Vasily Kokorev Ved Basu Vedant Rathore Vicky Close Vikram Natarajan Vincent Arel-Bundock
Vincent Barrielle Vladislav Iakovlev Víctor Zabalza Warren Weckesser Wendy Liu Wes McKinney Will Monroe Wim Glenn Yaroslav Halchenko Yevhenii Hyzyla Yoav Ram
Yoni Teitelbaum Yoshiki Vázquez Baeza Yotam Doron Yu Feng Yurii Shevchuk Yusuke Watanabe Yuxiang Wang Yves Delley Yves-Rémi Van Eycke Zach Ploskey Zaz Brown Ze
Vinicius Zhiqing Xiao Zé Vinícius abaecker annesylvie arcady ashish bsdz chanley dmorrill drlvk emmi474 fcady fo40225 fred.mailhot fullung hagberg hm
jfinkels jmiller joe jswhit jtaylor jtravs keuj6 ktritz lemonlaug martin michaelvmartin15 mohmmadd mszep nmarais nrnrk ondrej pjwerneck poolio prabhu
puenka roberto solarjoe swalton thorstenkranz timbalam tosh1ki trueprice tzito Åsmund Hjulstad
A big thank you to
• The SciPy core team
• All SciPy contributors
• Our users
• The wider scientific Python community