SlideShare a Scribd company logo
1 of 25
Download to read offline
Inside NumPy: preparing for the next decade
Ralf Gommers, Sebastian Berg, Matti Picus,

Tyler Reddy, Stéfan van der Walt, Charles Harris
A very brief history of NumPy
2005: NumPy created
2006: NumPy 1.0
…….: 1-2 releases every year, gradual progress
2015: governance and NumFOCUS relationship formalized
2018: first-ever paid developers hired
The Sloan & Moore grants to BIDS for NumPy
• Two grants for in total $1.3M, Apr 2018 — Oct 2020
• Current co-PIs: Stéfan van der Walt and Fernando Perez
• Social aims: Improve community engagement

Grow core team, Diversify contributors
• Technical aims: More flexible & sustainable code,
Frequent & consistent releases, Improve data type
system, New array protocol
An organizational view of the project
Steering Council
The NumPy project
Contributors
Core developers
Repositories
Websites
Code Documentation
NumFOCUS

Fiscal sponsor
people
BIDS

people
Quansight

people
your org?

Institutional Partners
Key questions we’ll try to answer in this talk
• Has grant funding over the last year invigorated the
NumPy project?
• How healthy/sustainable is NumPy today?
• What does NumPy need in order to thrive?
• What’s our vision for NumPy, and what is our plan to
achieve that vision?
Impact of grant funding — an attempt to quantify
Heavier-tailed distributions in 2018-19: bus factor increased.
Fernando Perez, Ten years of (interactive) scientific computing
Impact of grant funding — an attempt to quantify
Heavier-tailed distributions in 2018-19: bus factor increased.
Fernando Perez, Ten years of (interactive) scientific computing
Impact of grant funding — an attempt to quantify
Volunteer contributions relatively stable; total activity up.
Impact of grant funding — an attempt to quantify
Issue/PR count rate of change is key indicator of project health.
Nadia Eghbal, Methodologies for measuring project health
Impact of grant funding — qualitative
• We now have a roadmap
• We’ve been able to organize core team sprints
• NumPy Enhancement Proposal process revived
• We paid down a decent amount of technical debt
• The 1.17.0 release (out any day now) is the largest
release in a long time (at least since 1.7.0 in 2013).
• Planning the Array Developer Summit for next March.
Has grant funding over the last year invigorated
the NumPy project?
Yes, we think so.
Faster progress and more hands for
maintenance also makes it more fun.
How healthy/sustainable is NumPy today?
A little more healthy than a year ago.
Still a worry though. Bus factor estimate: ~5
Hiring
So who did we hire?
Two full-time engineers, ongoing (till Oct 2020 at least):
• Mar 2018 — : Matti Picus (PyPy maintainer)
• Jun 2018 — Jun 2019: Tyler Reddy (SciPy maintainer)
• May 2019 — : Sebastian Berg (NumPy Steering Council member)
Additionally:
• Supporting Kriti Singh through Outreachy (docs work)
• Planning for a few more short-term contractors (code & docs)
Funding
Funding — what to pay for?
• Pay for things that are important and otherwise won't get
done.
• Think broadly. There's more to a project than code!
• Balance maintenance & innovation: keep people
motivated!
• Plan for sustainability. Don't start depending on funding
unless you’re fairly certain it’s stable.
• Make the life of volunteer maintainers easier, not harder!
Funding — who to pay?
• All else being equal, give preference to existing
maintainers.
• Pay attention to communication and self-management
skills, in addition to technical skills and motivation.
• Consider this an opportunity to make your project more
diverse.
Challenges
Sustainability — maintainer bandwidth
Of the 11 Steering Council members:
• 3 are very active (Chuck, Eric, Stephan)
• 3 are paid to work on NumPy

(Sebastian full-time,

Stéfan & Ralf a small part of their time (~1 day/wk))
• 5 are in low-activity mode (infrequent emails/commits)
This is still a major challenge! NumPy depends both on a
handful of people, and probably on continued funding.
The project beyond code
It’s still hard getting work done on:
• High-level documentation
• Website
• Community building
• Governance and project management
• Long-term planning (both technical and organizational)
Improving NumPy’s culture
• We still struggle with a lack of diversity: all maintainers are
white and male.
• Our GitHub and mailing list culture is occasionally not as
friendly and welcoming as we’d all like it to be.
Vision
Scope & Vision
The key thing NumPy offers is:
an array object (N-dimensional, in-memory, on CPUs)
and array computing APIs
NumPy lives at the heart of the numerical Python
ecosystem. We want to:
evolve while remaining a stable base,
address bottlenecks that limit how the

ecosystem can grow,
and grow and diversify our team and community.
What does NumPy need in order to thrive?
• Sustained funding at a higher level than today.

Our estimate: 10 full-time people
• More bandwidth from key people for long-term planning,
managing the project
• Attracting people in roles that primarily focus on activities
other than coding
Parts of a plan to get there
• In the next year, work with tech writers and web
developers to start building a docs/web team.

Efforts already ongoing.
• Build a diverse and robust funding quilt:

Express our needs as a community (not just NumPy),

and ask funders and major users for contributions:

Recently started (e.g. pitched to NASA), want to make this much more
concrete at SciPy’19.

More Related Content

Similar to Inside NumPy: preparing for the next decade

How kubernetes works community, velocity, and contribution - osls 2017 (1)
How kubernetes works  community, velocity, and contribution - osls 2017 (1)How kubernetes works  community, velocity, and contribution - osls 2017 (1)
How kubernetes works community, velocity, and contribution - osls 2017 (1)Brian Grant
 
Cryptoassets: A qualitative due diligence framework of DEX 0x
Cryptoassets: A qualitative due diligence framework of DEX 0xCryptoassets: A qualitative due diligence framework of DEX 0x
Cryptoassets: A qualitative due diligence framework of DEX 0xEtienne Brunet
 
CMA Digital Strategy Workshop
CMA Digital Strategy WorkshopCMA Digital Strategy Workshop
CMA Digital Strategy WorkshopW. Ryan Dodge
 
Developing a Digitisation Framework for your Library. 2003
Developing a Digitisation Framework for your Library. 2003Developing a Digitisation Framework for your Library. 2003
Developing a Digitisation Framework for your Library. 2003Rose Holley
 
Quipmate bplan qprize-2.0
Quipmate bplan qprize-2.0Quipmate bplan qprize-2.0
Quipmate bplan qprize-2.0Kunal Singh
 
Quipmate bplan qprize-2.0
Quipmate bplan qprize-2.0Quipmate bplan qprize-2.0
Quipmate bplan qprize-2.0Kunal Singh
 
Piwik Retrospective 2008
Piwik Retrospective 2008Piwik Retrospective 2008
Piwik Retrospective 2008Matthieu Aubry
 
Dafiti R&D, Semana Acadêmica do Centro de Tecnologia (SACT), UFSM 2019
Dafiti R&D, Semana Acadêmica do Centro de Tecnologia (SACT), UFSM 2019Dafiti R&D, Semana Acadêmica do Centro de Tecnologia (SACT), UFSM 2019
Dafiti R&D, Semana Acadêmica do Centro de Tecnologia (SACT), UFSM 2019Georg Buske
 
Open core summit 2020: Building an Open Source office on a shoestring budget
Open core summit 2020: Building an Open Source office on a shoestring budgetOpen core summit 2020: Building an Open Source office on a shoestring budget
Open core summit 2020: Building an Open Source office on a shoestring budgetAliza Carpio
 
The Agile Shape-up method for collaborative developments in international con...
The Agile Shape-up method for collaborative developments in international con...The Agile Shape-up method for collaborative developments in international con...
The Agile Shape-up method for collaborative developments in international con...Daniele Bailo
 
Netflix OSS Meetup Season 4 Episode 4
Netflix OSS Meetup Season 4 Episode 4Netflix OSS Meetup Season 4 Episode 4
Netflix OSS Meetup Season 4 Episode 4aspyker
 
So Now You’re a UiPath Developer – What’s Next? Who are the Stakeholders?
So Now You’re a UiPath Developer – What’s Next?  Who are the Stakeholders?So Now You’re a UiPath Developer – What’s Next?  Who are the Stakeholders?
So Now You’re a UiPath Developer – What’s Next? Who are the Stakeholders?DianaGray10
 
So Now You’re a UiPath Developer – What’s Next? Who are the Stakeholders?
So Now You’re a UiPath Developer – What’s Next?  Who are the Stakeholders?So Now You’re a UiPath Developer – What’s Next?  Who are the Stakeholders?
So Now You’re a UiPath Developer – What’s Next? Who are the Stakeholders?DianaGray10
 
AgileLIVE Webinar: Build a DevOps Culture & Infrastructure for Success Part 1
AgileLIVE Webinar: Build a DevOps Culture & Infrastructure for Success Part 1AgileLIVE Webinar: Build a DevOps Culture & Infrastructure for Success Part 1
AgileLIVE Webinar: Build a DevOps Culture & Infrastructure for Success Part 1VersionOne
 
Totara User Group - Data and Your LMS
Totara User Group - Data and Your LMSTotara User Group - Data and Your LMS
Totara User Group - Data and Your LMSKineo
 
Community Management
Community ManagementCommunity Management
Community ManagementLee Schlenker
 
February 23rd RTI Roundtable Presentation
February 23rd RTI Roundtable PresentationFebruary 23rd RTI Roundtable Presentation
February 23rd RTI Roundtable PresentationNathan Ryall
 
Tips for Nonprofits- Manage Content & Collaborate in the Cloud.
Tips for Nonprofits-  Manage Content & Collaborate in the Cloud.Tips for Nonprofits-  Manage Content & Collaborate in the Cloud.
Tips for Nonprofits- Manage Content & Collaborate in the Cloud.Box
 
Open Source: Beyond the Bottom Line - David Duffett
Open Source: Beyond the Bottom Line - David DuffettOpen Source: Beyond the Bottom Line - David Duffett
Open Source: Beyond the Bottom Line - David DuffettDavid Duffett dCAP
 

Similar to Inside NumPy: preparing for the next decade (20)

How kubernetes works community, velocity, and contribution - osls 2017 (1)
How kubernetes works  community, velocity, and contribution - osls 2017 (1)How kubernetes works  community, velocity, and contribution - osls 2017 (1)
How kubernetes works community, velocity, and contribution - osls 2017 (1)
 
Cryptoassets: A qualitative due diligence framework of DEX 0x
Cryptoassets: A qualitative due diligence framework of DEX 0xCryptoassets: A qualitative due diligence framework of DEX 0x
Cryptoassets: A qualitative due diligence framework of DEX 0x
 
CMA Digital Strategy Workshop
CMA Digital Strategy WorkshopCMA Digital Strategy Workshop
CMA Digital Strategy Workshop
 
Developing a Digitisation Framework for your Library. 2003
Developing a Digitisation Framework for your Library. 2003Developing a Digitisation Framework for your Library. 2003
Developing a Digitisation Framework for your Library. 2003
 
Quipmate bplan qprize-2.0
Quipmate bplan qprize-2.0Quipmate bplan qprize-2.0
Quipmate bplan qprize-2.0
 
Quipmate bplan qprize-2.0
Quipmate bplan qprize-2.0Quipmate bplan qprize-2.0
Quipmate bplan qprize-2.0
 
Piwik Retrospective 2008
Piwik Retrospective 2008Piwik Retrospective 2008
Piwik Retrospective 2008
 
Dafiti R&D, Semana Acadêmica do Centro de Tecnologia (SACT), UFSM 2019
Dafiti R&D, Semana Acadêmica do Centro de Tecnologia (SACT), UFSM 2019Dafiti R&D, Semana Acadêmica do Centro de Tecnologia (SACT), UFSM 2019
Dafiti R&D, Semana Acadêmica do Centro de Tecnologia (SACT), UFSM 2019
 
Open core summit 2020: Building an Open Source office on a shoestring budget
Open core summit 2020: Building an Open Source office on a shoestring budgetOpen core summit 2020: Building an Open Source office on a shoestring budget
Open core summit 2020: Building an Open Source office on a shoestring budget
 
Case Study- Migration/NetHope
Case Study- Migration/NetHopeCase Study- Migration/NetHope
Case Study- Migration/NetHope
 
The Agile Shape-up method for collaborative developments in international con...
The Agile Shape-up method for collaborative developments in international con...The Agile Shape-up method for collaborative developments in international con...
The Agile Shape-up method for collaborative developments in international con...
 
Netflix OSS Meetup Season 4 Episode 4
Netflix OSS Meetup Season 4 Episode 4Netflix OSS Meetup Season 4 Episode 4
Netflix OSS Meetup Season 4 Episode 4
 
So Now You’re a UiPath Developer – What’s Next? Who are the Stakeholders?
So Now You’re a UiPath Developer – What’s Next?  Who are the Stakeholders?So Now You’re a UiPath Developer – What’s Next?  Who are the Stakeholders?
So Now You’re a UiPath Developer – What’s Next? Who are the Stakeholders?
 
So Now You’re a UiPath Developer – What’s Next? Who are the Stakeholders?
So Now You’re a UiPath Developer – What’s Next?  Who are the Stakeholders?So Now You’re a UiPath Developer – What’s Next?  Who are the Stakeholders?
So Now You’re a UiPath Developer – What’s Next? Who are the Stakeholders?
 
AgileLIVE Webinar: Build a DevOps Culture & Infrastructure for Success Part 1
AgileLIVE Webinar: Build a DevOps Culture & Infrastructure for Success Part 1AgileLIVE Webinar: Build a DevOps Culture & Infrastructure for Success Part 1
AgileLIVE Webinar: Build a DevOps Culture & Infrastructure for Success Part 1
 
Totara User Group - Data and Your LMS
Totara User Group - Data and Your LMSTotara User Group - Data and Your LMS
Totara User Group - Data and Your LMS
 
Community Management
Community ManagementCommunity Management
Community Management
 
February 23rd RTI Roundtable Presentation
February 23rd RTI Roundtable PresentationFebruary 23rd RTI Roundtable Presentation
February 23rd RTI Roundtable Presentation
 
Tips for Nonprofits- Manage Content & Collaborate in the Cloud.
Tips for Nonprofits-  Manage Content & Collaborate in the Cloud.Tips for Nonprofits-  Manage Content & Collaborate in the Cloud.
Tips for Nonprofits- Manage Content & Collaborate in the Cloud.
 
Open Source: Beyond the Bottom Line - David Duffett
Open Source: Beyond the Bottom Line - David DuffettOpen Source: Beyond the Bottom Line - David Duffett
Open Source: Beyond the Bottom Line - David Duffett
 

More from Ralf Gommers

Reliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdfReliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdfRalf Gommers
 
Parallelism in a NumPy-based program
Parallelism in a NumPy-based programParallelism in a NumPy-based program
Parallelism in a NumPy-based programRalf Gommers
 
The road ahead for scientific computing with Python
The road ahead for scientific computing with PythonThe road ahead for scientific computing with Python
The road ahead for scientific computing with PythonRalf Gommers
 
Python array API standardization - current state and benefits
Python array API standardization - current state and benefitsPython array API standardization - current state and benefits
Python array API standardization - current state and benefitsRalf Gommers
 
Building SciPy kernels with Pythran
Building SciPy kernels with PythranBuilding SciPy kernels with Pythran
Building SciPy kernels with PythranRalf Gommers
 
Standardizing on a single N-dimensional array API for Python
Standardizing on a single N-dimensional array API for PythonStandardizing on a single N-dimensional array API for Python
Standardizing on a single N-dimensional array API for PythonRalf Gommers
 
PyData NYC whatsnew NumPy-SciPy 2019
PyData NYC whatsnew NumPy-SciPy 2019PyData NYC whatsnew NumPy-SciPy 2019
PyData NYC whatsnew NumPy-SciPy 2019Ralf Gommers
 
The evolution of array computing in Python
The evolution of array computing in PythonThe evolution of array computing in Python
The evolution of array computing in PythonRalf Gommers
 
__array_function__ conceptual design & related concepts
__array_function__ conceptual design & related concepts__array_function__ conceptual design & related concepts
__array_function__ conceptual design & related conceptsRalf Gommers
 
NumPy Roadmap presentation at NumFOCUS Forum
NumPy Roadmap presentation at NumFOCUS ForumNumPy Roadmap presentation at NumFOCUS Forum
NumPy Roadmap presentation at NumFOCUS ForumRalf Gommers
 
NumFOCUS_Summit2018_Roadmaps_session
NumFOCUS_Summit2018_Roadmaps_sessionNumFOCUS_Summit2018_Roadmaps_session
NumFOCUS_Summit2018_Roadmaps_sessionRalf Gommers
 
SciPy 1.0 and Beyond - a Story of Community and Code
SciPy 1.0 and Beyond - a Story of Community and CodeSciPy 1.0 and Beyond - a Story of Community and Code
SciPy 1.0 and Beyond - a Story of Community and CodeRalf Gommers
 

More from Ralf Gommers (12)

Reliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdfReliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdf
 
Parallelism in a NumPy-based program
Parallelism in a NumPy-based programParallelism in a NumPy-based program
Parallelism in a NumPy-based program
 
The road ahead for scientific computing with Python
The road ahead for scientific computing with PythonThe road ahead for scientific computing with Python
The road ahead for scientific computing with Python
 
Python array API standardization - current state and benefits
Python array API standardization - current state and benefitsPython array API standardization - current state and benefits
Python array API standardization - current state and benefits
 
Building SciPy kernels with Pythran
Building SciPy kernels with PythranBuilding SciPy kernels with Pythran
Building SciPy kernels with Pythran
 
Standardizing on a single N-dimensional array API for Python
Standardizing on a single N-dimensional array API for PythonStandardizing on a single N-dimensional array API for Python
Standardizing on a single N-dimensional array API for Python
 
PyData NYC whatsnew NumPy-SciPy 2019
PyData NYC whatsnew NumPy-SciPy 2019PyData NYC whatsnew NumPy-SciPy 2019
PyData NYC whatsnew NumPy-SciPy 2019
 
The evolution of array computing in Python
The evolution of array computing in PythonThe evolution of array computing in Python
The evolution of array computing in Python
 
__array_function__ conceptual design & related concepts
__array_function__ conceptual design & related concepts__array_function__ conceptual design & related concepts
__array_function__ conceptual design & related concepts
 
NumPy Roadmap presentation at NumFOCUS Forum
NumPy Roadmap presentation at NumFOCUS ForumNumPy Roadmap presentation at NumFOCUS Forum
NumPy Roadmap presentation at NumFOCUS Forum
 
NumFOCUS_Summit2018_Roadmaps_session
NumFOCUS_Summit2018_Roadmaps_sessionNumFOCUS_Summit2018_Roadmaps_session
NumFOCUS_Summit2018_Roadmaps_session
 
SciPy 1.0 and Beyond - a Story of Community and Code
SciPy 1.0 and Beyond - a Story of Community and CodeSciPy 1.0 and Beyond - a Story of Community and Code
SciPy 1.0 and Beyond - a Story of Community and Code
 

Recently uploaded

Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxReal-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxRTS corp
 
2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shardsChristopher Curtin
 
Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Anthony Dahanne
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfDrew Moseley
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanyChristoph Pohl
 
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsSensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsChristian Birchler
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesKrzysztofKkol1
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogueitservices996
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odishasmiwainfosol
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Cizo Technology Services
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceBrainSell Technologies
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Matt Ray
 
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalLionel Briand
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationBradBedford3
 
SoftTeco - Software Development Company Profile
SoftTeco - Software Development Company ProfileSoftTeco - Software Development Company Profile
SoftTeco - Software Development Company Profileakrivarotava
 
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Natan Silnitsky
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Developmentvyaparkranti
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identityteam-WIBU
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 

Recently uploaded (20)

Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxReal-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
 
2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards
 
Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024Not a Kubernetes fan? The state of PaaS in 2024
Not a Kubernetes fan? The state of PaaS in 2024
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdf
 
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte GermanySuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
SuccessFactors 1H 2024 Release - Sneak-Peek by Deloitte Germany
 
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsSensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogue
 
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company OdishaBalasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
Balasore Best It Company|| Top 10 IT Company || Balasore Software company Odisha
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. Salesforce
 
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
Open Source Summit NA 2024: Open Source Cloud Costs - OpenCost's Impact on En...
 
Precise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive GoalPrecise and Complete Requirements? An Elusive Goal
Precise and Complete Requirements? An Elusive Goal
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion Application
 
SoftTeco - Software Development Company Profile
SoftTeco - Software Development Company ProfileSoftTeco - Software Development Company Profile
SoftTeco - Software Development Company Profile
 
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
Taming Distributed Systems: Key Insights from Wix's Large-Scale Experience - ...
 
VK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web DevelopmentVK Business Profile - provides IT solutions and Web Development
VK Business Profile - provides IT solutions and Web Development
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identity
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 

Inside NumPy: preparing for the next decade

  • 1. Inside NumPy: preparing for the next decade Ralf Gommers, Sebastian Berg, Matti Picus,
 Tyler Reddy, Stéfan van der Walt, Charles Harris
  • 2. A very brief history of NumPy 2005: NumPy created 2006: NumPy 1.0 …….: 1-2 releases every year, gradual progress 2015: governance and NumFOCUS relationship formalized 2018: first-ever paid developers hired
  • 3. The Sloan & Moore grants to BIDS for NumPy • Two grants for in total $1.3M, Apr 2018 — Oct 2020 • Current co-PIs: Stéfan van der Walt and Fernando Perez • Social aims: Improve community engagement
 Grow core team, Diversify contributors • Technical aims: More flexible & sustainable code, Frequent & consistent releases, Improve data type system, New array protocol
  • 4. An organizational view of the project Steering Council The NumPy project Contributors Core developers Repositories Websites Code Documentation NumFOCUS
 Fiscal sponsor people BIDS
 people Quansight
 people your org?
 Institutional Partners
  • 5. Key questions we’ll try to answer in this talk • Has grant funding over the last year invigorated the NumPy project? • How healthy/sustainable is NumPy today? • What does NumPy need in order to thrive? • What’s our vision for NumPy, and what is our plan to achieve that vision?
  • 6. Impact of grant funding — an attempt to quantify Heavier-tailed distributions in 2018-19: bus factor increased. Fernando Perez, Ten years of (interactive) scientific computing
  • 7. Impact of grant funding — an attempt to quantify Heavier-tailed distributions in 2018-19: bus factor increased. Fernando Perez, Ten years of (interactive) scientific computing
  • 8. Impact of grant funding — an attempt to quantify Volunteer contributions relatively stable; total activity up.
  • 9. Impact of grant funding — an attempt to quantify Issue/PR count rate of change is key indicator of project health. Nadia Eghbal, Methodologies for measuring project health
  • 10. Impact of grant funding — qualitative • We now have a roadmap • We’ve been able to organize core team sprints • NumPy Enhancement Proposal process revived • We paid down a decent amount of technical debt • The 1.17.0 release (out any day now) is the largest release in a long time (at least since 1.7.0 in 2013). • Planning the Array Developer Summit for next March.
  • 11. Has grant funding over the last year invigorated the NumPy project? Yes, we think so. Faster progress and more hands for maintenance also makes it more fun.
  • 12. How healthy/sustainable is NumPy today? A little more healthy than a year ago. Still a worry though. Bus factor estimate: ~5
  • 14. So who did we hire? Two full-time engineers, ongoing (till Oct 2020 at least): • Mar 2018 — : Matti Picus (PyPy maintainer) • Jun 2018 — Jun 2019: Tyler Reddy (SciPy maintainer) • May 2019 — : Sebastian Berg (NumPy Steering Council member) Additionally: • Supporting Kriti Singh through Outreachy (docs work) • Planning for a few more short-term contractors (code & docs)
  • 16. Funding — what to pay for? • Pay for things that are important and otherwise won't get done. • Think broadly. There's more to a project than code! • Balance maintenance & innovation: keep people motivated! • Plan for sustainability. Don't start depending on funding unless you’re fairly certain it’s stable. • Make the life of volunteer maintainers easier, not harder!
  • 17. Funding — who to pay? • All else being equal, give preference to existing maintainers. • Pay attention to communication and self-management skills, in addition to technical skills and motivation. • Consider this an opportunity to make your project more diverse.
  • 19. Sustainability — maintainer bandwidth Of the 11 Steering Council members: • 3 are very active (Chuck, Eric, Stephan) • 3 are paid to work on NumPy
 (Sebastian full-time,
 Stéfan & Ralf a small part of their time (~1 day/wk)) • 5 are in low-activity mode (infrequent emails/commits) This is still a major challenge! NumPy depends both on a handful of people, and probably on continued funding.
  • 20. The project beyond code It’s still hard getting work done on: • High-level documentation • Website • Community building • Governance and project management • Long-term planning (both technical and organizational)
  • 21. Improving NumPy’s culture • We still struggle with a lack of diversity: all maintainers are white and male. • Our GitHub and mailing list culture is occasionally not as friendly and welcoming as we’d all like it to be.
  • 23. Scope & Vision The key thing NumPy offers is: an array object (N-dimensional, in-memory, on CPUs) and array computing APIs NumPy lives at the heart of the numerical Python ecosystem. We want to: evolve while remaining a stable base, address bottlenecks that limit how the
 ecosystem can grow, and grow and diversify our team and community.
  • 24. What does NumPy need in order to thrive? • Sustained funding at a higher level than today.
 Our estimate: 10 full-time people • More bandwidth from key people for long-term planning, managing the project • Attracting people in roles that primarily focus on activities other than coding
  • 25. Parts of a plan to get there • In the next year, work with tech writers and web developers to start building a docs/web team.
 Efforts already ongoing. • Build a diverse and robust funding quilt:
 Express our needs as a community (not just NumPy),
 and ask funders and major users for contributions:
 Recently started (e.g. pitched to NASA), want to make this much more concrete at SciPy’19.