SlideShare a Scribd company logo
1 of 7
Download to read offline
__array_function__ conceptual
design & related concepts
Ralf Gommers
Context
These slides contain some sketches about
what __array_function__ does and is for, as
well as thoughts on how interoperability,
ndarray subclasses and “duck arrays”
relate to it.
2
NumPy API and semantics
3
Function body

(the “implementation”, defines
semantics)
Function signature (the API)
Example for one function - this applies to ufuncs
and other functions. Of course there are also other
objects in the NumPy API - let’s ignore those for simplicity.
__array_function__ concept
4
Function body
Function signature (the API)
tl;dr use the NumPy API, bring your own implementation
Function body
Function signature
Function signature (the API)
if input arg has __array_function__:
execute other_function
Function body
Function signature
==
“array_like” vs “duck typing”
Most NumPy functions accept “array-like” input, which is:
“An array, any object exposing the array interface, an object whose __array__
method returns an array, or any (nested) sequence.”
Duck typing:
"If it walks like a duck and it quacks like a duck, then it must be a duck”
Lists and tuples clearly aren’t the same animal as ndarray. Hence
the NumPy API does not use duck typing. Instead, it coerces
inputs to ndarray.
It’s unclear what an ndarray “duck” even means - MaskedArray
could be some sort of duck, or it could be a swan ….
5
Reusing NumPy implementations of the functions in its API?
6
Reusing part of the implementation of NumPy functions is an
order of magnitude harder than reusing the API. doable …??
Function body
Function signature
Function signature (the API)
if input arg has __some_attribute__:
customize implementation==
array_like “duck array” or
ndarray subclass
Function body
….
np.asarray
….
rest of function body
Function signature
Function signature (the API)
if input arg has __some_attribute__:
customize implementation==
vs.
NEP 18 goals
◎ Separate NumPy API from NumPy “execution engine”
◎ Allow other array libraries (Dask, CuPy, pydata/sparse,
PyTorch, …) to reuse the NumPy API
◎ In the bigger picture: reduce or avoid ecosystem
fragmentation (e.g. we don’t want to see a reimplementation of SciPy for
PyTorch, SciPy for Tensorflow, etc.).
Note: NEP 18 increases interoperability in a fully backwards compatible way.
Duck typing and handling “duck arrays” or ndarray subclasses differently are
specific kinds of interoperability that NEP 18 does not directly deal with.
7

More Related Content

What's hot

C++ Function
C++ FunctionC++ Function
C++ Function
Hajar
 
Compiled Python UDFs for Impala
Compiled Python UDFs for ImpalaCompiled Python UDFs for Impala
Compiled Python UDFs for Impala
Cloudera, Inc.
 

What's hot (20)

FUNCTIONS IN c++ PPT
FUNCTIONS IN c++ PPTFUNCTIONS IN c++ PPT
FUNCTIONS IN c++ PPT
 
C++ programming function
C++ programming functionC++ programming function
C++ programming function
 
Functions in C++
Functions in C++Functions in C++
Functions in C++
 
C++ Function
C++ FunctionC++ Function
C++ Function
 
Functions in C++
Functions in C++Functions in C++
Functions in C++
 
16717 functions in C++
16717 functions in C++16717 functions in C++
16717 functions in C++
 
03 function overloading
03 function overloading03 function overloading
03 function overloading
 
Variadic functions
Variadic functionsVariadic functions
Variadic functions
 
Php string function
Php string function Php string function
Php string function
 
C++ Function
C++ FunctionC++ Function
C++ Function
 
Functional Programming in JavaScript
Functional Programming in JavaScriptFunctional Programming in JavaScript
Functional Programming in JavaScript
 
Command line arguments
Command line argumentsCommand line arguments
Command line arguments
 
Advanced Python : Decorators
Advanced Python : DecoratorsAdvanced Python : Decorators
Advanced Python : Decorators
 
How much performance can you get out of Javascript? - Massimiliano Mantione -...
How much performance can you get out of Javascript? - Massimiliano Mantione -...How much performance can you get out of Javascript? - Massimiliano Mantione -...
How much performance can you get out of Javascript? - Massimiliano Mantione -...
 
Functions in c++
Functions in c++Functions in c++
Functions in c++
 
Introduction to Functional Programming (w/ JS)
Introduction to Functional Programming (w/ JS)Introduction to Functional Programming (w/ JS)
Introduction to Functional Programming (w/ JS)
 
Spring RTS Engine Checkup
Spring RTS Engine CheckupSpring RTS Engine Checkup
Spring RTS Engine Checkup
 
JavaScript Functions
JavaScript FunctionsJavaScript Functions
JavaScript Functions
 
Lecture5
Lecture5Lecture5
Lecture5
 
Compiled Python UDFs for Impala
Compiled Python UDFs for ImpalaCompiled Python UDFs for Impala
Compiled Python UDFs for Impala
 

Similar to __array_function__ conceptual design & related concepts

Introduction to scala for a c programmer
Introduction to scala for a c programmerIntroduction to scala for a c programmer
Introduction to scala for a c programmer
Girish Kumar A L
 
Scala overview
Scala overviewScala overview
Scala overview
Steve Min
 

Similar to __array_function__ conceptual design & related concepts (20)

The evolution of array computing in Python
The evolution of array computing in PythonThe evolution of array computing in Python
The evolution of array computing in Python
 
Scala in a nutshell by venkat
Scala in a nutshell by venkatScala in a nutshell by venkat
Scala in a nutshell by venkat
 
Introduction to scala for a c programmer
Introduction to scala for a c programmerIntroduction to scala for a c programmer
Introduction to scala for a c programmer
 
Scala for Java Programmers
Scala for Java ProgrammersScala for Java Programmers
Scala for Java Programmers
 
The Joy Of Ruby
The Joy Of RubyThe Joy Of Ruby
The Joy Of Ruby
 
Standardizing arrays -- Microsoft Presentation
Standardizing arrays -- Microsoft PresentationStandardizing arrays -- Microsoft Presentation
Standardizing arrays -- Microsoft Presentation
 
обзор Python
обзор Pythonобзор Python
обзор Python
 
Java 8 features
Java 8 featuresJava 8 features
Java 8 features
 
Scala final ppt vinay
Scala final ppt vinayScala final ppt vinay
Scala final ppt vinay
 
Howto argparse
Howto argparseHowto argparse
Howto argparse
 
Introduction of Java 8 with emphasis on Lambda Expressions and Streams
Introduction of Java 8 with emphasis on Lambda Expressions and StreamsIntroduction of Java 8 with emphasis on Lambda Expressions and Streams
Introduction of Java 8 with emphasis on Lambda Expressions and Streams
 
Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009Scala Talk at FOSDEM 2009
Scala Talk at FOSDEM 2009
 
ScalaDays 2013 Keynote Speech by Martin Odersky
ScalaDays 2013 Keynote Speech by Martin OderskyScalaDays 2013 Keynote Speech by Martin Odersky
ScalaDays 2013 Keynote Speech by Martin Odersky
 
PHP - Introduction to PHP Functions
PHP -  Introduction to PHP FunctionsPHP -  Introduction to PHP Functions
PHP - Introduction to PHP Functions
 
20 cool features that is in PHP 7, we missed in PHP 5. Let walkthrough with t...
20 cool features that is in PHP 7, we missed in PHP 5. Let walkthrough with t...20 cool features that is in PHP 7, we missed in PHP 5. Let walkthrough with t...
20 cool features that is in PHP 7, we missed in PHP 5. Let walkthrough with t...
 
Elegant Solutions For Everyday Python Problems - Nina Zakharenko
Elegant Solutions For Everyday Python Problems - Nina ZakharenkoElegant Solutions For Everyday Python Problems - Nina Zakharenko
Elegant Solutions For Everyday Python Problems - Nina Zakharenko
 
whats new in java 8
whats new in java 8 whats new in java 8
whats new in java 8
 
Scala overview
Scala overviewScala overview
Scala overview
 
Intellectual technologies
Intellectual technologiesIntellectual technologies
Intellectual technologies
 
Python Session - 4
Python Session - 4Python Session - 4
Python Session - 4
 

More from Ralf Gommers

More from Ralf Gommers (12)

Reliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdfReliable from-source builds (Qshare 28 Nov 2023).pdf
Reliable from-source builds (Qshare 28 Nov 2023).pdf
 
Parallelism in a NumPy-based program
Parallelism in a NumPy-based programParallelism in a NumPy-based program
Parallelism in a NumPy-based program
 
The road ahead for scientific computing with Python
The road ahead for scientific computing with PythonThe road ahead for scientific computing with Python
The road ahead for scientific computing with Python
 
Python array API standardization - current state and benefits
Python array API standardization - current state and benefitsPython array API standardization - current state and benefits
Python array API standardization - current state and benefits
 
Building SciPy kernels with Pythran
Building SciPy kernels with PythranBuilding SciPy kernels with Pythran
Building SciPy kernels with Pythran
 
Standardizing on a single N-dimensional array API for Python
Standardizing on a single N-dimensional array API for PythonStandardizing on a single N-dimensional array API for Python
Standardizing on a single N-dimensional array API for Python
 
Strengthening NumPy's foundations - growing beyond code
Strengthening NumPy's foundations - growing beyond codeStrengthening NumPy's foundations - growing beyond code
Strengthening NumPy's foundations - growing beyond code
 
PyData NYC whatsnew NumPy-SciPy 2019
PyData NYC whatsnew NumPy-SciPy 2019PyData NYC whatsnew NumPy-SciPy 2019
PyData NYC whatsnew NumPy-SciPy 2019
 
Inside NumPy: preparing for the next decade
Inside NumPy: preparing for the next decadeInside NumPy: preparing for the next decade
Inside NumPy: preparing for the next decade
 
NumPy Roadmap presentation at NumFOCUS Forum
NumPy Roadmap presentation at NumFOCUS ForumNumPy Roadmap presentation at NumFOCUS Forum
NumPy Roadmap presentation at NumFOCUS Forum
 
NumFOCUS_Summit2018_Roadmaps_session
NumFOCUS_Summit2018_Roadmaps_sessionNumFOCUS_Summit2018_Roadmaps_session
NumFOCUS_Summit2018_Roadmaps_session
 
SciPy 1.0 and Beyond - a Story of Community and Code
SciPy 1.0 and Beyond - a Story of Community and CodeSciPy 1.0 and Beyond - a Story of Community and Code
SciPy 1.0 and Beyond - a Story of Community and Code
 

Recently uploaded

Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
FIDO Alliance
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...Stronger Together: Developing an Organizational Strategy for Accessible Desig...
Stronger Together: Developing an Organizational Strategy for Accessible Desig...
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
JohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptxJohnPollard-hybrid-app-RailsConf2024.pptx
JohnPollard-hybrid-app-RailsConf2024.pptx
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 

__array_function__ conceptual design & related concepts

  • 1. __array_function__ conceptual design & related concepts Ralf Gommers
  • 2. Context These slides contain some sketches about what __array_function__ does and is for, as well as thoughts on how interoperability, ndarray subclasses and “duck arrays” relate to it. 2
  • 3. NumPy API and semantics 3 Function body
 (the “implementation”, defines semantics) Function signature (the API) Example for one function - this applies to ufuncs and other functions. Of course there are also other objects in the NumPy API - let’s ignore those for simplicity.
  • 4. __array_function__ concept 4 Function body Function signature (the API) tl;dr use the NumPy API, bring your own implementation Function body Function signature Function signature (the API) if input arg has __array_function__: execute other_function Function body Function signature ==
  • 5. “array_like” vs “duck typing” Most NumPy functions accept “array-like” input, which is: “An array, any object exposing the array interface, an object whose __array__ method returns an array, or any (nested) sequence.” Duck typing: "If it walks like a duck and it quacks like a duck, then it must be a duck” Lists and tuples clearly aren’t the same animal as ndarray. Hence the NumPy API does not use duck typing. Instead, it coerces inputs to ndarray. It’s unclear what an ndarray “duck” even means - MaskedArray could be some sort of duck, or it could be a swan …. 5
  • 6. Reusing NumPy implementations of the functions in its API? 6 Reusing part of the implementation of NumPy functions is an order of magnitude harder than reusing the API. doable …?? Function body Function signature Function signature (the API) if input arg has __some_attribute__: customize implementation== array_like “duck array” or ndarray subclass Function body …. np.asarray …. rest of function body Function signature Function signature (the API) if input arg has __some_attribute__: customize implementation== vs.
  • 7. NEP 18 goals ◎ Separate NumPy API from NumPy “execution engine” ◎ Allow other array libraries (Dask, CuPy, pydata/sparse, PyTorch, …) to reuse the NumPy API ◎ In the bigger picture: reduce or avoid ecosystem fragmentation (e.g. we don’t want to see a reimplementation of SciPy for PyTorch, SciPy for Tensorflow, etc.). Note: NEP 18 increases interoperability in a fully backwards compatible way. Duck typing and handling “duck arrays” or ndarray subclasses differently are specific kinds of interoperability that NEP 18 does not directly deal with. 7