This document summarizes a meeting of the Maths and Stats Working Group hosted by Howard Moody of eduserv on December 3rd, 2014. The agenda included an introduction to NAG, a summary of the current NAG agreement and product highlights, and a request for feedback from members on renewing the agreement. NAG provides numerical algorithms and HPC services to academic and commercial users.
NAG Presentation to Eduserv Maths and Stats Software Dec2014
1. Maths and Stats Working Group
Experts in numerical algorithms
and HPC services
John Holden
Louise Mitchell
Host: Howard Moody, eduserv
3rd December 2014
2. “Howard, what’s the agenda?”
The content is reasonably technical, perhaps some new
features, what’s coming in the development pipeline
etc. I would also like to get some feedback from the
members of the group as to any changes they would
like when we renew the NAG Chest Agreement next
summer.
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 2
3. AGENDA
Who are NAG?
Current agreement summary & highlights
Product highlights
“feedback from the members of the group”
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 3
4. 4
NAG Profile
• Academia, Government Research
• Software companies (finance, analytics, research), Investment banks,
aerospace, engineering
• Energy, manufacturing, consumer analytics
• HPC support - England, Wales, South Africa, US, Middle East
Customers
• US (Chicago, Houston)
• UK (Oxford, Manchester)
• Japan (Tokyo)
• Staff in Germany & France
Offices
• Started 1970 from five British universities & a national lab
• Commercial, Not-for-profit (no shareholders)
• ~60 staff, >50% technical, 20 PhDs in various disciplines Origins
5. 5
What we do - today
• Mathematical, statistical, data analysis
components for developers in libraries
• Bespoke (custom) methods
Algorithms
• Scaling and performance tuning of user/
commercial applications
• Specialty performance libraries
• Specialty ports of libraries
Numerical
Software
Engineering
• Procurement – planning, RFPs, benchmarks,
acceptance
• Computational science & engineering for user
applications
HPC
6. 6
What we continue to do
•Mathematical & statistical components for
•C/C++, Fortran, MATLAB, Python, .NET, Java, etc.
•A new release annually with 100+ new methods
Numerical
Libraries
•Tuned ports for hardware vendors of NAG Library
•Performance libraries for various chip architectures
Specialty
Libraries
•Training courses and code tuning for various
accelerator architectures (CUDA, OpenCL, etc)
•Scaling and performance tuning
•Support for various open source packages
Specialist
Services
Whilst collaborating with a wide
variety of academic partners
7. Current Product Portfolio
Numerical Libraries
NAG Fortran Library
NAG C Library
NAG Toolbox for MATLAB
NAG Library for .NET
NAG Library for Python (new)
NAG Library for Java (new)
NAG Data Mining Components
NAG HPC Library
NAG Library for SMP & Multi-core (Fortran)
NAG Library for Xeon Phi (Fortran) (new)
NAG Fortran Compiler
inc. Fortran Builder - GUI based Compiler for Windows
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 7
8. NAG Library and Toolbox Contents
Root Finding
Summation of Series
Quadrature
Ordinary Differential
Equations
Partial Differential Equations
Numerical Differentiation
Integral Equations
Mesh Generation
Interpolation
Curve and Surface Fitting
Optimization
Approximations of Special
Functions
Dense Linear Algebra
Sparse Linear Algebra
Correlation & Regression
Analysis
Multivariate Methods
Analysis of Variance
Random Number Generators
Univariate Estimation
Nonparametric Statistics
Smoothing in Statistics
Contingency Table Analysis
Survival Analysis
Time Series Analysis
Operations Research
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 8
9. NAG Libraries - Ease of integration
Supporting Wide Range of Operating systems…
Windows, Linux, Mac, …
…and a number of interfaces
C, C++,
Fortran,
VB, Excel & VBA,
C#, F#, VB.NET,
CUDA, OpenCL,
Java,
Python
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 9
…
Excel,
LabVIEW,
MATLAB,
Maple,
Mathematica
R, S-Plus,
Scilab, Octave
…
10. AGENDA
Who are NAG?
Current agreement summary & highlights
Product highlights
“feedback from the members of the group”
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 10
11. Current Agreement
Runs 1st August 2010 – 31st July 2015
Simplified licensing model
Smaller number of product categories
Changed to Operating System based from Implementation based
Dept. or site licence
Generous and flexible terms
Staff and students may install on personal machines as well as University
owned hardware such as HPC clusters, classroom work stations
Free product training
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 11
12. Current Agreement
Improved and more flexible licence management
Changed from various mechanisms (FLEXlm and other) to KUSARI
Annual “Open” Keys are the most popular and flexible – provided on
trust basis
Simplifies Image Roll out
Reduces headaches of server licences (sometimes caused by NAT routers, changing
IP address)
Admin. burden lightened for site representatives*
Software issued direct to users, technical support direct to users (staff
and students)
*subject to their agreement
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 12
13. Current Agreement – “user delight”
The “best” Toolbox for MATLAB and you get it from
NAG……………
“The NAG Library range of
implementations and
interfaces such as VBA,
MATLAB, Python allow fast
transition between
prototype on the desktop to
production C or Fortran code
on HPC facilities.”
“NAG’s SMP Libraries and NAG Toolbox
for MATLAB (also SMP enabled) allow me
to do HPC on desktop before switching to
the HPC machine for further speed up
with. NAG’s SMP offering means
sometimes MPI coding is not needed”
“A fraction of the price of other
equivalent math site licences &
delivering far more value & flexibility”
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 13
14. Current Agreement – “user delight”
NAG listen to academic user requests…
“Closed Form” Options Pricing functions were added to the NAG Library to facilitate
better teaching of financial mathematics for a number of MSc programmes
Non-negative least squares solver added to the NAG Library for a research project
…
The NAG Fortran Compiler
Perfect for teaching as well as research
Great licensing terms…. students can use on their own machines
Portable (Windows, Mac, Linux & other UNIX)
and
keeps pace with the latest Fortran Features
supports OpenMP
EXCELLENT (the world’s best) checking compiler
Windows version has a great GUI
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 14
15. Current Agreement – “partner delight”
Many active collaborations and contributions from UK
academia
Nick Higham, UoM – NCM &
Matrix Functions
Cathyrn Powell, UoM – Gaussian
Random Field generators
Rebecca Killick, Lancs
– Change Point Analysis
William Shaw, UCL – Quantiles
Michal Kocvara, Birmingham – Optimisation
Oleg Davydov, Strathclyde – Splines & Wavelets Radial
Basis functions
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 15
16. Current Agreement - frustration
With NAG being a not-for-profit organisation with its roots and
collaboration relationships firmly established with the academic
community we continue to provide our software on a “maximum
usage” rather than “maximum revenue” basis
this is reflected in the current eduserv agreement
Some Universities “cross charge” or enforce stricter terms on their users
preventing licences being used in the intended spirit of the agreement
We recognise there are only a few exceptions, some examples:
Although the licence allows installation on personal machines the site rep has
blocked this arrangement
Site licence only funded by two or three depts. and an unrealistic cross charge
scheme for individual users from other depts. meaning additional users are deterred
Possible solutions? Perhaps there isn’t one…
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 16
17. Current Agreement - observation
Most (not all) student and staff personal machines are Win or Mac o/s
Most HPC centres are Linux (or other UNIX)
The Universities who have been most successful in creating “student & staff buy in” for
encouraging good Mathematical Programming and broader usage of HPC have adopted
Linux and Windows and/or Mac o/s licence agreements
“If Universities are serious about encouraging wider adopt of HPC and
programming they need to make software and tools available to user in an
environment they are familiar with such as Windows and Mac o/s”
from leading scientist at Software Sustainability Institute
Universities who have taken this approach include (but not limited to)...
Birmingham, Heriot-Watt, Manchester, Warwick,…
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 17
18. Current Agreement - training
Free to all Universities who hold a Dept or Site Licence
At your University or NAG Offices (Oxford or Manchester)
Example training courses
Using NAG Toolbox for MATLAB
Using the NAG Library with Excel/VBA
Using the NAG C Library with C & C++
Using the NAG Library for Python
OpenMP Demystified featuring the NAG SMP Library
How to write Portable, Bug-Free Fortran using the NAG Compiler
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 18
..
…
19. AGENDA
Who are NAG?
Current agreement summary & highlights
Product highlights
“feedback from the members of the group”
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 19
20. What’s new and coming soon
Updates
NAG Libraries (inc. Toolbox for MATLAB) at Mark 24
NAG Library for .NET at release 2.0
NAG Fortran Compiler at release 6.0
New
NAG Library for Java released
NAG Library for Python released
NAG Library for Xeon Phi released
Coming soon
NAG C Library for SMP & Multi-core
Mark 25 – will be released in 2015
Implementations for ARM hardware
HPC Services
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 20
21. NAG Library, What’s new at Mark 24
Confluent Hypergeometric
Function (1F1, 2F1)
Two-stage spline
approximation to scattered
data
Further additions to
Nearest Correlation Matrix
(Mark 24)
More Multi-start
optimization
More Matrix
Functions
Inhomogeneous time
series
Gaussian mixture
model
Partial Least Squares /
Ridge Regression
Quantiles
Search routines
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 21
22. Skip
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 22
22
The NAG Library for Java
Offers easy access to NAG routines from Java.
History
It’s not too difficult to call the NAG Fortran Library from
Java. But most Java developers don't have time to write
the code to do this.
Now there is a full set of Java wrappers for NAG routines –
at no cost to the users. (Users need to have a licence for
the NAG Fortran Library)
Documentation for NAG Library and Users’ Note for Java
wrappers
*Users need to licence and install the NAG Fortran Library
23. 23
1 Java wrapper = 2 parts (2 files in one Zip):
1 Java class
1 C/JNI source file
User's Java App
NAG Java
Wrapper's class
NAG C/JNI
function
NAG Fortran
Library
User's callback
To use the NAG Library for Java
The design
24. The components
Need to install the NAG Fortran Library (MK 24)
Also download the Java wrappers
Single zip from NAG Includes 2 parts: Java class and C/JNI source file
The NAGLibraryFromJava.jar file contains all NAG Java classes
a shared library (a DLL file for windows / a .so file for Linux) the C/JNI interface to NAG
Library
Usage
A Java package under com.nag.routines is a single NAG chapter
A Java class is a single NAG routine
Other Notes:
Arguments are Java basic types, except for complex numbers;
Complex numbers are represented by a Java interface to implement;
Arguments are attributes of the routine class;
Callback functions are Java interfaces to implement (force the function signature but
leave the storage definition to the user)
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 24
25. 25
The developer’s view
For the Java developer:
import com.nag.routines.e04.E04GB;
[….]
public static main(String[ ] args) {
[…]
LSQFUN lsqfun = new LSQFUN();
[...]
E04GB e04gb = new E04GB(m, n, lsqlin, lsqfun, lsqmon, iprint, maxcal, eta, xtol, stepmx, x,
fsumq, fvec, fjac, ldfjac, s, v, ldv, niter, nf, iw, liw, w, lw, ifail);
[...]
e04gb.eval();
ifail = e04gb.getIFAIL();
[...] + Exception handling
}
private static class LSQLIN implements E04GBJ.E04GBJ_LSQLIN {
[...]
public void eval(int SELCT) {
E04HEV e04hev = new E04HEV(SELCT);
e04hev.eval();
this.SELCT = e04hev.getLSQLIN_SELECT();
}
}
Import routine as a
Java ‘class’
Create an ‘object’
of the Eo4GB class
Arguments are Java
basic types.
(in most cases)
26. NAG Library for Python
Full set of bindings available for NAG C Library
Supported by white papers for calling NAG Fortran or
C Library from Python
http://www.nag.co.uk/python.asp
Combines NAG’s high quality numerical routines with
the ease of use of Python
Access to NAG routines from Python for quick prototyping
Same high quality NAG routines used in production system
(C, Fortran, .NET, Java, …) as used under Python prototype
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 26
27. nag4py
Built on top of NAG C Library + Documentation
1600 NAG functions easily accessible from python
25 examples programs to help users call NAG
functions
from nag4py.c05 import c05ayc
from nag4py.util import NagError,Nag_Comm
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 27
28. Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 28
29. NAG & Python - Implied Vol Timings example
Method Timing
fsolve + python
fsolve + NAG
nag4py
NAG C
~180 seconds
~15 seconds
~10 seconds
~.29 seconds
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 29
30. Imports from the nag4py module for
nag_prob_normal
Hand written python function to wrap
nag_prod_normal using default arguments,
docstring and doctests
Call the nag function nag_prob_normal
Uses exception for error handling
Examples of calling the wrapped
function normal_prob for
nag_prob_normal
Python Example
from ctypes import c_double
from nag4py.util import NagError, Nag_LowerTail, Nag_UpperTail, INIT_FAIL
from nag4py.g01 import nag_prob_normal
def normal_prob(x, tail=Nag_LowerTail):
"“”
The lower tail probability for the standard Normal distribution
>>> normal_prob(1.0)
0.841344746069
>>> normal_prob(1.0, tail=Nag_UpperTail)
0.158655253931
>>> normal_prob(1.0, tail=1)
Exception: Argument tail has an illegal value: 1
""“
x = c_double(x)
fail = NagError()
INIT_FAIL(fail)
res = nag_prob_normal(tail,x,fail)
if (fail.code == 70):
raise Exception("Argument tail has an illegal value:" , tail)
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 30
return res
def main():
print normal_prob(1.0)
print normal_prob(1.0, tail=Nag_UpperTail)
print normal_prob(1.0, tail=1)
if __name__ == "__main__":main()
31. NAG and Python
Python is a great, easy to use, powerful language
NAG Library provides high quality numerical routines
We allow you to combine these strengths
Access to NAG routines from Python for quick prototyping
Same high quality NAG routines used in production system
(C, Fortran, .NET, Java, …) as used under Python prototype
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 31
32. NAG for Intel Xeon Phi
Numerical routines for Intel Many Integrated
Cores architecture
Library based on our SMP Library
Native mode: run entirely on MIC
Offload mode: heterogeneous execution on CPU/MIC
Automatic and Programmer controlled offload modes supported
*NAG had the first MIC board in UK outside of Intel
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 32
32
33. [HPC] Training Courses
Over the last 6 years NAG HPC services have trained >2000 course attendees
Users
• Introduction to HPC
• Using [Platform/Service]
e.g. HECToR
• Transitioning to
[Platform/Service]
e.g. Cray XE6
• Multicore Awareness
• [Application specific]
Programmers
• Fortran, C, C++
• MPI, OpenMP, CAF, UPC
• Xeon Phi, CUDA, OpenCL
• Parallel I/O
• Debugging, Profiling
and Optimisation
• Core Algorithms for
High Performance
Scientific Computing
• Best Practice in HPC
Software Development
Management
• What an R&D manager
needs to know about HPC
• Procurement
• Benchmarking
• Service Planning &
Delivery
• …
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 34
34. AGENDA
Who are NAG?
Current agreement summary & highlights
Product highlights
“feedback from the members of the group”
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 35
35. Experts in numerical algorithms
and HPC services
Ways to contact us:
www.nag.co.uk
Technical Support and Help
support@nag.co.uk
Account Manager
louise.mitchell@nag.co.uk
NAGNews:
http://www.nag.co.uk/NAGNews/Index.asp
Twitter:
www.twitter.com/NAGTalk
Blog:
http://blog.nag.com/
LinkedIn:
http://www.linkedin.com/e/vgh/2707514/
36. SPARES
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 37
37. Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 38
38
NAG Library (24)
1F1(a;b;x)
- Confluent Hypergeometric Function (Mark 24)
Also known as Kummer’s
function M(a,b,x)
This has a wide variety of
applications, including CIR
processes and pricing Asian
options.
38. Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 39
39
NAG Library (24)
Two-stage spline approximation to scattered
data
Computes a spline approximation for curve and surface
fitting.
Designed for large data sets: typical real-world input (from,
say, geosciences, data mining, or medical imaging) can
contain millions of points.
For sufficiently-dense input data
consisting of n points, n-linear
complexity and memory usage
can be attained.
39. Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 40
40
NAG Library (24)
Multi-start Optimization
Uses a local optimization routine with many different start
points to address a global optimization problem .
There are 2 routines.
E05UC is for an arbitrary smooth function subject to constraints.
E05US for constrained nonlinear least-squares problem.
The user can also get a set of solutions for other minima.
Useful when there are other criteria desirable in an
acceptable solution.
For example a need for a very stable solution that doesn’t change
much if the independent variables are slightly different from their
optimal values.
40. Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 41
41
NAG Library (24)
Gaussian mixture model
A mixture model is a segmentation method where each
subgroup is modelled by Gaussian (Normal) distribution.
It does not require that an observed data-set should
identify the sub-population to which an individual belongs.
Examples of use:
modelling buyer behaviour by identifying
different customer groups
the distribution of power loads over a
network
41. Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 42
42
NAG Library(24)
Matrix Functions (Mark 24)
Mark 24 contains new routines for estimating matrix
function condition numbers.
condition numbers for the exponential, logarithm, sine,
cosine, or hyperbolic sine or cosine of general real and
complex matrices; and
condition numbers for matrix functions corresponding to
user-supplied subroutines, using either user supplied
derivatives or numerical differentiation.
Note: The condition number of a matrix function is a measure
of the sensitivity of the solution to small changes in the input.
Also Mark 24 has routines for computing the action of the
matrix exponential on a vector (also with a reverse
communication interface for use with sparse matrices)
42. Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 43
43
NAG Library (24)
Inhomogeneous time series (Mark 24)
At Mark 24 we have introduced operators, that extract
robust information directly from an inhomogeneous time
series. The results are essentially independent of minor
changes to the sampling mechanism used when collecting
the data, for example, changing a number of time stamps
or adding or removing a few observations.
Note: An inhomogeneous time series has unevenly spaced data
and standard time series analysis techniques cannot be used.
43. Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 44
44
NAG Library (24)
Further additions to Nearest Correlation Matrix
The algorithm was described in a paper by Qi & Sun with a
superior rate of convergence over previous methods.
Further improvements, including different iterative solver
and a means of preconditioning the linear equations.
Performance enhancements, by better exploiting the
structure of the problem and added Normwise weighting
on bounds on eigenvalues
Functionality has been enhanced to allow element-wise
weighting.
44. How do you call the functions in Excel?
Excel function
wizard
Enter the
inputs from the
spreadsheet
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 45
45. NAG and Excel
Our libraries are easily accessible from Excel:
Calling NAG DLLs using VBA
NAG provide VB Declaration
Statements and Examples
NAG provide add-ins:
Stats & Survival Analysis
And examples:
Local volatility, Variance
Gamma, NCM, …
Calling NAG Library for .NET
using VSTO
functions with Reverse
Communication (useful for
Solver replication for
example)
Create NAG XLLs
Trusted experts in HPC & software innovation | professional services, training & impartial advice | Results Matter. Trust NAG. 46