An R-matrix approach for plasma modelling and the interpretation of astrophysical observation
An R-matrix approach for plasmamodelling and the interpretation of astrophysical plasmas thJuly 27 , Queens University, ICPEAC 2011 Connor Ballance Auburn UniversityCollaborators : T G Lee, S D Loch, M S Pindzola (AU) : N R Badnell ( Strathclyde ) : B M McLaughlin (QUB) : M A Bautista (WMU) Supported by : US DoE Fusion Energy Sciences
Overview● Introduction : Comprehensive approach to plasma modelling, our current capabilities and future directions.● Electron-impact ionisation : high n shell ionisation● Electron-impact excitation : scripted R-matrix calculations : parallel DARC code : Fe-Peak elements and beyond● Photo-ionisation of Mid-Z elements : parallel dipole (DARC) codes
Introduction● In recent years, EIE R-matrix calculations, have moved beyond the isolated, one-off serial calculations to parallel calculations along entire iso-nuclear/iso-electronic sequences Witthoeft et al 2007 (J. Phys. B Vol 40) Liang and Badnell 2010 (Astron. Astrophys. Vol 518 A64) Perl-scripted calculations, automatically calculate tabulated every effective collision strength for all transitions from user given structure.● This data is stored in a well-prescribed format that includes the atomic configurations, the energy levels, the A-Values for all E1,E2,M1,M2 transitions, Maxwellian averaged collision strengths for a range of temperatures and the Born/Bethe infinite energy limit points. (why?) .
Introduction● Modern computing archictectures have over 100,000 cpu cores Kraken, NICS Oak Ridge, TN (Cray ) Hopper, NERSC, Berkley, CA (Cray) and if utilised correctly, can support PetaFlop/sec calculations.● The serial codes (1973-1999) could take a week(s) to calculate an ion stage. The first generation of parallel codes (2-3 hundred levels, 1-2 thousand channels) a day● Now we need adapt to (1-3 thousand level calculations with 5-20 thousand channels) if we are to address open p-and-d shell systems.
R-matrix/R-matrix with Pseudostates (RMPS) review
The importance of excited state ionisationEffective ionisation rates include the contribution of excited state ionisation,●which becomes computationally demanding for non-perturbative methodssuch the RMPS Allain et al 2004 Nucl. Fusion 44, 655 (2004) How do we employ the RMPS method effectively, for high n shell ionisation ? The calculation of ionisation from every term of a high n can be an order of magnitude more computationally demanding than the groundstate alone.
There are many non-perturbativeionisation cross-sections from the groundstate For example, consider the closed shell groundstate of neutral neon Groundstate Excited terms
However systems with several valence electrons soon become problematic Consider the all LS coupling electron-impact ionisation (both ground and excited) state from a boron-like system such as B I / C II This will require ionisation from : 1s^2 2s^2 nl (where n=2-4, l=0-3) : 1s^2 2s 2p^2 : 1s^2 2p^3 (for C II ) In addition to the above spectroscopic terms, we shall require minimum pseudostate expansions of the form: 1. 1s^2 2s^2 nl (where n=5,14,l=0-6) 2. 1s^2 2s 2p nl (where n=5,14,l=0-6) 3. 1s^2 2p^2 nl (where n=5,14,l=0-6) If you want to calculate a) Direct ionisation of the outer shell electron b) Direct ionisation of the 2s electron c) All the excitation-autoionisations from every term ie. e + 1s^2 2s^2 3s --> 1s^2 2s2p 3s Well, 1444 terms , approximately 5000 close-coupled channels and 5 Tb of Hamiltonian matrices requiring diagonalisation poses an interesting challenge ...
This represents the current work, that extends beyond naively splitting the serial problem over more processors, to one in which the parallel code adapts to a particular problem Hamiltonian formation1. Serial : Each partial wave is calculated consecutively ( 50-100 ) .... a month2. Naive parallelism : each partial wave is carried out concurrently .... 3 days (remember a single partial wave > 200 Gbs) .... 100 procs3. Adaptive parallelism : As well as each partial wave being carried out concurrently the target terms are grouped into their L S Pi groups (perhaps 20-40 unique groups) .... 2000-4000 procs … 4hrs RESULT : Hamiltonian formation is reduced scattering from a set of target terms with the same L S Pi values. Hamiltonian diagonalisation 1.Serial : Impossible ! Every eigenvalue of a 200 K by 200 K Hamiltonian 2. Naive parallelism : sequential parallel diagonalisation using Scalapack, possible, but regardless of diagonalisation time, you must read 5 Tb .... 4 days 3.Adaptive parallelism : Each Hamiltonian is concurrently diagonalised in parallel , with an n^3 scaling law controlling the distribution of processors … 5 hrs
RESULT : Adaptive diagonalisation ---> 1 Hamiltonian read, 1 diagonalisationBetter load balancing as processing power is distributed to where it is needed
Electron-impact ionisation of neutral Boron , n=3 shell Notice: large excitation-autoionisation (EA) i.e. e + 1s^2 3l --> 1s^2 2s 2p 3l + e 3d 3p 3sLarge EA destroys any n^4 scaling law (or rescaling of an empirical formula) neededto extrapolate to higher n shell ..... solution ?
If we can explicitly caculate to high enough n shell, that the direct ionisation completely dominates over EA , then we can scale simple ionisation expressions from the last explicitly calculated n shell shell of the RMPS Below , we have statistically averaged the term resolved RMPS results used to rescale an expression, such as the Burgess-Vriens for the higher n shells. n=4B +B n=4 2+B n=5
Excited state ionisation from higher charged ions Higher partial waves begin to dominate the cross section, for high n , multiply charged systems Convergence is very slow Comparisons with distorted wave are favourable, again RMPS becomes computational Demanding, requiring 110-130 pseudo-orbitals ranging from n=6-16,18 and l=0-10 to achieve convergence Time : distorted wave (15 mins) RMPS (4 hrs) Distorted wave (Younger Potential) Distorted Wave (Macek Potential) RMPS
Electron-impact excitationand how we adapt to futurechallenges.
Scripted Semi-relativistic ICFT (Intermediate Coupling Frame Transformation) R-matrix calculations ● Intermediate Coupling Frame Transformation, developed by D C Griffin is an R-matrix approach primarily carried out in LS coupling but including mass-velocity and Darwin terms, that transforms term-resolved K-matrices into level-level K-matrices, and ultimately level resolved excitation rates. Provides comparable cross sections to the Breit-Pauli R-matrix codes. ● A perl-script has been developed that automatically takes structure calculations for atomic ions Z < 36 , through to effective collision strengths for ALL ions along an iso-electronic sequence Example Papers: G. Y. Liang et al Astron. Astrophys. 528 A69(15) (2011). Li-like sequence G. Y. Liang et al Astron. Astrophys. 518 A64(20) (2010). Neon sequence G. Y. Liang et al J. Phys. B 42 225002(12) (2009). Na-like sequence M. C. Witthoeft et al J. Phys. B 40 2969-93 (2007). Fl-like sequence Typically 100-300 levels calculations, with 1000-1500 channels
However, eventually as Z increases we must adopt the a parallel version of relativistic R-matrix codes. DARC (Dirac Atomic R-matrix Code)● These codes have been modified in an analguous way to the non-relativistic codes - distribution of integral generation - multi-layered parallelism of the Hamiltonian formation - Concurrent adaptive diagonalisation of the Hamiltonian Excitation/ionisation of Mo II 0.6 billion Racah coefficients per symmetry 4d^5,4d^4 4f-5f This is preparation 8000-10000 channels calculation for future still a challenge W ion stages.
e.g. DARC calculations for Fe III Target : 3d^6, 3d^54s, 3d^54p, 3p^43d^8, 3p^53d^7 (2-3 hundred level) ideally now we would add (3p^54d, 3p^43d^74s ,3p^43d^74p) which if we keep to 0.0-0.5 Ryds, provides a satisfactory description of Fe III
e.g. DARC calculations for Fe II Before After(Ith -Iobs)/Iobs Collision strengths : Zhang 1992 Collision stengths : parallel DARC A-values : Nahar and Pradhan A-values : HFR (cowan)
ICFT DARCA typical collision strength within the groundstate complex of Fe IIIillustrates the need for a very fine energy mesh !
However, if we want continued scaling to 5-12 thousand close-coupled channels, reconsider electron-impact ionisation of B I * 1444 terms * 5000 channels , * Hamiltionian Matrices close to 200,000 by 200, 000 in size * over 1 million possible transitions But the formation of the R-matrix is crippling in both memory and the time required! .... cannot be left to a single processor W ik * W kj 200 000 * 5000 * 8 * 2 R ij = k E-E = 16 Gb ! (takes mins ) k0.2 Gb N slices w ik / E - E k w = kj 5000*5000*8
We can also adapt the distribution of processors to energy points in the outer region Test run : 4 partial waves Se III (225 levels): 437,737,748,1017 channels 12,000 pts . CPU TIME= 1.380 MIN -- processors=: 250 sub world 0 CPU TIME= 3.851 MIN -- processors=: 250 sub world 1 CPU TIME= 3.990 MIN -- processors=: 250 sub world 2 CPU TIME= 6.496 MIN -- processors=: 250 sub world 3 Suggested Proc distribution: 89 245 253 413 CPU TIME= 3.563 MIN -- processors=: 245 sub world 1 CPU TIME= 3.868 MIN -- processors=: 413 sub world 3 CPU TIME= 3.869 MIN -- processors=: 89 sub world 0 CPU TIME= 3.977 MIN -- processors=: 253 sub world 2 ------------------------------------------------------------- Another 2.5 mins saved .... better balance of resources RECAP
An overview of the parallel DARC dipole photoionisation codes
An overview of the parallel DARC dipole photoionisation codes● The parallel dipole suite of codes, benefit from the changes made from excitation All the eigenvectors from a pair of dipole allowed symmetries are required for bound-free dipole matrix formationThe current approach ensures that every dipole matrix pair is carried out concurrently●with groups of processors assigned to an individual dipole. RESULT : ALL photoionisation, dielectronic-recombination or radiative-damped excitation takes the time for a single dipole formation● The capacity to perform photoionisations calculations with over 500 levels, improves the residual ion structure, the ionisation potential and resonance structure associated with over 3500 channels.As the code scales to 100,000 processors , we can have a resolution of 10^(-8) Rydbergs●ie (6-30 million pt) in the incident photon energy, which is vital when comparing withALS measurements at 4,6,9 meV.● Of course, theoretical calaculations can guide experimental measurement and determine metastable fractions in experimental measurement.
Example : Groundstate Photoionisation of Ca II (work in progress)Target : 3p^6, 3p^5[4s-5p] 3p^4 3d 4s hv + Ca II (^2 S) 3p^64s 3p^4 3d 4p 3P^4 3d 4d which gives rise to 513 levels Experiment : Lyon (1987) ALS(2010) Theoretical : DARC R-matrix Currently, the theoretical model is only from the groundstate, with metastable calculations ongoing.
Example : Photoionisation of Kr II and Xe II● Used a 326 level model for the residual ion in both cases● Target configurations for K III include : 4s^2 4p^4 4s 4p^5 4p^6 4s^2 4p^2 4d^2 4s^2 4p^3 4d 4s 4p^4 4d 4p^4 4d^2 ● Target configurations for Xe III (exactly the same : switching n=4 to 5) ● In the following graphs the R-matrix results have been statistically averaged over the initial p^5 levels