We take a closer look at the generation process of the SDOBenchmark. For more information about the machine learning dataset "SDOBenchmark", go check out http://i4ds.github.io/SDOBenchmark and https://www.kaggle.com/fhnw-i4ds/sdobenchmark

1. SDOBenchmark
HTTP://I4DS.GITHUB.IO/SDOBENCHMARK
HTTPS://WWW.KAGGLE.COM/FHNW-I4DS/SDOBENCHMARK

3. At its core, SDOBenchmark is an image dataset
tailored towards Machine Learning
AIA 171 AIA 1700 HMI magnetogram
Can we predict that a large X9 flare happens in 24h?

4. At its core, SDOBenchmark is an image dataset
tailored towards Machine Learning
Up to 40 images…
… for a single
prediction

5. At its core, SDOBenchmark is an image dataset
tailored towards Machine Learning
12h 5h 1.5h 10min 24h prediction

6. At its core, SDOBenchmark is an image dataset
tailored towards Machine Learning
AIA
HMI

7. What makes this dataset special?
1. Built for Machine Learners without a solar physics background
2. High accessibility and high scientific quality
3. Specifically engineered to avoid common overfitting issues
4. Public, open source: Website, Kaggle

8. Intermediate results
https://i4ds.github.io/SDOBenchmark/#current-state
15 June 2018

9. «First competitive model» (TSS 0.34)
Would the model have predicted the large flare of September 2017?
Peak class Predicted
X2 M5
X9 M5
Result: No. It would have predicted M instead of a strong X.

10. Intermediate results
Featured Dataset
15 June 2018

11. Dataset creation
PIPELINE FROM RAW DATA TO THE IMAGE DATASET

12. Pipeline overview
Raw data
download
Sample
selection
Sample
creation

13. Raw data
3 different sources of raw data:
• SWPC and SSW Latest Events from HEK
-> sample selection
• GOES profiles (X-ray measurements)
-> prediction label (peak_flux), sample selection
• AIA and HMI FITS
-> images
Raw data
download
Sample
selection
Sample
creation

14. Raw data: HEK events
Events SWPC and SSW latest events
SWPC = manual flare labels
SSW = backup for missing or inconsistent SWPC events
Raw data
download
Sample
selection
Sample
creation
Download
[{"hpc_bbox": "POLYGON((-84.402 202.566,84.402 202.566,84.3972 202.911,-84.3972 202.911,-84.402 202.566))",
"hpc_coord": "POINT(0 202.9314)", "event_starttime": "2012-01-01T00:00:00", "event_type": "AR", "intensmin":
null, "obs_meanwavel": 5e-05, "intensmax": null, "intensmedian": null, "obs_channelid": "visible",
"ar_noaaclass": "", "frm_name": "NOAA SWPC Observer", "obs_observatory": "various", "frm_daterun": "2012-04-
12T23:33:57", "hpc_y": 202.9314, "hpc_x": 0, "kb_archivdate": "2012-04-13T00:49:37", "ar_noaanum": 11390, …

15. Raw data: GOES profile
X-ray curves
GOES15 satellite
Raw data
download
Sample
selection
Sample
creation
Download
TIME_TAG, A_FLUX
2012-02-02 00:00:51.577, 1.0000E-09
2012-02-02 00:00:53.623, 1.0000E-09
2012-02-02 00:00:55.670, 1.0000E-09
2012-02-02 00:00:57.720, 1.0000E-09
2012-02-02 00:00:59.767, 1.0000E-09
2012-02-02 00:01:01.817, 1.0000E-09

16. Raw data: FITS files
Not yet:
Once we have the samples, we can download the FITS raw data.
Raw data
download
Sample
selection
Sample
creation

17. Sample selection: Definition
4 time steps -> 24h prediction period
12h 5h 1.5h 10min peak flux?
Constraints
• No overlaps
• Avoid Active Region overfitting
Raw data
download
Sample
selection
Sample
creation

18. Sample selection: Event Processing
Raw data
download
Sample
selection
Sample
creation
01.09. 02.09. 03.09. 04.09. 05.09. 06.09. 07.09. 08.09. 09.09. ...10.09. 11.09. 12.09. 13.09.
Each Active Region is split into ranges.
Range:
• start time
• end time
• largest flare (if any)
X8 (40h)
X1 (48h)
X2 (48h)
X9 (48h)
M1-9

19. Sample selection: Ranges -> Samples
Raw data
download
Sample
selection
Sample
creation
01.09. 02.09. 03.09. 04.09. 05.09. 06.09. 07.09. 08.09. 09.09. ...10.09. 11.09. 12.09. 13.09.
From each range 1 or 2 samples are created
Sample:
• start time
• end time
• largest flare in 24h (if any)
X8 (40h)
X2 (24h)
X9 (24h)
X1 (24h)
= sample input
= prediction period
X9 (24h)
X1 (24h)

20. Sample selection: Test/Training
Active Regions are split into a test and a training set
Lastly, samples go through plent of verification and validation.
Raw data
download
Sample
selection
Sample
creation

21. Output creation
Finally, the actual samples are created in three steps:
1. Request FITS data urls
2. Download FITS raw files
3. Process FITS files to create output images
Raw data
download
Sample
selection
Sample
creation

22. Output creation: FITS download
FITS = Image raw data
AIA and HMI FITS files
from JSOC with their Python REST client «Drms»
Download
query = f"hmi.Ic_45s[{qt:%Y.%m.%d_%H:%M:%S_TAI}]{magnetogram}“
client.export(query, method="url_quick", protocol="as-is")
Raw data
download
Sample
selection
Sample
creation

23. Output creation: Processing
For each sample, time step, wavelength:
1. Load the FITS file with Sunpy
2. Run aiaprep / hmiprep:
Rotates, scales and translates the image
3. Find the Active Region center
4. Crop out a square around it
5. Replace NaNs with 0
6. Clip and rescale image values to predefined ranges
(similar to helioviewer.org ranges)
7. Flag images whose FITS raw files are flagged (elipses, maintenance, etc.)
8. Save resulting JPEG in the sample folder
(8-bit, 256px from a 512 cropout)
Raw data
download
Sample
selection
Sample
creation
FITS
Image

24. Output creation: Processing
For each sample, time step, wavelength:
1. Load the FITS file with Sunpy
Raw data
download
Sample
selection
Sample
creation
FITS
Image
current_map = sunpy.map.Map(fits_file)

25. Output creation: Processing
For each sample, time step, wavelength:
2. Run aiaprep / hmiprep:
Rotates, scales and translates the image
Raw data
download
Sample
selection
Sample
creation
FITS
Image
if isinstance(current_map, sunpy.map.sources.AIAMap):
current_map = sunpy.instr.aia.aiaprep(current_map)
else:
hmi_scale_factor = current_map.scale.axis1 / (0.6 * u.arcsec)
current_map = current_map.rotate(recenter=True,
scale=hmi_scale_factor.value, missing=0.0)

26. Output creation: Processing
For each sample, time step, wavelength:
3. Find the Active Region center
Raw data
download
Sample
selection
Sample
creation
FITS
Imageregion_position_rotated = sunpy.physics.differential_rotation.solar_rotate_coordinate(
active_region_position,
observation_date
)
region_position = astropy.coordinates.SkyCoord(
float(closest_region_event["hpc_x"]) * u.arcsec,
float(closest_region_event["hpc_y"]) * u.arcsec,
frame="helioprojective",
obstime=closest_region_event["starttime"]
)
center_x, center_y = current_map.world_to_pixel(region_position_rotated)
a) Closest AR event HEK
b) Solar rotation  sample date
c) Hpc  pixel coordinates

27. Output creation: Processing
For each sample, time step, wavelength:
4. Crop out a square around the center
Raw data
download
Sample
selection
Sample
creation
FITS
Image
(3144, 1536)

28. Output creation: Processing
For each sample, time step, wavelength:
5. Replace NaNs with 0
6. Clip and rescale image values to predefined ranges
(similar to helioviewer.org ranges)
Raw data
download
Sample
selection
Sample
creation
FITS
Image
…
},
"171": {
'dataMin': 5,
'dataMax': 3500,
'dataScalingType': 3 # 0 - linear, 1 - sqrt, 3 - log10
},
"193": {
…

29. Output creation: Processing
For each sample, time step, wavelength:
7. If FITS file is flagged (eclipses, maintenance, etc.):
Add “flagged” to image meta data
8. Save resulting JPEG in the sample folder
(8-bit, 256px from a 512px crop out)
Raw data
download
Sample
selection
Sample
creation
FITS
Image

30. Output creation: Result
Raw data
download
Sample
selection
Sample
creation

31. Output creation: Result
Raw data
download
Sample
selection
Sample
creation
40 images
(4 time steps, 10 channels)
2012_01_01_19_06_00_0
RangeStart_SampleNr
Noaa_num

32. Want to know more?
• SDOBenchmark website at i4ds.github.io/SDOBenchmark
• GitHub repository at https://github.com/i4Ds/SDOBenchmark,
with publicly available source code and more information about the dataset creation process at
https://github.com/i4Ds/SDOBenchmark/blob/master/STRUCTURE.md
• Kaggle SDOBenchmark dataset at https://www.kaggle.com/fhnw-i4ds/sdobenchmark