1) Researchers are developing an ontology called CMR-QA to standardize the quality assessment of cardiac magnetic resonance images from the UK Biobank study. 2) Currently, quality assessment is done through free-text comments which are variable and ambiguous. The ontology aims to reduce this variability by providing a controlled vocabulary. 3) It will represent quality issues as ontology individuals connected to image scans, and include rules to infer new knowledge and support improved search and retrieval of scans.

1. Towards the Creation
of the Cardiovascular
Magnetic Resonance
Quality Assessment
Ontology (CMR-QA)
Ernesto Jim´enez-Ruiz, Valentina Carapella,
Elena Lukaschuk, Nay Aung, Kenneth Fung,
Jose Paiva, Mihir Sanghvi, Stefan Neubauer,
Steffen Petersen, Ian Horrocks, Stefan Piechnik
SWAT4LS Conference, Amsterdam, Dec. 7, 2016 1 / 18

2. Outline
Background
Motivating Examples
Semantic Layer: running example
Conclusions and Future work
2 / 18

3. Background: UK biobank
UK Biobank is a large scale population study that started in
2006.
500,000 volunteers aged 40-69 across UK.
Included a series of clinical tests.
Different clinical imaging modalities are also included in the
protocol applied to the volunteers.
Cardiac Magnetic Resonance Imaging (CMR) sequences:
cine-MRI, tag-MRI, T1-mapping and blood flow imaging.
(S. E. Petersen et al 2013) Imaging in population science:
cardiovascular magnetic resonance in 100,000 participants of UK
Biobank - rationale, challenges and approaches. Journal of
Cardiovascular Magnetic Resonance, 15(1):110, 2013.
(http://www.ukbiobank.ac.uk/)
3 / 18

4. Background: UK biobank
UK Biobank is a large scale population study that started in
2006.
500,000 volunteers aged 40-69 across UK.
Included a series of clinical tests.
Different clinical imaging modalities are also included in the
protocol applied to the volunteers.
Cardiac Magnetic Resonance Imaging (CMR) sequences:
cine-MRI, tag-MRI, T1-mapping and blood flow imaging.
(S. E. Petersen et al 2013) Imaging in population science:
cardiovascular magnetic resonance in 100,000 participants of UK
Biobank - rationale, challenges and approaches. Journal of
Cardiovascular Magnetic Resonance, 15(1):110, 2013.
(http://www.ukbiobank.ac.uk/)
3 / 18

5. Background: 5,000 CMR pilot study
Context: Data analysis (DA) of a specific UK-Biobank pilot
study of 5,000 CMR scans limited to cine-MRI.
DA was restricted to the following Cine MRI views: (i) Short
axis (SA), Horizontal long axis (HLA), and vertical long axis
(VLA).
DA was carried out by a team of eight observers from two
clinical research centres.
Quality assessment (QA) was carried out through a combination
of free-text comments and numerical quality scores. managed
through a shared spreadsheet by the team
Target: enhance the QA using Semantic Web technology.
V. Carapella et al 2016) Towards the Semantic Enrichment of
Freetext Annotation of Image Quality Assessment for UK Biobank
Cardiac Cine MRI Scans. In MICCAI-LABELS Workshop 2016
4 / 18

6. Background: 5,000 CMR pilot study
Context: Data analysis (DA) of a specific UK-Biobank pilot
study of 5,000 CMR scans limited to cine-MRI.
DA was restricted to the following Cine MRI views: (i) Short
axis (SA), Horizontal long axis (HLA), and vertical long axis
(VLA).
DA was carried out by a team of eight observers from two
clinical research centres.
Quality assessment (QA) was carried out through a combination
of free-text comments and numerical quality scores. managed
through a shared spreadsheet by the team
Target: enhance the QA using Semantic Web technology.
V. Carapella et al 2016) Towards the Semantic Enrichment of
Freetext Annotation of Image Quality Assessment for UK Biobank
Cardiac Cine MRI Scans. In MICCAI-LABELS Workshop 2016
4 / 18

7. Background: analysis pipeline
5 / 18

8. Background: QA typical issues
6 / 18

9. Outline
Background
Motivating Examples
Semantic Layer: running example
Conclusions and Future work
7 / 18

10. Limitations in current pipeline
! Free-text annotations provide a rich source of information
! Beyond the simple classification provided by the numerical
scores
% Free-text is prone to variability and ambiguity
% Hinders the efficient access of Cine MRI scans
8 / 18

11. Limitations in current pipeline
! Free-text annotations provide a rich source of information
! Beyond the simple classification provided by the numerical
scores
% Free-text is prone to variability and ambiguity
% Hinders the efficient access of Cine MRI scans
8 / 18

12. Example 1: variability
Variability for the LA off axis (*) example in numbers.
Synonyms Occurrence
foreshortened 2
off axis 179
off-axis 13
off axis 145
off axes 21
off plane 0
out of axis 26
wrong plane 7
(*) The plane chosen to acquire a long axis view was not optimally aligned to
measure left atrial (LA) volumes
9 / 18

13. Example 2: ambiguity
Ambiguity with HLA/4Ch (*) out of plane (no atrium
specified).
Off axis subset Occurrence
Total off axis sentences 515
Only referring to LA 173
Only referring to RA 27
Reference to both atria 53
HLA (4Ch) without reference 226
Other annotations 36
(*) In Horizontal Lon Axis (4 Chamber view) both left and the right atria are
contoured.
10 / 18

14. Outline
Background
Motivating Examples
Semantic Layer: running example
Conclusions and Future work
11 / 18

15. From QA comments to ontology data
QA comment associated to Cine MRI “mri”
“basal slice is missing. wrong planes ra/la”
Ontology data
CineMRI Scan(mri)
Lack coverage(issue1)
has technical issue(mri,issue1)
RA offaxis(issue2)
has technical issue(mri,issue2)
LA offaxis(issue3)
has technical issue(mri,issue3)
12 / 18

16. From QA comments to ontology data
QA comment associated to Cine MRI “mri”
“basal slice is missing. wrong planes ra/la”
Ontology data
CineMRI Scan(mri)
Lack coverage(issue1)
has technical issue(mri,issue1)
RA offaxis(issue2)
has technical issue(mri,issue2)
LA offaxis(issue3)
has technical issue(mri,issue3)
12 / 18

17. From QA comments to ontology data
QA comment associated to Cine MRI “mri”
“basal slice is missing. wrong planes ra/la”
Ontology data
CineMRI Scan(mri)
Lack coverage(issue1)
has technical issue(mri,issue1)
RA offaxis(issue2)
has technical issue(mri,issue2)
LA offaxis(issue3)
has technical issue(mri,issue3)
12 / 18

18. Controlled vocabulary: the ontology
13 / 18

19. Inferring new knowledge: the ontology rules
RA off-axis are always detected in the Horizontal long axis
(HLA) view:
has issue in HLA view(mri, issue) ←←← CineMRI Scan(mri) ∧∧∧
has technical issue(mri, issue) ∧∧∧ RA offaxis(issue)
Lack of coverage will always lead to a quality score associated
to the right and left ventricle equal or greater than 2 (e.g.,
suboptimal or unreliable):
has RV quality score(mri, 2) ←←← CineMRI Scan(mri) ∧∧∧
has technical issue(mri, issue) ∧∧∧ Lack coverage(issue)
LA off axis is incomplete if the cardiac cycle is not indicated:
IncompleteIssueDefinition(issue) ←←←
RA offaxis(issue) ∧∧∧ not observed in cardiac cycle(issue, cc)
14 / 18

20. Inferring new knowledge: the ontology rules
RA off-axis are always detected in the Horizontal long axis
(HLA) view:
has issue in HLA view(mri, issue) ←←← CineMRI Scan(mri) ∧∧∧
has technical issue(mri, issue) ∧∧∧ RA offaxis(issue)
Lack of coverage will always lead to a quality score associated
to the right and left ventricle equal or greater than 2 (e.g.,
suboptimal or unreliable):
has RV quality score(mri, 2) ←←← CineMRI Scan(mri) ∧∧∧
has technical issue(mri, issue) ∧∧∧ Lack coverage(issue)
LA off axis is incomplete if the cardiac cycle is not indicated:
IncompleteIssueDefinition(issue) ←←←
RA offaxis(issue) ∧∧∧ not observed in cardiac cycle(issue, cc)
14 / 18

21. Inferring new knowledge: the ontology rules
RA off-axis are always detected in the Horizontal long axis
(HLA) view:
has issue in HLA view(mri, issue) ←←← CineMRI Scan(mri) ∧∧∧
has technical issue(mri, issue) ∧∧∧ RA offaxis(issue)
Lack of coverage will always lead to a quality score associated
to the right and left ventricle equal or greater than 2 (e.g.,
suboptimal or unreliable):
has RV quality score(mri, 2) ←←← CineMRI Scan(mri) ∧∧∧
has technical issue(mri, issue) ∧∧∧ Lack coverage(issue)
LA off axis is incomplete if the cardiac cycle is not indicated:
IncompleteIssueDefinition(issue) ←←←
RA offaxis(issue) ∧∧∧ not observed in cardiac cycle(issue, cc)
14 / 18

22. Outline
Background
Motivating Examples
Semantic Layer: running example
Conclusions and Future work
15 / 18

23. Expected benefits
Development of training material for future observers
analysing clinical imaging data.
CMR-QA will provide a controlled set of preferred quality
assessment comments.
Reduction of variability and ambiguity
Improved quality of the information content
Ease the retrieval of Cine MRI scans according to their QA by
Uk Biobank target users:
A clinical researcher interested in building a control group for a
study on pathologies affecting the atria, might want to query
only for those images whose atrial volumes have been reliably
estimated.
A biomedical engineer expert in development of algorithms for
automatic contouring of the left ventricle might be interested in
testing a newly developed tool on those Cine MRI scans where
it was most difficult to define the basal slice.
16 / 18

24. Expected benefits
Development of training material for future observers
analysing clinical imaging data.
CMR-QA will provide a controlled set of preferred quality
assessment comments.
Reduction of variability and ambiguity
Improved quality of the information content
Ease the retrieval of Cine MRI scans according to their QA by
Uk Biobank target users:
A clinical researcher interested in building a control group for a
study on pathologies affecting the atria, might want to query
only for those images whose atrial volumes have been reliably
estimated.
A biomedical engineer expert in development of algorithms for
automatic contouring of the left ventricle might be interested in
testing a newly developed tool on those Cine MRI scans where
it was most difficult to define the basal slice.
16 / 18

25. Future work
Complete the CMR-QA ontology and define the necessary
ontology rules
Finalise the implementation of the techniques to text mine the
comments to extract ontology data.
Evaluation to analyse the correctness of our approach.
Design a prototype software for query and retrieval of Cine MRI
scans
Integration at different levels within UK Biobank and with
external vocabularies.
17 / 18

26. Future work
Complete the CMR-QA ontology and define the necessary
ontology rules
Finalise the implementation of the techniques to text mine the
comments to extract ontology data.
Evaluation to analyse the correctness of our approach.
Design a prototype software for query and retrieval of Cine MRI
scans
Integration at different levels within UK Biobank and with
external vocabularies.
17 / 18

27. Questions?
Thank you for your attention
ernesto.jimenez.ruiz@gmail.com
ernestoj@ifi.uio.no
vcarapella@gmail.com
Acknowledgements:
UK Biobank
EPSRC project DBOnto.
18 / 18