1.
Identifying ontology authoring
strategies and patterns
Markel Vigo★, Caroline Jay, Robert Stevens
University of Manchester (UK)
WhatIf: Answering “What if...” questions for Ontology Authoring.
EPSRC reference EP/J014176/1
@markelvigo
markel.vigo@manchester.ac.uk
www.markelvigo.info
★
UK Ontology Meeting, UKON 2014
April 24, Birmingham

2.
Identifying ontology authoring strategies and patterns UKON 2014
Goals
• Find authoring patterns
• Identify the challenges of the authoring process
• Inform authoring tools
• Support for whatif questions

3.
Exploratory study
• Interview with 15 ontology authors
• Strategies inform a set of design recommendations
- Sensemaking, exploration, searching
- Building
- Debugging and evaluation
More info:
M. Vigo, C. Jay, R. Stevens (2014) Design insights for the next wave ontology authoring tools.
Proceedings of ACM SIGCHI Conference on Human Factors, CHI‟ 14
Identifying ontology authoring strategies and patterns UKON 2014

4.
Confirmatory study
• Protégé4US: modified Protégé to generate log data
of authoring and interaction events
More info:
M. Vigo, C. Jay, R. Stevens (2014) Protégé4US: Harvesting Ontology Authoring Data with Protégé.
Workshop on Human-Semantic Web Interaction, HSWI‟ 14
1389972276585,Classes,Element edited,Juliette subclass of: Potato and hasCroppingTime some „Main cropping‟
1389972277786,Classes,Save ontology,http://owl.cs.manchester.ac.uk/ontology/start-here.owl
1389972280204,Classes,Reasoner invoked,HermiT 1.3.8
1389972280647,Classes,Mouse entered,Class hierarchy (inferred)
1389972282910,Classes,Element hovered,Early_cropping_potato
1389972283049,Classes,Element selected,Early_cropping_potato
1389972283661,Classes,Hierarchy expanded,Early_cropping_potato
• 15 users, 3 authoring tasks with Protégé4US
- Correlations bw usability metrics and authoring events
- Profiling users by their use of Protégé tab
Identifying ontology authoring strategies and patterns UKON 2014

5.
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0 21.264
34.687
35.551
36.646 38.051 44.065
44.576
45.699 46.993
49.639
49.641
52.595 67.301
82.323 83.856 84.346 87.534 100.204
100.206
103.323
103.399
106.425
106.973
109.182
111.445
111.551
113.586
114.127
115.684
120.664
125.186
125.189 132.665 134.922 136.838
136.911
137.917
138.275
140.152
145.648 146.456 147.656148.151 150.318 151.474 152.08 152.84153.224
153.628
154.191154.609 155.177155.504 162.584 166.837
167.054
175.037
176.486
179.565
179.637
181.845 183.762 187.775
188.456
191.224
202.23 203.021
203.142
204.069
206.05
208.413208.831 210.559210.999 219.79 222.062
222.215
228.172 229.549
229.669
230.687 232.188
232.316
235.78
235.892
270.412
270.72
271.3
272.351
272.906
273.225
274.187
274.73
279.773
281.91
285.304 286.691
286.796
297.471
298.322 300.944 301.924302.26
302.35
324.924
325.819 327.548
327.669
337.77
338.609 339.515
339.637
350.104
350.906 352.148
352.228
366.415
367.798 368.619
368.717
380.316
383.506 384.363
384.453
391.367
392.305 394.668
394.781
403.081
403.721 404.531
404.646
416.546
418.278 419.131
419.22
427.324
429.048 429.788
429.885
436.924
436.928 438.657 439.467
439.571
446.157
448.583
451.182
451.226
455.662 457.157 464.376
464.465
477.697
477.788
477.855
477.856
477.859477.861
477.862
501.441
509.581
530.629
530.647 531.514
531.644
547.333
552.251
553.993
554.032
558.619558.636
558.638
568.501
568.504 571.181
579.638
595.037
595.043 596.802
596.891
614.57
620.325 621.657
621.762
632.493
632.496 633.954
634.043
641.751
644.4
654.612
654.618 655.553
655.649
665.133
667.577
667.601
669.348
723.762
726.698726.718
726.722
726.734
726.735
728.211
728.226
728.808
730.139
731.803 734.17
734.172
734.47
734.472 740.49740.843
745.42
746.021 751.308
751.309
752.368
752.458
758.922
759.776
759.89
764.039
766.225 767.004
768.003
771.513
772.139 776.154
776.251
785.746
786.447
788.827 789.616
789.715
798.897
799.622 800.305
800.403
810.839
812.641 815.593
815.691
817.17
820.066820.337 821.097
821.203
824.886
827.762 829.153
829.25
832.114
836.033 837.417
837.523
840.014
844.528844.986 845.968
846.059
852.208
858.938 859.544
859.658
861.219
871.865 873.681
873.778
878.309
880.936 881.696
881.794
883.677
891.505 892.225
892.32
905.493
906.593 907.465
907.579
908.916
909.684 911.161 912.811913.465913.785913.984914.193914.373914.561914.741 915.647916.046916.335916.603916.873 917.391 930.919
938.143
938.148 939.408
939.602
958.379
959.901
969.248
969.253 970.423
970.513
974.682
975.741
983.088
983.093 983.839
983.944
989.611
990.634
1000.465
1000.471 1003.015
1003.119
1008.958
1011.7111011.728
1011.729
1020.63
1020.712
1025.826
1025.828
1029.714 1031.705
1031.834
1035.479
1035.591
1045.947
1045.981
1047.675 1049.002
1049.096
1066.386
1070.024
1070.085
1081.442
1098.18
1098.184 1099.39
1099.497
1141.238
1141.261 1141.942
1142.031
1157.55
1163.542
1163.561 1169.568 1170.326
1170.431
1180.857
1183.493
1183.54
1199.855
1199.951
1224.136
1224.246
1236.095
1248.741
1248.838
1276.551 1277.462 1285.101 1286.398 1287.544 1290.303
1292.199
1293.031
1294.438
1295.803
1295.825
1305.064 1306.3571306.861 1308.3891308.853 1309.613 1311.549 1312.933 1323.743 1324.597 1325.989
1326.117
1327.802
1328.566
1330.146
1333.773
1333.86
1340.543
1350.399
1350.444
1352.509 1357.623
1357.718
1363.158
1363.238
1365.694
1365.765
1366.653
1366.741
1369.246
1370.495
1371.213
1374.968 1390.928
1391.487 1435.484
1435.589
1454.203
1454.307
1455.052
1455.148
1490.778
1490.804
1490.821 1493.2031493.587
1493.676
1533.912 1540.749 1545.651 1548.341 1548.878 1549.71550.061550.331
1550.437
1552.532
1553.542
1554.556
1554.91
1555.78
1556.933
1561.052
1561.148
1561.209
1561.21
1561.2131561.2141561.215
1561.216
1602.235
1603.755
1603.853
1608.143
1614.105
1614.133
1614.14 1621.126
1621.237
1648.868
1649.909
1650.875
1652.528
1652.57
1656.629
1656.717
1675.981
1676.6351676.9811677.58
1679.453
1679.956 1681.605 1682.988 1683.62 1688.557 1691.995
1692.084
1693.996
1695.86 1700.442 1703.154 1704.586 1706.348 1708.466
1710.6
1713.762
1713.806
1716.26
1720.413
1721.4221721.844 1724.109 1726.789 1728.301 1729.581 1733.421734.02 1734.597 1736.532 1738.38 1740.13 1741.986 1743.157 1743.756 1746.6031746.932 1747.994
1748.266
1753.611 1755.442 1756.313 1758.257 1760.818
1762.494
1765.121
1767.729
1767.774
1768.941 1770.171
1770.274
1778.868
1784.538
1784.573
1787.536
1787.582
1789.426
1789.515
1804.897 1806.5511806.657
1806.668
1807.616
1807.748
1809.718 1810.166
1810.318
1817.786 1820.207
1820.326
1821.289 1823.041 1824.592
1824.701
1827.305
1827.341
1829.649 1832.345
1889.938
1891.007 1894.817 1896.024 1896.663 1900.367
1900.456
1901.458
1902.3591902.7351903.079
1904.032
1905.948 1907.72
1909.9371909.942
1909.9431910.216
1913.804
1913.844
1972.568
1973.879
1974.495
1975.302
1976.861
1976.884
1980.471 1981.2231981.807 1982.365 1983.167 1983.9191984.422 1992.1831992.5341993.119
1994.44
1994.461
1995.832 2000.303 2007.608 2010.087 2011.143 2012.201
2014.109 2015.096
2015.6322016.167 2017.7582018.0942018.574 2019.1512019.698 2021.273 2024.286 2031.399 2039.847
2051.576
2051.581
2052.253
2079.325
2082.259
2082.287
2086.334
2102.108
2102.113
2103.046
2116.132
2133.83
2133.843
2134.398
2135.094
2143.888
2143.892
2152.317
2152.323
2152.324
2157.779
2157.783
2163.069
2163.072
2168.756
2168.762
2173.602
2173.607
2177.034
2177.04 2186.788
2193.884
2193.885
2193.887
2196.455
2196.477 2197.678 2198.478
2199.549
2203.248
2204.421
2210.345 2212.835
2214.365 2217.094
2219.094
2232.35
2233.34
2235.782
2253.592
2255.892
2256.965
2280.924
2284.288
2289.205
2302.943
2306.382
2306.439
2306.44
2306.4422306.4442306.445
2306.446
2311.018
2313.815
2315.189
2327.377
2329.5082329.821
2334.557
2348.003
2350.355
2352.316
2352.401
2352.403
2367.508
2369.507
2372.629
2380.229
2381.202
2383.307
2394.977
2395.857
2398.412
2412.207
2413.36 2417.82 2421.196
2422.508
2425.307
2426.103
2426.877
2441.359
2442.072
2442.772
2453.575
2462.484 2463.924
2465.531
2493.033
2493.039
2494.556
2525.924
2528.422
2528.548
2534.723
2561.769
2561.774 2562.795
2564.533
2566.814
2567.867
2582.893
2590.819
2590.825
2590.827
2592.538
2592.568
2593.522
2602.373
2603.891
2610.626
2610.631
2611.179
2619.301
2632.949
2632.954
2632.955
2636.51
2636.542
2638.106
2647.725
2649.691
2649.774
2685.651
2689.102
2689.181
2697.171 2713.545 2714.919 2716.09 2717.618
2718.9212719.441
2719.447
2720.826
2723.282 2726.868 2739.9212740.425
2751.0762751.081
●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●
●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●
●
●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●
●●●●●●● ●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●
●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●● ●●●●●●●●
●●●
●●
●●●●● ●●●●
●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●● ●●
●●●●●●●●●
●●●●●●●●●●●●●
● ●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
● ●●
●●●●●●●●●
●●●●●●●●●●●
● ●●
●●●●●●●●
●●●●●●●
● ●●
●●●●●●●●●
●●●●●
● ●●
●●●●●●●
●●●●●●●●●
● ●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
● ●●●●●
●●●
●●●●●
● ●●
●●●●●●●●
●●●●●●●●
● ●●
●●●●●●●●●
●●●●
● ●●●●●●
●●●●●●●●●●●●●●●●
● ●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●● ●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●● ●●
●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●
●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●● ●●●●
● ●●
●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●
●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
● ●●
●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●
● ●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●● ●●●●●●
● ●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●● ●●●●●●●●● ●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●
●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●●●●●●●●●●●●●●●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
● ●●●●●●● ●
●●●●
●●●●
● ●
●●●●●●●
●●●●●●
●●
●●●●●●●
●●●●●
●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●
●●●●●●●●
●
●●● ●●●●●●●●●
●●●●
●●
●●●●●●●●● ●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●●
●●●
●●
●●●●●●●●●
●●●●●●
● ●
●●●●●
●●●●●●●●
●●
●●●●●●●●●●
● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●
● ●●●●
●●●●●●●●●
●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●
●●●●●●● ●●●●
● ●
●●●●●●●●●●●●●●●
●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●● ●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●● ●●●●●
●●●●● ●●●●●●●●●●●●●●●●●●●●●
● ●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●
●●●
● ●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●● ●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●
● ●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●● ●
●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●● ●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●●●●
●●●●●●●●●●●
●●
●●●●
●●●● ●●●●●●●●●●●●●●● ●
●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●
● ●●●●●●●●●●●●●
●●
● ●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●
●●●●●●
●●●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●● ●●● ●●
●●●●●●●
● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●
●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●
●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●● ●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●●●
●●●●
●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●●●●
●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
● ●
●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●
●●● ●●●●●●●●●●● ●●●●●●●●●● ●●●●●●●●●●●● ●●●●●●●●●●● ●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●
●●●●●●●●●●●
●●
●●●●●●●●
●
●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●●●●●
●●●●●●●●●●
●●●●●●●●●●
● ●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●
●●●●●●●●
● ●
●●●●●●●●●●●●●●●●●●●
●●●●●●
● ●
●●●●●●●●●●
● ●
●●●●●●●●●●●●●
●●●
● ●
●●●●●●
●●●●●●●●
● ●
●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●
●●●●●●●
● ●
●●●●●●●●●●●●●●●●●●●
●●
● ●
●●●●●●●●●●●● ●●●●●●●●●●
●●●●●●●●●●●
● ●●●●●●●●●●●●●●●●●●
●
● ●●●●●●●● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●● ●●●●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●●●●●●●●
●●● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●
●
●●●●●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●●●●●●●●●●●●●
●●●●●● ●
● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●
●
●●●●●●●●●●●●●●●
● ●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●
●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●
●●●●●●
Set active ontology
Entity selected
Hierarchy expanded
Entity edited:start
Entity edited:finish
Convert into
defined class:start
Convert into
defined class:finished
Save
Reasoner invoked
Reasoner finished
Class addition
Property addition
Check property
Explanation invoked
Hierarchy collapsed
Undo
Back
Entity renamed
Entity deleted
Entity dragged
Link clicked
0 5:00 10:00 15:00 20:00 25:00 30:00 35:00 40:00 45:00
P15
Confirmatory study II
• Data analysis shows emerging patterns
More info:
M. Vigo, C. Jay, R. Stevens (2014) Protégé4US: Harvesting Ontology Authoring Data with Protégé.
Workshop on Human-Semantic Web Interaction, HSWI‟ 14
Identifying ontology authoring strategies and patterns UKON 2014

6.
Confirmatory study II
• Data analysis shows emerging patternsP15 log
Back
Check property
Class addition
Convert into
defined class:finished
Convert into
defined class:start
Entity renamed
Explanation invoked
Entity deleted
Entity dragged
Entity edited:finishEntity edited:start
Entity selected
Link clicked
Property addition
Reasoner finished
Reasoner invoked
Save
Set active ontology
Hierarchy collapsed
Hierarchy expanded Undo
P9 log
Back
Check property
Class addition
Convert into
defined class:finished
Convert into
defined class:start
Entity renamed
Explanation invoked
Entity deleted
Entity dragged
Entity edited:finishEntity edited:start
Entity selected
Link clicked
Property addition
Reasoner finished
Reasoner invoked
Save
Set active ontology
Hierarchy collapsed
Hierarchy expanded Undo
Identifying ontology authoring strategies and patterns UKON 2014

7.
Confirmatory study II
• Mantel test to compute correlation between matrices (test
used for measuring the genetic distance bw organisms)
p value
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10P11P12P13P14P15
0.00
0.02
0.04
0.06
0.08
0.10
Correlation
P1
P2
P3
P4
P5
P6
P7
P8
P9
P10
P11
P12
P13
P14
P15
P1 P2 P3 P4 P5 P6 P7 P8 P9 P10P11P12P13P14P15
0.0
0.2
0.4
0.6
0.8
1.0
Identifying ontology authoring strategies and patterns UKON 2014

8.
Identifying ontology authoring
strategies and patterns
Markel Vigo★, Caroline Jay, Robert Stevens
University of Manchester (UK)
Datasets and Protégé4US
http://owl.cs.manchester.ac.uk/whatif/
Papers
Design insights for the next wave ontology authoring tools
www.manchester.ac.uk/escholar/uk-ac-man-scw:217528
Protégé4US: Harvesting Ontology Authoring Data with Protégé
www.manchester.ac.uk/escholar/uk-ac-man-scw:223543
@markelvigo
markel.vigo@manchester.ac.uk
www.markelvigo.info
★