Semantic Pattern Transformation

  • 97 views
Uploaded on

Talk at IKNOW 2013, describing the Semantic Pattern Transformation. …

Talk at IKNOW 2013, describing the Semantic Pattern Transformation.
This process transforms feature vectors, which are commonly used in machine learning into a semantic representation. The advantage is that we can use this model across all domains, which is not possible for the raw feature vectors without cumbersome preprocessing operations.

More in: Technology , Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
97
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. IAIK Semantic Pattern Transformation IKNOW 2013 Peter Teufl, Herbert Leitold, Reinhard Posch peter.teufl@iaik.tugraz.at
  • 2. IAIK Our Background Topics Mobile device security Cloud security Security consulting for public insititutions (Austria) IT security research IT security lectures e-Government A-SIT
  • 3. IAIK Why does he talk about Knowledge Discovery? How does IT security relate to knowledge discovery? eGov - eParticipation: document analysis, twitter etc. intrusion detection systems (network traffic analysis) malware detection (network traffic, mobile phones) mobile application analysis (metadata, market descriptions) mobile application security (hot topic, BYOD, etc.)
  • 4. IAIK What to expect? Motivation for the Semantic Pattern Transformation Basic concepts, techniques How does it work? Evaluation? Applications, results, current topics!
  • 5. IAIK Environment Arbitrary features No apriori knowledge Heteregenous domains Clustering Supervised learning Anomaly Detection Semantic search Visualization Extracting knowledge Text analysis Android market descriptions histograms flexible deployment new domains terms numbers
  • 6. IAIK Process... •Different processing steps •From defining the goals •To extracting the desired knowledge •Machine learning algorithms are often used within KDD •However, the complete machine learning process is quite similar to KDD Knowledge discovery goals Target data set Preprocessing Data extraction Data mining method Data mining algorithm Knowledge extraction Data mining Knowledge processing Fayyad et al. Machine learning Domain-specific data set KDT Machine learning goals Instance extraction Feature selection, construction Instance selection Machine learning algorithm Preprocessing Algorithm application Interpretation ML-KDT
  • 7. IAIK ADAPTATION COMPLEXITY? •Assuming an arbitrary data-set (e-Participation, Android Market applications) •Further assuming: a knowledge discovery goal: e.g., unsupervised clustering •Then: we need to adapt the steps on the left •And: We need to adapt this setup when the data changes, even when the knowledge discovery goals remain the same! •Android Market applications vs. text documents vs. network traffic vs. malware detection? Domain-specific data set Machine learning goals Instance extraction Feature selection, construction Instance selection Algorithm selection Preprocessing Algorithm application Interpretation Machine Learning High Dependence on domain data and goals Medium Low
  • 8. IAIK TOWARDS A SEMANTIC REPRESENTATION •Finding a new representation... •New representation is called Semantic Patterns •Key properties: •Still a vector representation (compatible to old representation) •Not the feature values themselves, but their semantic relations are represented •All values have the same meaning and feature type (activation) •Transformation from raw data into Semantic Patterns: Semantic Pattern Transformation
  • 9. IAIK SEMANTIC PATTERN TRANSFORMATION •The Semantic Pattern Transformation is arranged in five layers •Layer 1 - Feature extraction •Layer 2 - Associative network - Node generation •Layer 3 - Associative network - Link generation •Layer 4 - Spreading activation (SA) •Layer 5 - Analysis (machine learning, semantic search etc.) Data set Relation FROM TO TIME FROM TO TIME FROM TO TIME SF 2 Instance SF 1 DF 1 DF 2SF 2 SV MV SV SV SV MV SV MV MV P 1 P 3 P 4 P 2 Supervised learning Unsupervised clustering Semantic relations Feature value relevance Anomaly detection Semantic development over time Pattern similarity Layer 1 Feature Extraction Layer 2 - 3 Associative Network Generation Layer 4 Spreading Activation Layer 5 Analysis SF 2 Instances Map Map Map
  • 10. IAIK SPT: Layer 1 - Feature extraction Extract features, their values and determine the type (categorical, distance-based) Categorical: Exports Distance-based: Unemployment rate, fertility rate Country Exports Unemployment rate Fertility rate C1 coffee 20% 5 C2 cacao 20% 5 C3 coffee, cacao 20% 5 C4 machinery 5% 2 C5 chemicals 5% 2 C6 chemicals, machinery 5% 2 C7 chemicals, cacao 20% missing data C8 missing data 20% 5 C9 coffee, cacao missing data missing data
  • 11. IAIK SPT: Layer 2 - Node generation 20% 5% coffee cocoa machinery chemicals 5 2 Country Exports Unemployment rate Fertility rate C1 coffee 20% 5 C2 cacao 20% 5 C3 coffee, cacao 20% 5 C4 machinery 5% 2 C5 chemicals 5% 2 C6 chemicals, machinery 5% 2 C7 chemicals, cacao 20% missing data C8 missing data 20% 5 C9 coffee, cacao missing data missing data Categorical feature values: one node for each value Distance-based feature values: map value ranges to single nodes Associative network
  • 12. IAIK SPT: Layer 3 - Link generation 0.25 0.75 0.5 Link Weight 1.00 20% 5 5% coffee cocoa machinery chemicals 2 Country Exports Unemployment rate Fertility rate C1 coffee 20% 5 C2 cacao 20% 5 C3 coffee, cacao 20% 5 C4 machinery 5% 2 C5 chemicals 5% 2 C6 chemicals, machinery 5% 2 C7 chemicals, cacao 20% missing data C8 missing data 20% 5 C9 coffee, cacao missing data missing data coffee, 20%, 5 chemicals, cacao, 20%
  • 13. IAIK SPT: Layer 4 - Spreading activation Creating a Semantic Pattern: in this case for “coffee” and “cacao” Set activation value of the two nodes to 1.0 Spread this activation value to neighboring nodes via the weighted links 20% 5 5% coffee cocoa machinery chemicals 2 1.0 1.0
  • 14. IAIK SPT: Layer 4 - Spreading activation Typically, one would create Semantic Patterns for all instances within the data set E.g. a pattern for C1 by activating coffee, 20% and 5 However, we can also create patterns for feature values: e.g. “coffee” Country Exports Unemployment rate Fertility rate C1 coffee 20% 5 C2 cacao 20% 5 C3 coffee, cacao 20% 5 C4 machinery 5% 2 C5 chemicals 5% 2 C6 chemicals, machinery 5% 2 C7 chemicals, cacao 20% missing data C8 missing data 20% 5 C9 coffee, cacao missing data missing data
  • 15. IAIK SPT: Layer 4 - Spreading activation After SA: each node in the network has an activation value By representing the nodes and their activation values as a vector, we gain a Semantic Pattern coffee cocoa machinery chemicals 20% 5% 5 2 0.00 0.08 0.38 0.300.00 0.001.151.15 cocoa 1.15 coffee 1.15 20% 0.38 5 0.30 chemicals 0.08 2 0.00 5% 0.00 machinery 0.00
  • 16. IAIK 0 0.25 0.50 coffee cacao machinery chemicals 20% 5% 5 2 Export: Cacao Unsorted Semantic Pattern 0 0.25 0.50 coffee cacao machinery chemicals 20% 5% 5 2 Export: Coffee Unsorted Semantic Pattern 0 0.25 0.50 coffee cacao machinery chemicals 20% 5% 5 2 Fertility: 2 Unsorted Semantic Pattern Country Exports Unemployment rate Fertility rate C1 coffee 20% 5 C2 cacao 20% 5 C3 coffee, cacao 20% 5 C4 machinery 5% 2 C5 chemicals 5% 2 C6 chemicals, machinery 5% 2 C7 chemicals, cacao 20% missing data C8 missing data 20% 5 C9 coffee, cacao missing data missing data Each feature value is represented by a semantic fingerprint Allows for an instant analysis of semantic relations to other feature values Sort, mean, variance, adding, subtracting
  • 17. IAIK SPT: Layer 5 - Analysis Calculating the distance between two patterns (Euclidean distance, Cosine similarity) For unsupervised clustering, semantic- aware search algorithms Keyword search for coffeeKeyword search for coffeeKeyword search for coffeeKeyword search for coffee C1 coffee 20% 5 C3 coffee, cacao 20% 5 C9 coffee, cacao missing data missing data Semantic aware search for coffeeSemantic aware search for coffeeSemantic aware search for coffeeSemantic aware search for coffee C9 coffee, cacao missing data missing data C1 coffee 20% 5 C3 coffee, cacao 20% 5 C2 cacao 20% 5 C8 missing data 20% 5 C7 chemicals, cacao 20% missing data C5 chemicals 5% 2 C6 chemicals, machinery 5% 2 C4 machinery 5% 2
  • 18. IAIK SPT: Layer 5 - Analysis Machine learning: apply any machine learning algorithm to the Semantic Patterns Unsupervised clustering Supervised learning Semantic-aware search Knowledge discovery: semantic relations, arbitrary procedures: mean, variance etc. Anomaly detection, feature relevance, simple operations (variance, mean, etc.) Visualization
  • 19. IAIK Benefits? Domain-specific data set Machine learning goals Instance extraction Feature selection, construction Instance selection Algorithm selection Preprocessing Algorithm application Interpretation Machine Learning Domain-specific data set Machine learning goals Instance extraction Feature selection, construction Instance selection Algorithm selection Preprocessing Algorithm application Interpretation High Dependence on domain data and goals Medium Low Application in heterogeneous domains regardless of the nature of the data Except for Layer 1, we do not need any manual setup for the layers Regardless of the analyzed data, the Semantic Patterns always use the same model This means: Regardless of the deployed knowledge discovery method, we can always use the same methods for knowledge extraction!
  • 20. IAIK Comparing the two models Country Coffee Cacao Machinery Chemicals 20% 5% 5 2 C1 1.30 0.53 0.00 0.08 1.45 0.00 1.45 0.00 C2 0.45 1.38 0.00 0.15 1.53 0.00 1.45 0.00 C3 1.45 1.53 0.00 0.15 1.68 0.00 1.60 0.00 C4 0.00 0.00 1.30 0.38 0.00 1.38 0.00 1.38 C5 0.00 0.08 0.38 1.30 0.08 1.38 0.00 1.38 C6 0.00 0.08 1.37 1.37 0.08 1.53 0.00 1.53 C7 0.30 1.30 0.08 1.15 1.30 0.15 0.45 0.15 C8 0.30 0.38 0.00 0.08 1.30 0.00 1.30 0.00 C9 1.15 1.15 0.00 0.08 0.38 0.00 0.30 0.00 0 0.75 1.50 coffee cacao machinery chemicals 20% 5% 5 2 Mean pattern: C4, C5, C6 Unsorted Semantic Pattern 0 1.00 2.00 coffee cacao machinery chemicals 20% 5% 5 2 Mean pattern: C1, C2, C3 Unsorted Semantic Pattern Country Coffee Cacao Machinery Chemicals Unemployment rate Fertility rate C1 1 0 0 0 20% 5 C2 0 1 0 0 20% 5 C3 1 1 0 0 20% 5 C4 0 0 1 0 5% 2 C5 0 0 0 1 5% 2 C6 0 0 1 1 5% 2 C7 0 1 0 1 20% missing data C8 missing datamissing datamissing datamissing data 20% 5 C9 1 1 0 0 missing data missing data Same model: Android application, a country or a document... the activation values always have the same meaning Semantic Patterns Value-centric feature vectors
  • 21. IAIK Evaluation 26 data sets from the UCI machine learning repository Supervised: SVM Unsupervised: EM and k-Means Application to raw data and to Semantic Patterns Data set Label Inst DF SF Classes SVM (N) SVM (NN) SVM (P) KM (N) KM (NN) KM (P) EM (NN) EM (P) Breast Cancer BC Dermatology DE KR vs. KP KR Lymph LY Mushroom MU Soybean SO Splice SP Vote VO Zoo ZO Anneal AN Colic CO Credit-A CA Credit-G CG Heart-C HC Heart-H HH Hepatitis HE Breast-w BW Diabetes DI Glass GL Heart-Statlog HS Ionosphere IO Iris IR Segment SE Sonar SO Vehicle VE Vowel VO SVMSVMSVM K-MeansK-MeansK-Means EMEM SP-Parameters: D=0.5, Comb=E, Norm=L, MDL=1.5, σ = 0.2SP-Parameters: D=0.5, Comb=E, Norm=L, MDL=1.5, σ = 0.2SP-Parameters: D=0.5, Comb=E, Norm=L, MDL=1.5, σ = 0.2SP-Parameters: D=0.5, Comb=E, Norm=L, MDL=1.5, σ = 0.2SP-Parameters: D=0.5, Comb=E, Norm=L, MDL=1.5, σ = 0.2SP-Parameters: D=0.5, Comb=E, Norm=L, MDL=1.5, σ = 0.2SP-Parameters: D=0.5, Comb=E, Norm=L, MDL=1.5, σ = 0.2SP-Parameters: D=0.5, Comb=E, Norm=L, MDL=1.5, σ = 0.2 CategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategorical 286 9 2 0.03 0.04 0.04 0.01 0.01 0.06 0.00 0.08 366 1 33 6 0.93 0.92 0.95 0.58 0.09 0.86 0.87 0.87 3196 36 2 0.75 0.75 0.72 0.00 0.01 0.00 0.04 0.00 148 18 4 0.53 0.51 0.48 0.13 0.18 0.25 0.26 0.27 8124 22 2 1.00 1.00 1.00 0.48 0.47 0.45 0.61 0.59 683 35 19 0.92 0.92 0.93 0.59 0.62 0.73 0.79 0.79 3190 60 3 0.71 0.72 0.80 0.03 0.03 0.44 0.41 0.31 435 16 2 0.76 0.74 0.67 0.47 0.48 0.47 0.49 0.45 101 17 7 0.94 0.94 0.97 0.78 0.78 0.82 0.82 0.85 TotalTotalTotalTotal 0.73 0.73 0.73 0.34 0.30 0.45 0.48 0.47 MixedMixedMixedMixedMixedMixedMixedMixedMixedMixedMixedMixed 898 6 32 6 0.86 0.86 0.92 0.23 0.03 0.30 0.31 0.32 368 7 15 2 0.31 0.32 0.31 0.13 0.03 0.05 0.10 0.12 689 6 9 2 0.41 0.41 0.39 0.16 0.02 0.25 0.17 0.21 1000 7 13 2 0.11 0.10 0.12 0.01 0.01 0.00 0.01 0.02 303 6 7 5 0.36 0.36 0.29 0.24 0.01 0.36 0.31 0.28 294 6 7 5 0.32 0.31 0.33 0.27 0.01 0.32 0.28 0.25 155 5 14 2 0.25 0.28 0.21 0.13 0.00 0.21 0.22 0.24 TotalTotalTotalTotal 0.37 0.38 0.37 0.17 0.02 0.21 0.20 0.20 NumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumerical 699 9 2 0.78 0.78 0.77 0.73 0.74 0.82 0.72 0.58 768 8 2 0.18 0.18 0.15 0.05 0.03 0.10 0.10 0.08 214 9 7 0.30 0.30 0.50 0.34 0.39 0.33 0.37 0.36 270 13 2 0.36 0.36 0.37 0.25 0.02 0.39 0.29 0.27 351 34 2 0.48 0.48 0.50 0.12 0.12 0.16 0.25 0.25 150 4 3 0.87 0.87 0.87 0.71 0.71 0.75 0.81 0.78 2310 19 7 0.88 0.88 0.90 0.61 0.53 0.59 0.62 0.60 208 60 2 0.23 0.23 0.23 0.01 0.01 0.02 0.01 0.01 846 18 4 0.51 0.51 0.48 0.11 0.19 0.19 0.10 0.19 990 10 3 11 0.63 0.63 0.76 0.06 0.34 0.23 0.19 0.25 TotalTotalTotalTotal 0.52 0.52 0.55 0.30 0.31 0.36 0.35 0.34
  • 22. IAIK •Applications described in several publications, which analyze •e-Participation (Egyptian revolution, Fukoshima, Mitmachen): text documents •Intrusion detection: event correlation •RDF data analysis (semantic web) •WiFi privacy (analyzing captured emails) •Android Market application analysis DOES IT WORK?
  • 23. IAIK Current Project Android application security Container applications for BYOD (require encryption, secure communication, key derivation functions, root checks etc.) Manual analysis is cumbersome Semantic Patterns Extract Dalvik VM code, features (opcodes, methods, local variables etc.) Apply Semantic Patterns technique Clustering, supervised learning, anomaly detection etc.
  • 24. IAIK Current Project
  • 25. IAIK Current Project Also works directly on the phone... Detecting SMS catchers/sniffers More fine grained detection assymmetric cryptography symmetric cryptography
  • 26. IAIK Outlook Publish the Java API... basically a converter from arbitrary feature vectors to Semantic Patterns (e.g. in/out in ARFF format) Deep learning...
  • 27. IAIK Thx! IAIK
  • 28. IAIK K-Means Par K-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-Means EMEMEMEMEMEMEMEMEMEM Total BC DE KR LY MU SO SP VO ZO Total BC DE KR LY MU SO SP VO ZO N NN D 0.0 D 0.1 D 0.3 D 0.5 D 0.7 D 0.1 D 0.3 D 0.5 D 0.7 D 0.1 D 0.3 D 0.5 D 0.7 D 0.1 D 0.3 D 0.5 D 0.7 Raw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw Data 0.341 0.012 0.584 0.004 0.131 0.475 0.587 0.031 0.467 0.782 Not availableNot availableNot availableNot availableNot availableNot availableNot availableNot availableNot availableNot available 0.296 0.007 0.094 0.010 0.176 0.472 0.616 0.030 0.476 0.783 0.477 0.002 0.871 0.036 0.258 0.610 0.789 0.410 0.494 0.822 Semantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic Patterns 0.443 0.025 0.849 0.003 0.199 0.413 0.728 0.465 0.493 0.814 0.449 0.004 0.767 0.001 0.222 0.590 0.740 0.423 0.489 0.801 Comb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=LComb=E Norm=L 0.442 0.029 0.811 0.004 0.245 0.545 0.726 0.387 0.476 0.759 0.441 0.074 0.885 0.000 0.271 0.615 0.786 0.004 0.505 0.826 0.447 0.068 0.846 0.004 0.241 0.482 0.724 0.424 0.476 0.758 0.460 0.079 0.875 0.001 0.258 0.592 0.788 0.250 0.449 0.846 0.452 0.061 0.856 0.000 0.245 0.448 0.733 0.437 0.467 0.820 0.468 0.079 0.874 0.001 0.265 0.592 0.789 0.306 0.452 0.850 0.422 0.069 0.826 0.000 0.209 0.275 0.728 0.419 0.463 0.804 0.465 0.079 0.874 0.001 0.252 0.579 0.799 0.312 0.445 0.847 Comb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=LComb=S Norm=L 0.441 0.056 0.853 0.000 0.244 0.453 0.733 0.399 0.476 0.759 0.433 0.079 0.872 0.001 0.270 0.572 0.794 0.001 0.476 0.829 0.434 0.075 0.820 0.000 0.228 0.411 0.718 0.431 0.472 0.750 0.466 0.079 0.881 0.001 0.280 0.592 0.802 0.298 0.437 0.828 0.439 0.060 0.792 0.000 0.235 0.416 0.741 0.405 0.463 0.836 0.466 0.079 0.871 0.001 0.251 0.581 0.805 0.310 0.445 0.848 0.422 0.067 0.798 0.000 0.224 0.364 0.726 0.376 0.462 0.782 0.462 0.087 0.875 0.001 0.254 0.580 0.776 0.292 0.445 0.845 Comb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=SComb=E Norm=S 0.418 0.029 0.790 0.006 0.236 0.311 0.705 0.449 0.496 0.742 0.472 0.002 0.893 0.000 0.263 0.571 0.767 0.432 0.495 0.820 0.452 0.030 0.860 0.001 0.231 0.470 0.715 0.475 0.491 0.799 0.476 0.002 0.914 0.000 0.261 0.586 0.775 0.427 0.495 0.823 0.448 0.048 0.799 0.009 0.215 0.539 0.725 0.450 0.493 0.758 0.472 0.002 0.897 0.000 0.267 0.584 0.758 0.427 0.484 0.829 0.448 0.033 0.850 0.000 0.230 0.495 0.712 0.435 0.493 0.787 0.473 0.002 0.903 0.000 0.250 0.586 0.773 0.427 0.484 0.829 Comb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=SComb=S Norm=S 0.439 0.029 0.806 0.009 0.250 0.435 0.727 0.439 0.494 0.760 0.475 0.002 0.903 0.000 0.254 0.576 0.764 0.429 0.495 0.852 0.420 0.015 0.775 0.004 0.210 0.436 0.717 0.409 0.443 0.774 0.474 0.002 0.901 0.000 0.271 0.584 0.763 0.427 0.484 0.837 0.429 0.030 0.789 0.009 0.226 0.410 0.716 0.448 0.485 0.749 0.476 0.002 0.904 0.000 0.255 0.586 0.767 0.427 0.484 0.854 0.438 0.040 0.839 0.006 0.246 0.418 0.726 0.409 0.480 0.775 0.480 0.002 0.910 0.000 0.269 0.615 0.771 0.431 0.494 0.825
  • 29. IAIK K-Means Par K-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-Means EMEMEMEMEMEMEMEM Total AN CO CA CG HC HH HE Total AN CO CA CG HC HH HE N NN σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 Raw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw Data 0.165 0.226 0.129 0.155 0.009 0.237 0.269 0.131 Not availableNot availableNot availableNot availableNot availableNot availableNot availableNot available 0.017 0.028 0.030 0.016 0.012 0.014 0.012 0.004 0.201 0.312 0.103 0.171 0.013 0.309 0.278 0.223 Semantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic Patterns D=0.0 MDL=2.0D=0.0 MDL=2.0D=0.0 MDL=2.0D=0.0 MDL=2.0D=0.0 MDL=2.0D=0.0 MDL=2.0D=0.0 MDL=2.0D=0.0 MDL=2.0 D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0 0.193 0.253 0.135 0.113 0.007 0.356 0.293 0.195 0.190 0.291 0.098 0.227 0.003 0.228 0.258 0.227 0.198 0.271 0.147 0.116 0.007 0.356 0.301 0.189 0.182 0.280 0.098 0.162 0.003 0.244 0.258 0.231 0.204 0.240 0.157 0.145 0.009 0.356 0.327 0.194 0.184 0.226 0.099 0.229 0.004 0.245 0.258 0.227 0.194 0.221 0.154 0.145 0.008 0.359 0.275 0.196 0.194 0.291 0.097 0.240 0.003 0.217 0.281 0.229 0.200 0.258 0.152 0.098 0.007 0.358 0.327 0.197 0.192 0.293 0.097 0.232 0.004 0.228 0.258 0.230 D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0 D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0 0.211 0.320 0.042 0.262 0.001 0.325 0.311 0.215 0.210 0.327 0.127 0.218 0.021 0.237 0.311 0.229 0.201 0.257 0.032 0.262 0.001 0.323 0.311 0.222 0.210 0.322 0.126 0.218 0.021 0.237 0.320 0.229 0.208 0.299 0.035 0.261 0.001 0.326 0.311 0.220 0.211 0.322 0.127 0.218 0.021 0.237 0.320 0.229 0.204 0.281 0.029 0.262 0.001 0.325 0.311 0.220 0.211 0.321 0.128 0.218 0.021 0.237 0.320 0.229 0.207 0.292 0.041 0.263 0.001 0.326 0.311 0.216 0.209 0.310 0.127 0.218 0.021 0.237 0.320 0.229 D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5 D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5 0.216 0.317 0.065 0.249 0.001 0.357 0.320 0.203 0.204 0.322 0.123 0.212 0.016 0.275 0.247 0.233 0.211 0.295 0.052 0.247 0.000 0.355 0.320 0.209 0.204 0.322 0.123 0.212 0.016 0.275 0.247 0.236 0.216 0.314 0.074 0.248 0.001 0.357 0.320 0.198 0.205 0.323 0.123 0.206 0.016 0.275 0.252 0.237 0.212 0.308 0.046 0.249 0.001 0.356 0.320 0.209 0.204 0.320 0.125 0.208 0.016 0.275 0.246 0.236 0.211 0.293 0.063 0.248 0.000 0.354 0.320 0.201 0.204 0.323 0.125 0.208 0.016 0.275 0.249 0.232 D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0 D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0 0.217 0.304 0.048 0.244 0.000 0.390 0.311 0.219 0.206 0.319 0.117 0.229 0.010 0.255 0.277 0.233 0.218 0.313 0.062 0.244 0.000 0.388 0.311 0.208 0.207 0.317 0.126 0.239 0.010 0.255 0.268 0.233 0.221 0.309 0.084 0.243 0.000 0.389 0.311 0.209 0.205 0.319 0.127 0.224 0.010 0.255 0.268 0.233 0.213 0.285 0.057 0.243 0.000 0.387 0.311 0.210 0.206 0.307 0.127 0.240 0.010 0.255 0.268 0.233 0.211 0.295 0.036 0.244 0.000 0.387 0.311 0.205 0.204 0.305 0.127 0.240 0.010 0.255 0.259 0.233 D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0 D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0 0.203 0.294 0.030 0.248 0.000 0.335 0.315 0.196 0.192 0.323 0.108 0.248 0.009 0.201 0.250 0.205 0.208 0.306 0.059 0.248 0.000 0.334 0.315 0.193 0.190 0.321 0.107 0.237 0.009 0.201 0.251 0.205 0.205 0.310 0.050 0.248 0.000 0.334 0.315 0.178 0.193 0.322 0.122 0.243 0.009 0.201 0.249 0.205 0.207 0.300 0.063 0.248 0.001 0.333 0.313 0.192 0.192 0.321 0.122 0.243 0.010 0.201 0.245 0.205 0.210 0.330 0.050 0.246 0.001 0.336 0.315 0.191 0.192 0.323 0.122 0.243 0.009 0.201 0.240 0.205
  • 30. IAIK K-Means Par K-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-MeansK-Means EMEMEMEMEMEMEMEMEMEMEM Total BW DI GL HS IO IR SE SO VE VO Total BW DI GL HS IO IR SE SO VE VO N NN σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 σ 0.0 σ 0.2 σ 0.4 σ 0.6 σ 0.8 Raw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw DataRaw Data 0.299 0.734 0.052 0.335 0.254 0.121 0.708 0.608 0.006 0.113 0.057 Not availableNot availableNot availableNot availableNot availableNot availableNot availableNot availableNot availableNot availableNot available 0.307 0.735 0.030 0.388 0.019 0.123 0.705 0.529 0.008 0.188 0.342 0.346 0.718 0.103 0.370 0.289 0.254 0.806 0.621 0.005 0.103 0.194 Semantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic PatternsSemantic Patterns D=0.0 MDL=1.5D=0.0 MDL=1.5D=0.0 MDL=1.5D=0.0 MDL=1.5D=0.0 MDL=1.5D=0.0 MDL=1.5D=0.0 MDL=1.5D=0.0 MDL=1.5 D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0D=0.0 MDL=1.0 0.315 0.724 0.039 0.329 0.309 0.045 0.717 0.582 0.026 0.198 0.183 0.317 0.777 0.006 0.312 0.239 0.218 0.651 0.592 0.016 0.174 0.186 0.323 0.724 0.025 0.334 0.344 0.071 0.730 0.590 0.012 0.198 0.196 0.327 0.752 0.001 0.318 0.240 0.218 0.766 0.598 0.016 0.167 0.197 0.318 0.719 0.026 0.285 0.316 0.051 0.769 0.600 0.008 0.199 0.203 0.323 0.727 0.011 0.287 0.229 0.217 0.749 0.600 0.018 0.176 0.218 0.317 0.722 0.025 0.298 0.357 0.040 0.712 0.602 0.013 0.199 0.201 0.317 0.732 0.009 0.316 0.232 0.221 0.637 0.606 0.025 0.175 0.214 0.299 0.646 0.015 0.294 0.328 0.026 0.686 0.581 0.014 0.198 0.200 0.325 0.703 0.006 0.305 0.233 0.216 0.796 0.594 0.019 0.181 0.195 D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0D=0.5 MDL=1.0 D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0D=0.7 MDL=1.0 0.333 0.817 0.072 0.293 0.338 0.181 0.611 0.614 0.009 0.164 0.234 0.302 0.579 0.082 0.332 0.285 0.184 0.633 0.634 0.006 0.099 0.183 0.333 0.817 0.076 0.278 0.340 0.181 0.621 0.621 0.009 0.151 0.237 0.300 0.579 0.082 0.307 0.285 0.184 0.636 0.632 0.006 0.117 0.176 0.326 0.817 0.068 0.286 0.335 0.181 0.587 0.604 0.009 0.149 0.228 0.301 0.579 0.086 0.310 0.285 0.184 0.639 0.643 0.006 0.095 0.183 0.327 0.817 0.072 0.269 0.337 0.181 0.604 0.580 0.009 0.166 0.232 0.301 0.579 0.076 0.319 0.285 0.184 0.639 0.632 0.006 0.109 0.185 0.334 0.817 0.071 0.303 0.336 0.181 0.610 0.605 0.011 0.163 0.244 0.300 0.579 0.079 0.311 0.285 0.184 0.633 0.633 0.006 0.109 0.183 D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5D=0.5 MDL=1.5 D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5D=0.7 MDL=1.5 0.352 0.817 0.099 0.298 0.382 0.143 0.751 0.601 0.018 0.193 0.218 0.339 0.579 0.086 0.348 0.324 0.242 0.761 0.596 0.013 0.187 0.252 0.358 0.817 0.100 0.330 0.385 0.163 0.751 0.588 0.015 0.194 0.232 0.339 0.579 0.086 0.356 0.324 0.242 0.761 0.595 0.012 0.192 0.239 0.352 0.817 0.096 0.315 0.387 0.143 0.738 0.576 0.019 0.193 0.231 0.340 0.579 0.092 0.348 0.324 0.242 0.761 0.603 0.012 0.194 0.241 0.348 0.817 0.103 0.288 0.383 0.158 0.716 0.579 0.015 0.194 0.226 0.339 0.579 0.094 0.355 0.324 0.242 0.761 0.602 0.012 0.181 0.240 0.356 0.817 0.098 0.296 0.378 0.166 0.776 0.604 0.012 0.190 0.225 0.338 0.579 0.107 0.355 0.324 0.242 0.752 0.597 0.012 0.177 0.236 D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0D=0.5 MDL=2.0 D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0D=0.7 MDL=2.0 0.329 0.817 0.054 0.339 0.330 0.064 0.752 0.563 0.017 0.151 0.199 0.323 0.579 0.105 0.347 0.266 0.228 0.784 0.585 0.015 0.092 0.227 0.328 0.817 0.052 0.320 0.330 0.064 0.753 0.585 0.017 0.144 0.196 0.325 0.579 0.098 0.359 0.266 0.228 0.784 0.584 0.015 0.098 0.238 0.331 0.817 0.055 0.313 0.330 0.109 0.767 0.562 0.012 0.149 0.194 0.323 0.579 0.105 0.358 0.266 0.228 0.784 0.576 0.015 0.090 0.230 0.330 0.817 0.059 0.335 0.328 0.073 0.765 0.560 0.019 0.148 0.199 0.326 0.579 0.099 0.351 0.266 0.228 0.798 0.595 0.015 0.091 0.235 0.333 0.817 0.064 0.321 0.330 0.068 0.764 0.593 0.013 0.158 0.200 0.326 0.579 0.104 0.361 0.266 0.228 0.798 0.585 0.015 0.090 0.237 D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0D=0.5 MDL=3.0 D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0D=0.7 MDL=3.0 0.322 0.817 0.026 0.326 0.333 0.099 0.739 0.567 0.022 0.136 0.153 0.304 0.579 0.001 0.362 0.200 0.228 0.728 0.574 0.032 0.114 0.224 0.322 0.817 0.029 0.326 0.320 0.127 0.702 0.583 0.017 0.150 0.150 0.307 0.579 0.000 0.364 0.208 0.228 0.735 0.573 0.029 0.113 0.236 0.317 0.817 0.035 0.318 0.320 0.099 0.705 0.556 0.024 0.140 0.154 0.306 0.579 0.001 0.355 0.211 0.228 0.726 0.572 0.035 0.113 0.237 0.328 0.817 0.026 0.342 0.328 0.118 0.759 0.563 0.020 0.150 0.153 0.307 0.579 0.001 0.363 0.219 0.228 0.729 0.575 0.029 0.113 0.233 0.323 0.817 0.029 0.330 0.322 0.099 0.731 0.563 0.023 0.151 0.161 0.304 0.579 0.001 0.356 0.204 0.224 0.713 0.589 0.030 0.119 0.226
  • 31. IAIK Distance Data Missing EucEucEucEucEucEucEucEuc CosCosCosCosCosCosCosCos RawRawRawRaw Semantic PatternsSemantic PatternsSemantic PatternsSemantic Patterns RawRawRawRaw Semantic PatternsSemantic PatternsSemantic PatternsSemantic Patterns 0% 10% 50% 90% 0% 10% 50% 90% 0% 10% 50% 90% 0% 10% 50% 90% BC DE KR LY MU SO SP VO ZO Total AN CO CA CG HC HH HE Total BW DI GL HS IO IR SE SO VE VO Total CategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategorical 0.52 0.52 0.52 0.52 0.54 0.54 0.53 0.50 0.53 0.53 0.53 0.51 0.54 0.54 0.53 0.51 0.68 0.66 0.55 0.32 0.81 0.80 0.38 0.22 0.66 0.66 0.67 0.36 0.81 0.80 0.74 0.46 0.54 0.54 0.53 0.52 0.52 0.52 0.51 0.50 0.54 0.54 0.53 0.51 0.52 0.52 0.52 0.51 0.63 0.68 0.63 0.30 0.63 0.59 0.64 0.48 0.59 0.53 0.51 0.32 0.61 0.58 0.56 0.35 0.64 0.64 0.62 0.57 0.68 0.67 0.62 0.53 0.57 0.57 0.56 0.54 0.67 0.67 0.67 0.62 0.65 0.63 0.53 0.22 0.75 0.70 0.09 0.08 0.58 0.56 0.50 0.18 0.73 0.72 0.63 0.28 0.48 0.47 0.44 0.38 0.62 0.46 0.39 0.39 0.44 0.44 0.41 0.37 0.57 0.57 0.54 0.45 0.80 0.79 0.76 0.67 0.78 0.78 0.68 0.51 0.62 0.63 0.67 0.62 0.79 0.79 0.78 0.72 0.83 0.81 0.72 0.31 0.86 0.85 0.64 0.24 0.80 0.79 0.71 0.31 0.86 0.84 0.76 0.41 0.64 0.64 0.59 0.42 0.69 0.66 0.50 0.38 0.59 0.58 0.57 0.41 0.68 0.67 0.64 0.48 MixedMixedMixedMixedMixedMixedMixedMixedMixedMixedMixedMixedMixedMixedMixedMixed 0.64 0.63 0.55 0.38 0.66 0.67 0.51 0.38 0.44 0.46 0.50 0.38 0.66 0.66 0.61 0.42 0.59 0.59 0.56 0.51 0.59 0.58 0.52 0.50 0.50 0.50 0.51 0.51 0.62 0.62 0.60 0.57 0.62 0.61 0.59 0.54 0.65 0.65 0.60 0.52 0.55 0.55 0.54 0.51 0.65 0.64 0.63 0.57 0.52 0.52 0.52 0.50 0.52 0.53 0.54 0.53 0.51 0.51 0.52 0.51 0.52 0.52 0.52 0.52 0.86 0.86 0.85 0.81 0.87 0.87 0.85 0.81 0.81 0.81 0.82 0.81 0.87 0.87 0.86 0.84 0.87 0.86 0.85 0.82 0.87 0.87 0.83 0.80 0.84 0.84 0.83 0.81 0.88 0.88 0.87 0.83 0.59 0.58 0.56 0.50 0.64 0.64 0.58 0.55 0.52 0.51 0.55 0.52 0.65 0.65 0.64 0.57 0.67 0.67 0.64 0.58 0.69 0.69 0.63 0.58 0.60 0.60 0.61 0.58 0.69 0.69 0.68 0.62 NumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumericalNumerical 0.86 0.86 0.76 0.68 0.91 0.91 0.84 0.69 0.62 0.61 0.59 0.50 0.90 0.89 0.88 0.84 0.55 0.54 0.53 0.53 0.56 0.55 0.54 0.50 0.53 0.53 0.52 0.50 0.56 0.55 0.55 0.53 0.49 0.45 0.31 0.30 0.53 0.52 0.42 0.31 0.51 0.51 0.48 0.29 0.53 0.52 0.48 0.34 0.64 0.63 0.59 0.52 0.69 0.69 0.61 0.53 0.54 0.54 0.55 0.51 0.69 0.69 0.65 0.60 0.51 0.52 0.55 0.54 0.61 0.61 0.56 0.46 0.46 0.46 0.47 0.51 0.61 0.61 0.60 0.57 0.81 0.60 0.47 0.33 0.83 0.81 0.75 0.67 0.87 0.84 0.77 0.34 0.84 0.81 0.76 0.75 0.61 0.53 0.21 0.15 0.57 0.57 0.43 0.17 0.39 0.40 0.44 0.27 0.57 0.57 0.55 0.41 0.54 0.53 0.51 0.50 0.54 0.54 0.51 0.50 0.52 0.52 0.52 0.52 0.54 0.54 0.54 0.53 0.35 0.33 0.29 0.26 0.37 0.37 0.35 0.28 0.36 0.36 0.36 0.31 0.37 0.37 0.36 0.33 0.15 0.15 0.12 0.09 0.22 0.21 0.16 0.10 0.20 0.20 0.17 0.10 0.21 0.21 0.20 0.13 0.55 0.51 0.43 0.39 0.58 0.58 0.52 0.42 0.50 0.50 0.49 0.38 0.58 0.58 0.56 0.50
  • 32. IAIK Data set EUC (N) EUC (NN) COS (NN) EUC (NN) COS (NN) EUC (NN) COS (NN) BC DE KR LY MU SO SP VO ZO Total AN CO CA CG HC HH HE Total BW DI GL HS IO IR SE SO VE VO Total RAWRAWRAW BaselineBaseline Semantic PatternsSemantic Patterns CategoricalCategoricalCategoricalCategoricalCategoricalCategoricalCategorical 0.52 0.53 0.53 0.52 0.53 0.54 0.54 0.68 0.68 0.66 0.67 0.67 0.81 0.81 0.54 0.54 0.54 0.54 0.54 0.52 0.52 0.63 0.63 0.59 0.60 0.57 0.63 0.61 0.64 0.64 0.57 0.64 0.64 0.68 0.67 0.65 0.65 0.58 0.69 0.70 0.75 0.73 0.48 0.48 0.44 0.48 0.48 0.62 0.57 0.80 0.80 0.62 0.80 0.80 0.78 0.79 0.84 0.83 0.80 0.85 0.84 0.86 0.86 0.64 0.64 0.59 0.64 0.64 0.69 0.68 MixedMixedMixedMixedMixedMixedMixed 0.64 0.64 0.44 0.64 0.65 0.65 0.66 0.59 0.59 0.50 0.59 0.60 0.58 0.62 0.62 0.62 0.55 0.61 0.61 0.61 0.65 0.52 0.52 0.51 0.52 0.52 0.52 0.52 0.86 0.86 0.81 0.85 0.85 0.86 0.87 0.87 0.87 0.84 0.86 0.86 0.86 0.88 0.59 0.59 0.52 0.61 0.60 0.63 0.65 0.67 0.67 0.60 0.67 0.67 0.67 0.69 NumericalNumericalNumericalNumericalNumericalNumericalNumerical 0.86 0.86 0.62 0.74 0.74 0.89 0.90 0.55 0.55 0.53 0.54 0.54 0.55 0.56 0.49 0.49 0.51 0.51 0.51 0.53 0.53 0.64 0.64 0.54 0.63 0.63 0.66 0.69 0.51 0.51 0.46 0.55 0.55 0.63 0.61 0.81 0.81 0.87 0.73 0.73 0.81 0.83 0.61 0.61 0.39 0.54 0.54 0.57 0.57 0.54 0.54 0.52 0.54 0.54 0.54 0.54 0.35 0.35 0.36 0.37 0.37 0.36 0.37 0.15 0.15 0.20 0.21 0.21 0.22 0.21 0.55 0.55 0.50 0.54 0.54 0.58 0.58