Semantic Analysis in IA Matthew Hodgson ACT regional-lead, Web and Information Management 23 Sept 2007

Jeffrey Veen on analysing content “a mind-numbingly detailed odyssey through your web site... …this process…is a relatively straightforward process of clicking through your web site and recording what you find.” Source: http://www.adaptivepath.com/ideas/essays/archives/000040.php

“a mind-numbingly detailed odyssey through your web site...

…this process…is a relatively straightforward process of clicking through your web site and recording what you find.”

Content overview – first take Medical restrictions text Free-text built in Word and hand-crafted (*grrr*) Unclassified Varied consistency within and between texts Highly complex sentence structures in pseudo-legalese Style reflects the author rather than the meaning in the communication Content needed for re-use Content output was needed for reuse by others Multiple audiences Multiple purposes for re-use Codification Codification (after authoring) takes too long Need to reduce timeframes!

Medical restrictions text

Free-text built in Word and hand-crafted (*grrr*)

Unclassified

Varied consistency within and between texts

Highly complex sentence structures in pseudo-legalese

Style reflects the author rather than the meaning in the communication

Content needed for re-use

Content output was needed for reuse by others

Multiple audiences

Multiple purposes for re-use

Codification

Codification (after authoring) takes too long

Need to reduce timeframes!

The task . . .analyse and codify

Linguistics … a whole discipline devoted to the study of language

Linguistics

… a whole discipline devoted to the

study of language

“You’re joking!?” All language has structure – even someone’s pseudo-legal English Analysing language is actually easier than you might think

All language has structure – even someone’s pseudo-legal English

Analysing language is actually easier than you might think

The approach Analyse semantics of content There is a predicable structure It’s all just Lego™ building blocks (nouns, verbs, adjectives, etc) Implied meaning can be made overt New tools for IAs to play with! Understand semantics, the structure of sentences, and you can analyse, categorise and codify English!

Analyse semantics of content

There is a predicable structure

It’s all just Lego™ building blocks (nouns, verbs, adjectives, etc)

Implied meaning can be made overt

New tools for IAs to play with!

Understand semantics, the structure of sentences, and you can analyse, categorise and codify English!

Language as Lego™ Building blocks Subject (S) Verb (V) Object (O) Order of blocks Differs depending on the language

Building blocks

Subject (S)

Verb (V)

Object (O)

Order of blocks

Differs depending on the language

Order from chaos SVO languages English, French, Chinese, Bulgarian, Swahili SOV Japanese, Turkish, Korean VSO Classical Arabic, Celtic and Hawaiian VOS Fijian, Yoda’s amusing phrases

SVO languages

English, French, Chinese, Bulgarian, Swahili

SOV

Japanese, Turkish, Korean

VSO

Classical Arabic, Celtic and Hawaiian

VOS

Fijian, Yoda’s amusing phrases

Subjects, verbs and objects Sometimes, though, the SVO structure is hidden: The apple is red or The apple is a red apple? Uncovering the hidden structure helps to differentiate between the subject and the object and identify the who and what

Sometimes, though, the SVO structure is hidden:

The apple is red or

The apple is a red apple?

Uncovering the hidden structure helps to differentiate between the subject and the object and identify the who and what

Sentences as (apple) trees

Semantic analysis Medical restrictions wording: Restricted benefit Gastro-oesophageal reflux disease; Scleroderma oesophagus; Authority required Peptic ulcer

Medical restrictions wording:

Restricted benefit Gastro-oesophageal reflux disease; Scleroderma oesophagus;

Authority required Peptic ulcer

Semantic analysis (cont.) Actual sentence Peptic ulcer Implied sentence The prescription of medicine is restricted to the initial treatment of patients with peptic ulcer

Actual sentence

Peptic ulcer

Implied sentence

The prescription of medicine is restricted to the initial treatment of patients with peptic ulcer

“ Who Treated” semantic model

“ Authority Action” semantic model

High-level semantic overview

How did the ‘trees’ help? Inferred How people think about and structure content Described Business processes that produce content Identified Where content quality is poor so it can be improved Critical components of the sentence for codification Designed Taxonomies and describe folk taxonomies Built Systems to help bring some structure to content authoring

Inferred

How people think about and structure content

Described

Business processes that produce content

Identified

Where content quality is poor so it can be improved

Critical components of the sentence for codification

Designed

Taxonomies and describe folk taxonomies

Built

Systems to help bring some structure to content authoring

How can I do this stuff too?! (a side-step) Theory is important An understanding of semantics - sentence trees and grammar Text books by authors like Fromkin and Rodman can help through the tricky bits Need good tools Conexor : www.conexor.fi/demo/syntax Big sheets of paper (and an electronic whiteboard) Visio (not PowerPoint!)

Theory is important

An understanding of semantics - sentence trees and grammar

Text books by authors like Fromkin and Rodman can help through the tricky bits

Need good tools

Conexor : www.conexor.fi/demo/syntax

Big sheets of paper (and an electronic whiteboard)

Visio (not PowerPoint!)

Demo Connexor www.conexor.fi/demo/syntax

Connexor

www.conexor.fi/demo/syntax

Introducing ways to codify restrictions How are we actually going to codify the stuff?! Give people Lego™ or ‘fridge-magnets’ to build sentences Build a prototype to explore and demonstrate conceptual design Communicate Talk about ideas with business owners Explore possibilities with end-users Build-in ‘no surprises’ into change management Iterate Iterate and refine concepts and design before it was built Inform Developers of intent and requirements The building of an ‘tool’ for codifying content (hooray for Axure!)

How are we actually going to codify the stuff?!

Give people Lego™ or ‘fridge-magnets’ to build sentences

Build a prototype to explore and demonstrate conceptual design

Communicate

Talk about ideas with business owners

Explore possibilities with end-users

Build-in ‘no surprises’ into change management

Iterate

Iterate and refine concepts and design before it was built

Inform

Developers of intent and requirements

The building of an ‘tool’ for codifying content (hooray for Axure!)

Demo Protyotyping with Axure

Protyotyping with Axure

Why should I care about this? Google uses semantic analysis to index content Translation software uses semantic analysis to identify ‘components’ for translation Good sentence structure equals: Accurate indexing Higher rank relevance of content Happy people (they find what they’re looking for)

Google uses semantic analysis to index content

Translation software uses semantic analysis to identify ‘components’ for translation

Good sentence structure equals:

Accurate indexing

Higher rank relevance of content

Happy people (they find what they’re looking for)

Summing up Content is still king, but: Is it’s quality any good? Does it match your website’s categories? Is your metadata ok? Can people find the content they need? Do you need to understand your content better? Semantic analysis can: Make your content audits more objective Inform processes to improve the quality of the content Inform processes to improve search engine indexing Inform metadata creation Improve website navigation design

Content is still king, but:

Is it’s quality any good?

Does it match your website’s categories?

Is your metadata ok?

Can people find the content they need?

Do you need to understand your content better?

Semantic analysis can:

Make your content audits more objective

Inform processes to improve the quality of the content

Inform processes to improve search engine indexing

Inform metadata creation

Improve website navigation design

email: [email_address] web: www.smsmt.com blog: magia3e.wordpress.com twitter: magia3e community: iacanberra.org cartoons: © Garry Larson Please Sir, can I have some more…?

email: [email_address] web: www.smsmt.com

blog: magia3e.wordpress.com twitter: magia3e community: iacanberra.org

cartoons: © Garry Larson

Fin

Fin