SEMANTIC CONTENT MANAGEMENT FOR ENTERPRISES AND NATIONAL SECURITY

SEMANTIC CONTENT MANAGEMENT FOR ENTERPRISES AND NATIONAL SECURITY Amit Sheth CTO, Voquette*, Inc. Large Scale Distributed Information Systems (LSDIS) Lab University Of Georgia; http://lsdis.cs.uga.edu *Now Semagix, http://www.semagix.com July 15, 2002 © Amit Sheth Keynote CONTENT- AND SEMANTIC-BASED INFORMATION RETRIEVAL @ SCI 2002

New Enterprise Content Management Challenges
More variety and complexity
More formats (MPEG, PDF, MS Office, WM, Real, AVI, etc)
More types (Docs, Images -> Audio, Video, Variety of text-structured, unstructured)
More sources (internal, extranet, internet, feeds)
Saclability, Information Overload
Too much data, precious little information (Relevance)
Creating Value from Content
How to Distribute the right content to the right people as needed? (Personalization -- book of business)
Customized delivery for different consumption options (mobile/desktop, devices)
Insight, Decision Making (Actionable)

New Enterprise Content Management Technical Challenges
Aggregation
Feed handlers/Agents that understand content representation and media semantics
Push-pull, Web-DB-Files, Structured-Semi-structured-Unstructured data of different types
Homogenization and Enhancement
Enterprise-wide common view
Domain model, taxonomy/classification, metadata standards
Semantic Metadata– created automatically if possible
Semantic Applications
Search, personalization, directory, alerts, etc. using metadata and semantics (semantic association and correlation), for improved relevance, intelligent personalization, customization

The Semantic Web -- a vision with several views:
· “The Web of data (and connections) with meaning in the sense that a computer program can learn enough about what data means to process it.” [B99]
· “The semantic Web is an extension of the current Web in which information is given well-defined meaning, better enabling computers and people to work in cooperation.” [BHL01]
· “The Semantic Web is a vision: the idea of having data on the Web defined and linked in a way that it can be used by machines not just for display purposes, but for automation, integration and reuse of data across various applications. [W3C01]
Semantics: The Next Step in the Web’s Evolution

Semantics for the Web
On the Semantic Web every resource (people, enterprises, information services, application services, and devices) are augmented with machine processable descriptions to support the finding, reasoning about (e.g., which service is best), and using (e.g., executing or manipulating) the resource. The idea is that self-descriptions of data and other techniques would allow context-understanding programs to selectively find what users want, or for programs to work on behalf of humans and organizations to make them more efficient and productive.

Digital Content and Semantics

Central Role of Metadata Applications Back End "A Web content repository without metadata is like a library without an index." - Jack Jia, IWOV “ Metadata increases content value in each step of content value chain.” Amit Sheth Where is the content? Whose is it? Produce Aggregate What is this content about? Catalog/ Index What other content is it related to? Integrate Syndicate What is the right content for this user? Personalize What is the best way to monetize this interaction? Interactive Marketing Broadcast, Wireline, Wireless, Interactive TV Semantic Metadata

A Metadata Classification Data (Heterogeneous Types/Media) Content Independent Metadata (creation-date, location, type-of-sensor...) Content Dependent Metadata (size, max colors, rows, columns...) Direct Content Based Metadata (inverted lists, document vectors, LSI) Domain Independent (structural) Metadata (C++ class-subclass relationships, HTML/SGML Document Type Definitions, C program structure...) Domain Specific Metadata area, population (Census), land-cover, relief (GIS),metadata concept descriptions from ontologies Ontologies Classifications Domain Models User More Semantics for Relevance to tackle Information Overload!!

Semantic Content Organization and Retrieval Engine (SCORE) technology
Automatically aggregates and extracts information from disparate sources and multiple formats
Automatically tags/annotates and categorizes content
Automatically creates relevant associations
Maps content topics and their relationships
Semantic query engine relates information and knowledge both internal and external to the organization into a single view

SCORE Architecture

SCORE Architecture Distributed agents that automatically extract relevant semantic metadata from structured and unstructured content Fast main-memory based query engine with APIs and XML output CACS provides automatic classification (w.r.t. WorldModel) from unstructured text and extracts contextually relevant metadata Distributed agents that automatically extract/mine knowledge from trusted sources Toolkit to design and maintain the Knowledgebase Knowledgebase represents the real-world instantiation (entities and relationships) of the WorldModel WorldModel specifies enterprise’s normalized view of information (ontology)

Voquette Enterprise Semantic Platform Product Components World Model WM Toolkit Knowledgebase and Metabase Main Memory Index XML APIs Web Services Enterprise Applications EA EA EA Semantic Engine Search Alerts Portals Directory Personalize Enhancement Engine CA CA CA Content Agent Monitor Content Agents Databases XML/Feeds Websites Email Content Sources Entity Extraction, Enhanced Metadata, Domain Experts Automatic Classification Classification Committee Reports Documents Structured Semi- Structured Unstructured CA Toolkit Knowledge Agent Monitor KS KS KS KS KA KA KA Knowledge Sources Knowledge Agents KA Toolkit Knowledgebase KB Toolkit Knowledge Agent Monitor KS KS KS KS KA KA KA Knowledge Sources Knowledge Agents KA Toolkit Metabase Enhancement Engine CA CA CA Content Agent Monitor Content Agents Databases XML/Feeds Websites Email Content Sources Entity Extraction, Enhanced Metadata, Domain Experts Automatic Classification Classification Committee Reports Documents Structured Semi- Structured Unstructured CA Toolkit

PERSON (OFAC, FBI, DPL) -politician (OFAC, FBI, CIA, CA) politician associated with politicalOrganziation politician held politicalOffice politician associated with politicalOffice -terrorist (OFAC, FBI, DPL) terrorist memberOf organization terrorist appears on watchList -companyExecutive (MG) companyExecutive holdsOffice companyPosition person has permanent address address (OFAC, FBI) person has dob(date of birth) (OFAC, FBI) person has pob(place of birth) (OFAC, FBI) Knowledge Sources Used THING -event (ICT) terroristOrganization participated in terroristSponsoredEvent (ICT) -politicalOffice (CIA, CA) politicalOffice office(s) within govtOrganization politicalOffice associated with organization -watchList (OFAC, FBI, DPL) terroristOrganization appears on watchList (OFAC, FBI, DPL) -organization (OFAC, FBI, FAS, ICT, CA, CIA) organization appears on watchList organization memberOf suborganization -company company manufactures product (ZD) company identifiedBy tickeySymbol (H) companyposition position in company (MG) company memberOf industry (H) -tickerSymbol (H) tickerSymbol memberOf exchange (H) PLACE -organization located in place (H, OFAC) -religiousAffiliation practiced in place (CIA) -company headquarters in city (H) Entity Classes and Relationships populated by these knowledge sources: JIVA Market Guide (MG) ZDNet (ZD) Hoover’s (H) Data supplied from NASA (DPL) Federation of American Scientists (FAS) C entral Intelligence Agency (CIA) The Interdisciplinary Center (ICT) Federal Bureau of Investigation (FBI) Capital Advantage (CA) Office of Foreign Assets Control ( OFAC)

SCORE Capabilities
Semantics (understanding of content and user needs)
Extreme relevance
Semantic associations
Near real-time
Multiple applications/usage patterns (not just search)
Automation
Scalability in all aspects

Technologies Involved
Ontology driven architecture (definitional, assertional components
Automatic Classification with classifier committee (multiple technologies, rather than one size fits all)
Automatic Semantic Metadata Extraction/Annotation
Semantic associations/ knowledge inferences
Scalability throughout with distributed architecture and implementation (number of content and knowledge sources, indexing, etc.)
Main memory implementation, incremental check pointing

Performance > 10,000 entities/relationships per hr. Population/update rate in a Knowledgebase with 1 million entities/relationships 1 minute (near real-time) Incremental Index Update Frequency 65ms Query Response Time (64 concurrent users) 1 - 10 ms Query Response Time (light load) > 1,980,000 Queries per server per hour

Information Extraction for Metadata Creation METADATA EXTRACTORS Key challenge: Create/extract as much (semantics) metadata automatically as possible WWW, Enterprise Repositories Digital Maps Nexis UPI AP Feeds/ Documents Digital Audios Data Stores Digital Videos Digital Images . . . . . . . . .

Video with Editorialized Text on the Web Automatic Categorization & Metadata Tagging (Web page) Auto Categorization Semantic Metadata

Extraction Agent Web Page Enhanced Metadata Asset Content Extraction and Knowledgebase Enhancement

Content Enhancement Workflow Semantic Metadata Syntax Metadata

Content Asset Index Evolution Extractor Agent for Bloomberg Scans text for analysis Metadata extracted automatically Asset Syntax Metadata Producer: BusinessWire Source: Bloomberg Date: Sept. 10 2001 Location: San Jose, CA URL: http://bloomberg.com/1.htm Media: Text Semantic Metadata Company: Cisco Systems, Inc. Creates asset (index) out of extracted metadata Asset Syntax Metadata Producer: BusinessWire Source: Bloomberg Date: Sept. 10 2001 Location: San Jose, CA URL: http://bloomberg.com/1.htm Media: Text Semantic Metadata Company: Cisco Systems, Inc. Topic: Company News Categorization & Auto-Cataloging System (CACS) Scans text for analysis Classifies document into pre-defined category/topic Appends topic metadata to asset Cisco Systems CSCO NASDAQ Company Ticker Exchange Industry Sector Executives John Chambers Telecomm. Computer Hardware Competition Nortel Networks Knowledge Base CEO of Competes with Syntax Metadata Asset Producer: BusinessWire Source: Bloomberg Date: Sept. 10 2001 Location: San Jose, CA URL: http://bloomberg.com/1.htm Media: Text Semantic Metadata Company: Cisco Systems, Inc. Topic: Company News Ticker: CSCO Exchange: NASDAQ Industry: Telecomm. Sector: Computer Hardware Executive: John Chambers Competition: Nortel Networks Headquarters: San Jose, CA Leverages knowledge to enhance metatagging Enhanced Content Asset Indexed Headquarters San Jose XML Feed Semantic Engine

Content which does contain the words the user asked for Extractor Agents Content which does not contain the words the user asked for, but is about what he asked for. Value-added Metadata Content the user did not think to ask for , but which he needs to know . Semantic Associations + + Intelligent Content End-User Intelligent Content Empowers the User

Example 1 – Snapshots (“Jamal Anderson”) Click on first result for Jamal Anderson View metadata. Note that Team name and League name are also included in the metadata Search for ‘Jamal Anderson’ in ‘Football’ View the original source HTML page. Verify that the source page contains no mention of Team name and League name . They are value-additions to the metadata to facilitate easier search.

Semantic Application Example – Research Dashboard Focused relevant content organized by topic ( semantic categorization ) Automatic Content Aggregation from multiple content providers and feeds Related relevant content not explicitly asked for (semantic associations) Competitive research inferred automatically Automatic 3 rd party content integration

Internal Source 1 Research Internal Source 2 External feeds/Web (e.g. Reuters) Voquette Metabase World Model Third-party Content Mgmt And Syndication Semantic Engine 1 2 3 4 Cisco story from Source 1 passed on to add semantic associations Consults Knowledge Base for Cisco ’s competition Returns result: Lucent is a competitor of Cisco Lucent story from external feeds picked for publishing as “semantically related” to Cisco story – passed on to Dashboard Story on Lucent Story on Cisco XCM-compliant metadata, XML or other format Semantic Application ASP/Enterprise hosted Extractor Agent 1 Extractor Agent 2 Extractor Agent 3 Metadata centric Content Management Architecture

Related Stock News Semantic Web – Intelligent Content Industry News Technology Products COMPANY EPA Regulations Competition COMPANIES in Same or Related INDUSTRY COMPANIES in INDUSTRY with Competing PRODUCTS Impacting INDUSTRY or Filed By COMPANY Important to INDUSTRY or COMPANY Intelligent Content = What You Asked for + What you need to know! SEC

led by Same entity Human-assisted inference Knowledge-based & Manual Associations Syntax Metadata Semantic Metadata

Blended Semantic Browsing and Querying (Intelligence Analyst Workbench)

Innovations that affect User Experience
BSBQ: Blended Semantic Browsing and Querying
Ability to query and browse relevant desired content in a highly contextual manner
Seamless access/processing of Content, Metadata and Knowledge
Ability to retrieve relevant content, view related metadata, access relevant knowledge and switch between all the above, allowing user to follow his train of thought
dACE: dynamic Automatic Content Enhancement
Ability to provide enhanced annotation features, allowing the user to retrieve relevant knowledge about significant pieces of content during content consumption
Semantic Engine APIs with XML output
Ability to create customized APIs for the Semantic Engine involving Semantic Associations with XML output to cater to any user application

Visionics AcSys Security Portal Check-in Interrogation Boarding Gate Airport Airspace Voquette Knowledgebase Metabase Threat Scoring Gov’t Watchlists News Media Web Info LexisNexis RiskWise Passenger Records Reservation Data Airline Data Airport Data Airline and Airport Data Future and Current Risks Airport LEO ARC AvSec Manager Data Management Data Mining IPG

Sources Used Content Sources : Africa News Service AFX News – Asia/UK/Europe AP Worldstream Asia Pulse BusinessWire ComputerWire (CTW) EFE News Services FWN Select Itar-TASS Knight Ridder News (Open) Knight-Ridder Open M2 - International M2 Airline Industry Information New World Publishing PR Newswire PRLine (PRL) Resource News International RosBusiness United Press International UPI Spotlights Knowledge Sources: FBI - Most Wanted Terrorists Denied Persons Lists Terrorism Files ICT Office of Foreign Asset Control (OFAC) Hamas terrorists CNN Locations FAA_Airport_Codes About.com Comtex_International Hindustan Times JerusalemPost CNN Newstrove_Hamas

Voquette’s Semantic Technology enables flight authorities to : - take a quick look at the passenger’s history - check quickly if the passenger is on any official watchlist - interpret and understand passenger’s links to other organizations (possibly terrorist) - verify if the passenger has boarded the flight from a “high risk” region - verify if the passenger originally belongs to a “high risk” region - check if the passenger’s name has been mentioned in any news article along with the name of a known bad guy Interrogation Kiosk – Unique Advantages of Voquette Smith John

Threat Score Components Smith John WATCHLIST ANALYSIS Action : Voquette’s rich knowledgebase is automatically searched for the possible appearance of this name on any of the watchlists Ability Proven : Ability to automatically aggregate relevant rich domain knowledge and automatically co-relate it and rank the threat factors to indicate threat level of the passenger on the watchlist front METABASE SEARCH Action : Voquette’s rich metabase is searched for this name and associated content stories mentioning the passenger’s name are retrieved Ability Proven : Ability to automatically aggregate and retrieve relevant content stories, field reports, etc. about the passenger that can be used by flight officials to determine if the passenger has any connections with known bad people or organizations appearsOn watchList : FBI KNOWLEDGEBASE SEARCH Action : Voquette’s rich knowledgebase is searched for this name and associated information like position, aliases, relationships (past or present) of this name to other organizations, watchlists, country, etc. are retrieved Ability Proven : Ability to automatically aggregate relevant rich domain knowledge about a passenger and automatically co-relate it with other data in the knowledgebase to present a visual association picture to the flight official LEXIS NEXIS ANNOTATION Action : Information about or related to the passenger returned by Lexis Nexis is enhanced by linking important entities to Voquette’s rich knowledgebase Ability Proven : Ability to automatically aggregate relevant rich domain knowledge, recognize entities in a piece of text and further automatically co-relate it with other data in the knowledgebase to present a clear picture about the passenger to the flight official Flight Coutry Check 45 0.15 Person Country Check 25 0.15 Nested Organizations Check 75 0.8 Aggregate Link Analysis Score: 17.7 LINK ANALYSIS Action : Semantic analysis of the various components (watchlist, Lexis Nexis, knowledgebase search, metabase search, etc.) to come up with an aggregate threat score for the passenger Ability Proven : Ability to automatically aggregate relevant rich domain knowledge, recognize entities in a piece of text, automatically co-relate it with other data in the knowledgebase, search for relevant content to present an overall idea of the threat level fo the passenger, allowing him to take quick action

Query Comparison: Voquette vs. RDBMS

JIVA Semantic Console Start-up Interface The mission of the JIVA project is to gather and analyze as much information of diverse kinds about suspected individuals, terrorist and other groups, organizations, events, etc. For this Terrorism domain, the JIVA Semantic Console provides an information retrieval interface (shown below) that displays some fundamental semantic attributes (based on a corresponding Terrorism domain model) to enable information retrieval in the right context. Most fundamental semantic attributes specific to the Terrorism domain (fully customizable) Syntactic or domain-independent attributes for general and media-specific search Analyst can enter search values in the appropriate attribute fields (to search in the right context) Analyst can choose the type of media of the desired content Once all other values are set, click the “ Search” button to search semantically Search interface with more search features (explained later) JIVA Functionality Interface

“ Complete Picture” View – Knowledgebase Results This section of the ‘Complete Picture’ shows factually known real-world information about the entity (person, organization, event, etc.) of interest along with its contextual classification(s) and relationships with other entities in the Knowledgebase, to provide a comprehensive overview of the entity. Such knowledge is kept up-to-date by means of automated knowledge extractor agents that aggregate such knowledge about millions of entities from various trusted knowledge sources. Entity’s canonical name Entity’s classifications in taxonomy Entity’s aliases and other names
Entity’s real-world
relationships to various
other entities across
multiple entity classes
(as defined in the
Terrorism domain model)
Individual related
entities are clickable
to navigate to a new
knowledge page for
that entity e.g. Al Qaeda
Knowledgebase
Navigation
While browsing through relevant knowledge, analyst can search for content on the focal entity or any of the related entities. The analyst can also search for specific relationships between two or more entities by checking corresponding entity boxes for search - Blended Semantic Browsing & Querying (BSBQ) Fraud investigation of focal entity placing it in one of five levels of threats, based on score JIVA

Facilitating Knowledge Discovery On clicking any bin Laden-related entity (e.g. Al Qaeda), a page is displayed to the analyst showing knowledge pertaining to that entity, which can be used in a BSBQ mode, as described on the previous screen. Continuing this integrated approach of Semantic Browsing and Querying, the analyst has the necessary ammunition to perform Knowledge Discovery . The analyst can follow his train of thought as he browses and queries to possibly discover unexpected relationships and links between entities at various levels in an indirect manner. Automatically uncovering such hidden related entities facilitates addition of new and meaningful entities and relationships to the analyst’s assessment tasks. JIVA

Wireless Application of Semantic Metadata and Automatic Content Enrichment  Clicking on the link for Cisco Analyst Calls displays a listing sorted by date. Semantic filtering uses just the right metadata to meet screen and other constrains. E.g., Analyst Call focuses on the source and analyst name or company. The icon denote additional metadata, such as “Strong Buy” by H&Q Analyst. MyStocks News Sports Music MyMedia    $  My Stocks CSCO NT IBM Market CSCO Analyst Call Conf Call Earnings    11/08 ON24 Payne 11/07 ON24 H&Q  11/06 CBS Langlesis CSCO Analysis

Scene Description Tree Retrieve Scene Description Track “ NSF Playoff” Node Enhanced XML Description MPEG-2/4/7 Enhanced Digital Cable Video MPEG Encoder MPEG Decoder Node = AVO Object Voqutte/Taalee Semantic Engine
Produced by: Fox Sports
Creation Date: 12/05/2000
League: NFL
Teams: Seattle Seahawks,
Atlanta Falcons
Players: John Kitna
Coaches: Mike Holmgren,
Dan Reeves
Location: Atlanta
Object Content Information (OCI) Metadata-rich Value-added Node Create Scene Description Tree  GREAT USER EXPERIENCE Metadata’s role in emerging iTV infrastructure Channel sales through Video Server Vendors, Video App Servers, and Broadcasters License metadata decoder and semantic applications to device makers “ NSF Playoff”

Metadata for Automatic Content Enrichment Interactive Television This segment has embedded or referenced metadata that is used by personalization application to show only the stocks that user is interested in. This screen is customizable with interactivity feature using metadata such as whether there is a new Conference Call video on CSCO. Part of the screen can be automatically customized to show conference call specific information– including transcript, participation, etc. all of which are relevant metadata Conference Call itself can have embedded metadata to support personalization and interactivity.

Future
Multimodal interfaces
Multimodal semantics
Multivalent Semantics

Metadata Usage: Keyword, Attribute and Content Based Access The VisualHarness system at LSDIS/UGA

SEMANTIC CONTENT MANAGEMENT FOR ENTERPRISES AND NATIONAL SECURITY

by Amit Sheth

Accessibility

Categories

Upload Details

Usage Rights

Statistics