Wouldn't it be nice if... an introduction to Enterprise Data Mashups

Wouldn’t it be nice if…? An Introduction to Enterprise Data Mashups at the School of Information, UC Berkeley Feb 11th, 2008 Justo N. Hidalgo VP Product Management and Consulting Denodo

… I could use information for all these sources without going crazy? CRM (MySQL) Billing System (Web Service - Rest) Incidences System (Web Application) Inventory System (MS SQL Server) Product Catalog (Web Service -SOAP) Knowledge Base (Internet) Product Data (CSV) Log files (.txt/.log files)

Extracción de Datos Web Estructuración y Navegación Automática. Estado del Arte y Propuesta Investigadora de Denodo
Justo N. Hidalgo, VP Product Management and Consulting
21 de Enero de 2009
… I could do it in real time, where required? CRM (MySQL) Billing System (Web Service - Rest) Incidences System (Web Application) Inventory System (MS SQL Server) Product Catalog (Web Service -SOAP) Knowledge Base (Internet) Product Data (CSV) Log files (.txt/.log files)

… I could do it in batch mode, where required?

… I could react rapidly to changes?

… I could do this without having to code it all?

… , finally, I could do this for actual ENTERPRISE PROBLEMS AND NEEDS???

Mashups are Crossing the Chasm

Agenda
Background
Enterprise Mashups
Web mashups and Enterprise Data Mashups
Some real-world examples
Research Areas

Denodo
Company
Founded 1999, Europe 2005, NA HQ in 2006, APAC SIs 2007, East, Midwest 2009
Profitable in Europe; Venture-backed global expansion
Operations
Americas & EMEA; VARs in Asia
90+ Customers – Telco, Fin Svcs, Energy, Health, Govt / Defense
Expanding Partners - Focused ISVs, SaaS, SI/VARs
Product (Denodo Platform 4.5 - Enterprise Hardened)
Enterprise Data Mashups & Web Automation
Data Integration of Structured, Web and Unstructured Information
Fast, Agile Deployments in under 30 days
Technology Focus: Data Integration, Web 2.0 & Mashup Enabler
Targeted Solutions: Intelligence, Single View, Web Automation
Tenths of projects in the last 18 months to learn from
Q4 2009: Denodo Platform 5.0

Justo
VP Product Management and Consulting, Denodo Technologies
Responsible for Product Management and Sales Engineering
Ph.D. Computer Science, Query Optimization in Mediators of Web Sources
Working in Enterprise Data Integration and Web Process Automation since 2000
jhidalgo at domain denodo dot com
If you read spanish: http://justohidalgo.blogspot.com
This presentation is available at:
http://www.slideshare.net/justohidalgo/

Enterprise Mashups

Enterprise Mashup Architecture, according to the Analysts

A little bit more detail

What, Enterprise Mashups are not for cool things? 1. Aggregate Data 3. Exchange Data Enterprise Suppliers Customers 2. Enrich Data Apps Data Services

Web and Data Mashups

Web Mashups vs. Data Mashups
Web Mashups
Visual centric
Data relationships are simple
End-user driven enabled by APIs / tools
No QoS guarantees – as is
Simple Web security
Data Mashups
Data & Information centric
Deep transformations & semantic relationships
Interact with Enterprise ecosystem
Business analyst driven w/ user control options
Enterprise QoS- Query Optimization, Reliable, Failover, etc.
Policy-based security
ENTERPRISE ECOSYSTEM Enterprise Infrastructure / Web Platform Enterprise & Web Applications RIA, Social Software, GUI, Portals

Mutual Benefits Web Mashups Data Mashups Enterprise Data Integration BPM ESB EAI ETL Data Warehouses Enterprise Infrastructure / Web Platform Enterprise & Web Applications RIA, Social Software, GUI, Portals ENTERPRISE ECOSYSTEM WEB MASHUPS DATA MASHUPS User experience / global access Enterprise-class features ?

Enterprise Areas

Mashups for Business Users / For Power Users and IT
User experience
Pipes&Filters, Data Federation
Heterogeneous granularity for different levels

Enterprise-class features…
Security
LDAP access, user and role permission levels, …
But also: Secure VPNs, secure communications, encryption, SSO
HA/Scalability
Load balancing
Federated and distributed architectures
Performance
Asynchronous access
Parallelism
Execution Plan Optimization techniques

… Enterprise-class features
Data Transformations
Textual similarity, contextual summaries, patterns and regular expressions
Data Management… don´t get me started 
What are the boundaries between mashups and MDM, Data Quality tools?
Standards and Integration
JSR-168, JMX, JMX, JDBC, ODBC, WS-*, microformats, … don´t you love standardization bodies?

Automatic Browsing and Structure
Maintenance
Sources change… specially Web Sites!!!
Automatic Extraction
Schema matching
Automatic wrapper induction
Automatic Browsing
Taxonomy-based path recognition
Form updates

Research
Architecture
A new Mashup Reference Architecture
Web Data Extraction
Automatic Web Data Extraction
Automatic Web Maintenance
Extraction Templates
Enterprise-class
Query Optimization Techniques

A few references
Information Structuring
Manuel Álvarez, Alberto Pan, Juan Raposo, Fernando Bellas, Fidel Cacheda. Finding and Extracting Data Records from Web Pages. Journal of Signal Processing Systems (Special issue on EUC 2007).
Manuel Álvarez, Alberto Pan, Juan Raposo, Fernando Bellas, Fidel Cacheda: Extracting lists of data records from semi-structured web pages. Data Knowl. Eng. 64(2): 491-509 (2008)
Manuel Álvarez, Alberto Pan, Juan Raposo, Fernando Bellas, Fidel Cacheda: Using Clustering and Edit Distance Techniques for Automatic Web Data Extraction. WISE 2007: 212-224
Juan Raposo, Alberto Pan, Manuel Álvarez, Justo Hidalgo: Automatically maintaining wrappers for semi-structured web sources. Data Knowl. Eng. 61(2): 331-358 (2007)
Focused Crawling
Manuel Álvarez, Juan Raposo, Alberto Pan, Fidel Cacheda, Fernando Bellas, Victor Carneiro: DeepBot: a focused crawler for accessing hidden web content. DEECS 2007: 18-25
Manuel Álvarez, Juan Raposo, Alberto Pan, Fidel Cacheda, Fernando Bellas, Victor Carneiro: Crawling the Content Hidden Behind Web Forms. ICCSA (2) 2007: 322-333
Cost Optimization
Justo Hidalgo, Alberto Pan, José Losada, Manuel Álvarez. Preloading Browsers for Optimizing Automatic Access to Hidden Web: a Ranking-based Repository Solution. ADBIS 2006. pp.171-183.
Justo Hidalgo, Alberto Pan, Manuel Álvarez, Jaime Guerrero. Efficiently Updating Cost Repository Values for Query Optimization on Web Data Sources in a Mediator/Wrapper Environment. NGITS 2006. Vol. 4032, pp. 1-12.

Q&A Justo Hidalgo VP Product Management and Consulting [email_address]

All images shown that are not property of Denodo come from the search engine search.creativecommons.org, that searches Flickr images with CreativeCommons license
Credits

Wouldn't it be nice if... an introduction to Enterprise Data Mashups

by Justo Hidalgo on Feb 11, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 50

Statistics

Wouldn’t it be nice if... an introduction to Enterprise Data Mashups — Presentation Transcript

http://justohidalgo.blogspot.com	40
http://www.slideshare.net	7
http://www.loscuentosdelabuelo.com	2
http://justohidalgo.blogspot.com:80	1