Enterprise Mashup Infrastructure Andreas Krohn Product Manager Kapow Technologies

Agenda What is a Mashup The Business Value of Mashups Mashup Infrastructure Enterprise Mashup Examples Demo

What is a Mashup

The Business Value of Mashups

Mashup Infrastructure

Enterprise Mashup Examples

Demo

Introduction Why is Excel so popular inside Enterprises? It allows the business users to look at data they way they want to It give the business users the ability to build mini-applications as needed without support from central IT Why is data integration and SOA so important inside Enterprises? Sharing data between systems increases the overall value of that data Together several systems can solve more problems than the systems can individually Mashups combines and simplifies all this!

Why is Excel so popular inside Enterprises?

It allows the business users to look at data they way they want to

It give the business users the ability to build mini-applications as needed without support from central IT

Why is data integration and SOA so important inside Enterprises?

Sharing data between systems increases the overall value of that data

Together several systems can solve more problems than the systems can individually

Mashups combines and simplifies all this!

Wikipedia Mashup (or mash it up) is a Jamaican Creole term meaning to destroy. Mashup (web application hybrid), a web application that combines data and/or functionality from more than one source Mashup (music), a musical genre of songs that consist entirely of parts of other songs Mashup (video), a video that is edited from more than one source to appear as one Mashup , in parts of the UK means a mash or pot of tea (colloq. Yorkshire), other areas brew or stand tea

Mashup (or mash it up) is a Jamaican Creole term meaning to destroy.

Mashup (web application hybrid), a web application that combines data and/or functionality from more than one source

Mashup (music), a musical genre of songs that consist entirely of parts of other songs

Mashup (video), a video that is edited from more than one source to appear as one

Mashup , in parts of the UK means a mash or pot of tea (colloq. Yorkshire), other areas brew or stand tea

Mashup: Web Application Hybrid Combines content from several sources via services Focus on services not software Often a Mashup is built without support from the providers of the services The Web as a Plattform Most mashups on the web today are simple A website + Google Maps Enterprise mashups Intranet apps + Internet sites + … You are already using Mashups! Excel spreadsheets Ad-hoc reporting Data Migration Content Aggregation Management Dashboards

Combines content from several sources via services

Focus on services not software

Often a Mashup is built without support from the providers of the services

The Web as a Plattform

Most mashups on the web today are simple

A website + Google Maps

Enterprise mashups

Intranet apps + Internet sites + …

You are already using Mashups!

Excel spreadsheets

Ad-hoc reporting

Data Migration

Content Aggregation

Management Dashboards

Data: The Emerging Productivity Driver Transformational Tasks (20%) Transactional Tasks (40%) Tacit-Oriented Tasks (40%) Individualized Collaborative Structured Unstructured Task-Intensive Data-Intensive IT-Controlled Do-it-Yourself Productivity Potential Low High

Mashups: The Next IT Productivity Wave Huge untapped productivity potential in knowledge workers Strategic corporate data not being used effectively Application controls & restricts data generation and flow Central IT only supports key business processes, not ad-hoc information resources There is a need for lightweight access to corporate data The right data, to the right people, at the right time Logical extension of SOA investment to the web tier

Huge untapped productivity potential in knowledge workers

Strategic corporate data not being used effectively

Application controls & restricts data generation and flow

Central IT only supports key business processes, not ad-hoc information resources

There is a need for lightweight access to corporate data

The right data, to the right people, at the right time

Logical extension of SOA investment to the web tier

The Long Tail of Projects Using traditional approaches just the most important projects can be implemented With Mashups the long tail of projects can be implemented # Users # Projects

Enterprise Mashup Infrastructure Enable Enterprise Mashups Enterprise – up time, maintainance, security, load balancing etc Leverage existing infrastructure Existing databases, servers etc Fit into existing choice of technology Extend existing infrastructure Complementing SOA Improved access to existing data sources (web sites for example) Major components Mashup Builders Mashup Enablers

Enable Enterprise Mashups

Enterprise – up time, maintainance, security, load balancing etc

Leverage existing infrastructure

Existing databases, servers etc

Fit into existing choice of technology

Extend existing infrastructure

Complementing SOA

Improved access to existing data sources (web sites for example)

Major components

Mashup Builders

Mashup Enablers

Mashup Builders Produce the user interface of a Mashup Today this is mostly done by programming Java, C#, Ruby on Rails. etc The trend is towards doing this by assembling Mashups by non-developers Connecting widgets to create composite applications Examples Excel BEA AquaLogic Pages IBM QEDWiki Yahoo! Pipes Need access to data!

Produce the user interface of a Mashup

Today this is mostly done by programming

Java, C#, Ruby on Rails. etc

The trend is towards doing this by assembling

Mashups by non-developers

Connecting widgets to create composite applications

Examples

Excel

BEA AquaLogic Pages

IBM QEDWiki

Yahoo! Pipes

Need access to data!

So, Where Is The Data? Web Based Content & Data MS Office Doc’s, PDF, email RDMS, Flat files Examples Web Based (Unstructured) File system (Unstructured) Database (Structured) Page Views Document Retrieval SQL Queries Data Access Method Full Page Document Level Only Data Element Access Granularity High Med Low Knowledge Worker Productivity Potential No Proprietary Yes Programmatic API’s Exploding Exploding Modest Growth Trend 80%-85% of data 15%-20% of data Enterprise Scope

Access the Data: Mashup Enablers Access the unstructured data Webscraping – access all internal and external websites Use HTML as an API! Serve functionality to Mashup Builders REST, RSS, Atom etc Makes internal and external resources available Examples: Kapow Mashup Server Openkapow Feed43

Access the unstructured data

Webscraping – access all internal and external websites

Use HTML as an API!

Serve functionality to Mashup Builders

REST, RSS, Atom etc

Makes internal and external resources available

Examples:

Kapow Mashup Server

Openkapow

Feed43

Inside a Mashup Mashup Mashup Builder REST Mashup Enabler Atom RSS API Web Scraping HTML AJAX API SQL Users Data Sources RPC JMS API WS* WS* DB Web Apps SQL WS* Widget Widget Widget

Enabling Mashups by giving access to unstructured web data Handles unstructured HTML Handles JavaScript Handles cookies, sessions, authentication, HTTPs etc Making data & functionality from HTML available as needed REST Service, RSS/Atom feed, Java API, C# API, Portlets, SOAP WS, Written to a database etc Automating what a person can do in a browser = Robot 250+ customers Audi, Bank of America, Simply Hired etc Openkapow.com Free Mashup Community Build robots and share with the Community Create REST services and RSS/Atom feeds from web sites Kapow Mashup Server

Enabling Mashups by giving access to unstructured web data

Handles unstructured HTML

Handles JavaScript

Handles cookies, sessions, authentication, HTTPs etc

Making data & functionality from HTML available as needed

REST Service, RSS/Atom feed, Java API, C# API, Portlets, SOAP WS, Written to a database etc

Automating what a person can do in a browser = Robot

250+ customers

Audi, Bank of America, Simply Hired etc

Openkapow.com

Free Mashup Community

Build robots and share with the Community

Create REST services and RSS/Atom feeds from web sites

Access the functionality of a web app without an API Handles multi-page navigation Does it with great stability and robustness Create RSS/Atom feeds, REST services from almost any web site. Advanced JavaScript handling and execution. Handles logins to protected sites using cookies, HTTP & form authentication, HTTPs. Very powerful data extraction and HTML interaction using regular expressions, converters and patterns. Flexible error handling and debugging. Graphical point-and-click development environment with 1-click deployment. Full control over the process flow in a robot, including conditions and loops Highlighted Features

Access the functionality of a web app without an API

Handles multi-page navigation

Does it with great stability and robustness

Create RSS/Atom feeds, REST services from almost any web site.

Advanced JavaScript handling and execution.

Handles logins to protected sites using cookies, HTTP & form authentication, HTTPs.

Very powerful data extraction and HTML interaction using regular expressions, converters and patterns.

Flexible error handling and debugging.

Graphical point-and-click development environment with 1-click deployment.

Full control over the process flow in a robot, including conditions and loops

Types of Enterprise Mashups Opportunistic Applications Situational Applications Mashups-on-the-fly Repurposing Portal Enablement Mobilising Applications Web Automation Content Migration Swivel-chair automation Data Collection Competative Intelligence Reputation Management

Opportunistic Applications

Situational Applications

Mashups-on-the-fly

Repurposing

Portal Enablement

Mobilising Applications

Web Automation

Content Migration

Swivel-chair automation

Data Collection

Competative Intelligence

Reputation Management

Opportunistic Applications Addressing the Long Tail Allowing individuals, small groups or departments to build there on applications Solving small problems not worth the investment from central IT Taken together these small apps increase productivity Short lifespan Throw-away apps Apps owned by business users, infrastructure owned by central IT Self-Service IT

Addressing the Long Tail

Allowing individuals, small groups or departments to build there on applications

Solving small problems not worth the investment from central IT

Taken together these small apps increase productivity

Short lifespan

Throw-away apps

Apps owned by business users, infrastructure owned by central IT

Self-Service IT

Opportunistic Applications: Audi Many integration problems Built a platform to be able to quickly solve new problems Applications driven by the Business Side Central IT provides the infrastructure Using Kapow Mashup Server to provide lightweight SOA enablement (among other things) Aggregate content and services from internal and external sources

Many integration problems

Built a platform to be able to quickly solve new problems

Applications driven by the Business Side

Central IT provides the infrastructure

Using Kapow Mashup Server to provide lightweight SOA enablement (among other things)

Aggregate content and services from internal and external sources

Repurposing Taking existing applications and repurposing them for different platforms and/or use cases No need to change the code in the existing applications Avoid costly implementation projects Avoid risking new security issues Interact with existing applications via Mashup Enablers & Existing Infrastructure Service enable Portlet generation for Enterprise Portals Common use cases Portal enablement Mobilization

Taking existing applications and repurposing them for different platforms and/or use cases

No need to change the code in the existing applications

Avoid costly implementation projects

Avoid risking new security issues

Interact with existing applications via Mashup Enablers & Existing Infrastructure

Service enable

Portlet generation for Enterprise Portals

Common use cases

Portal enablement

Mobilization

Repurposing: Deutsche Post World Net 20+ business units, each with it’s own unique portal DHL Mailingfactory eFiliale etc New Deutsche Post portal with features from all business units Mydeutschepost.de Built by repurposing the existing portals Take the features from each portal as needed Package it into one united portal The original portals are still up and running

20+ business units, each with it’s own unique portal

DHL

Mailingfactory

eFiliale

etc

New Deutsche Post portal with features from all business units

Mydeutschepost.de

Built by repurposing the existing portals

Take the features from each portal as needed

Package it into one united portal

The original portals are still up and running

Web Automation Automate jobs that a person could perform Improved efficiency Improved quality Improved reliability Common use cases Automated Content Migration (as oppose to ”bio-migration”) Swivel chair automation

Automate jobs that a person could perform

Improved efficiency

Improved quality

Improved reliability

Common use cases

Automated Content Migration (as oppose to ”bio-migration”)

Swivel chair automation

Web Automation: GMX GMX.net Major hosting company, 1&1 hosting Free Email provider Automating integration with other online email providers Via the web interface check email Synchronize address books etc

GMX.net

Major hosting company, 1&1 hosting

Free Email provider

Automating integration with other online email providers

Via the web interface check email

Synchronize address books

etc

Data Collection Collect and structure large amounts of unstructured data Data sources both on internet and intranet Add value by adding custom analyzis or visualizastion to the data Common use cases: Competative Intelligence Information Syndication Reputation Management

Collect and structure large amounts of unstructured data

Data sources both on internet and intranet

Add value by adding custom analyzis or visualizastion to the data

Common use cases:

Competative Intelligence

Information Syndication

Reputation Management

Data Collection: Simply Hired Job search engine Collects jobs from other job sites (Monster.com) as well as from individual companies pages No need for the job sites to provide an API, all data collected via web pages Set industry record Collected 1 million job profiles in 3 months 5 million job profiles in 2 years Provide job search both on SimplyHired.com and on 3rd party sites LinkedIn MySpace

Job search engine

Collects jobs from other job sites (Monster.com) as well as from individual companies pages

No need for the job sites to provide an API, all data collected via web pages

Set industry record

Collected 1 million job profiles in 3 months

5 million job profiles in 2 years

Provide job search both on SimplyHired.com and on 3rd party sites

LinkedIn

MySpace

Demo Scenario – Competitive Intelligence Web Based Pricing Data

Questions? www.kapowtech.com www.openkapow.com More information at the Kapow Technologies booth