Enterprise Mashup Infrastructure Andreas Krohn Product Manager Kapow Technologies
Agenda What is a Mashup The Business Value of Mashups Mashup Infrastructure Enterprise Mashup Examples Demo
Introduction Why is Excel so popular inside Enterprises? It allows the business users to look at data they way  they  want to It give the business users the ability to  build mini-applications  as needed without support from central IT Why is data integration and SOA so important inside Enterprises? Sharing data  between systems increases the overall value of that data Together several systems can  solve more problems  than the systems can individually Mashups combines and simplifies all this!
Wikipedia Mashup  (or mash it up) is a Jamaican Creole term meaning to   destroy.  Mashup  (web application hybrid), a web application that combines data and/or functionality from more than one source  Mashup  (music), a musical genre of songs that consist entirely of parts of other songs  Mashup  (video), a video that is edited from more than one source to appear as one  Mashup , in parts of the UK means a mash or pot of tea (colloq. Yorkshire), other areas brew or stand tea
Mashup: Web Application Hybrid Combines content from several sources via services Focus on services not software Often a Mashup is built without support from the providers of the services The Web as a Plattform Most mashups on the web today are simple A website + Google Maps Enterprise mashups Intranet apps + Internet sites + …  You are already using Mashups! Excel spreadsheets Ad-hoc reporting Data Migration Content Aggregation Management Dashboards
Data: The Emerging Productivity Driver Transformational Tasks (20%) Transactional Tasks (40%) Tacit-Oriented Tasks (40%) Individualized Collaborative Structured Unstructured Task-Intensive Data-Intensive IT-Controlled Do-it-Yourself Productivity Potential Low High
Mashups: The Next IT Productivity Wave Huge untapped productivity potential in knowledge workers  Strategic corporate data not being used effectively Application controls & restricts data generation and flow Central IT only supports key business processes, not ad-hoc information resources There is a need for lightweight access to corporate data The right data, to the right people, at the right time Logical extension of SOA investment to the web tier
The Long Tail of Projects Using traditional approaches just the most important projects can be implemented With Mashups the long tail of projects can be implemented # Users # Projects
Enterprise Mashup Infrastructure Enable Enterprise Mashups Enterprise – up time, maintainance, security, load balancing etc Leverage existing infrastructure Existing databases, servers etc Fit into existing choice of technology Extend existing infrastructure Complementing SOA Improved access to existing data sources (web sites for example)  Major components Mashup Builders Mashup Enablers
Mashup Builders Produce the user interface of a Mashup Today this is mostly done by programming Java, C#, Ruby on Rails. etc The trend is towards doing this by assembling Mashups by non-developers Connecting widgets to create composite applications Examples Excel BEA AquaLogic Pages IBM QEDWiki Yahoo! Pipes Need access to data!
So, Where Is The Data? Web Based Content & Data MS Office Doc’s, PDF, email RDMS, Flat files Examples Web Based (Unstructured) File system (Unstructured) Database (Structured) Page Views Document Retrieval SQL Queries Data Access Method Full Page Document Level Only Data Element Access Granularity High Med Low Knowledge Worker Productivity Potential No Proprietary Yes Programmatic API’s Exploding Exploding Modest Growth Trend 80%-85% of data 15%-20% of data Enterprise Scope
Access the Data: Mashup Enablers Access the unstructured data Webscraping – access all internal and external websites Use HTML as an API! Serve functionality to Mashup Builders REST, RSS, Atom etc Makes internal and external resources available Examples: Kapow Mashup Server Openkapow Feed43
Inside a Mashup Mashup Mashup Builder REST Mashup  Enabler Atom RSS API Web Scraping HTML AJAX API SQL Users Data Sources RPC JMS API WS* WS* DB Web Apps SQL WS* Widget Widget Widget
Enabling Mashups by giving access to unstructured web data Handles unstructured HTML Handles JavaScript Handles cookies, sessions, authentication, HTTPs etc Making data & functionality from HTML available as needed REST Service, RSS/Atom feed, Java API, C# API, Portlets, SOAP WS, Written to a database etc Automating what a person can do in a browser = Robot 250+ customers Audi, Bank of America, Simply Hired etc Openkapow.com Free Mashup Community Build robots and share with the Community Create REST services and RSS/Atom feeds from web sites Kapow Mashup Server
Access the functionality of a web app without an API Handles multi-page navigation Does it with great stability and robustness Create RSS/Atom feeds, REST services from almost any web site. Advanced JavaScript handling and execution.  Handles logins to protected sites using cookies, HTTP & form authentication, HTTPs. Very powerful data extraction and HTML interaction using regular expressions, converters and patterns. Flexible error handling and debugging. Graphical point-and-click development environment  with 1-click deployment. Full control over the process flow in a robot, including conditions and loops Highlighted Features
Types of Enterprise Mashups Opportunistic Applications Situational Applications Mashups-on-the-fly Repurposing Portal Enablement Mobilising Applications Web Automation Content Migration Swivel-chair automation Data Collection Competative Intelligence Reputation Management
Opportunistic Applications Addressing the Long Tail Allowing individuals, small groups or departments to build there on applications Solving small problems not worth the investment from central IT Taken together these small apps increase productivity Short lifespan Throw-away apps Apps owned by business users, infrastructure owned by central IT Self-Service IT
Opportunistic Applications: Audi Many integration problems Built a platform to be able to quickly solve new problems Applications driven by the Business Side Central IT provides the infrastructure Using Kapow Mashup Server to provide lightweight SOA enablement (among other things) Aggregate content and services from internal and external sources
Repurposing Taking existing applications and repurposing them for different platforms and/or use cases No need to change the code in the existing applications Avoid costly implementation projects Avoid risking new security issues Interact with existing applications via Mashup Enablers & Existing Infrastructure Service enable Portlet generation for Enterprise Portals Common use cases Portal enablement Mobilization
Repurposing: Deutsche Post World Net 20+ business units, each with it’s own unique portal DHL Mailingfactory eFiliale etc New Deutsche Post portal with features from all business units Mydeutschepost.de Built by repurposing the existing portals Take the features from each portal as needed Package it into one united portal The original portals are still up and running
Web Automation Automate jobs that a person could perform Improved efficiency Improved quality Improved reliability Common use cases Automated Content Migration (as oppose to ”bio-migration”) Swivel chair automation
Web Automation: GMX GMX.net Major hosting company, 1&1 hosting Free Email provider Automating integration with other online email providers Via the web interface check email Synchronize address books  etc
Data Collection Collect and structure large amounts of unstructured data Data sources both on internet and intranet Add value by adding custom analyzis or visualizastion to the data Common use cases: Competative Intelligence Information Syndication Reputation Management
Data Collection: Simply Hired Job search engine Collects jobs from other job sites (Monster.com) as well as from individual companies pages No need for the job sites to provide an API, all data collected via web pages Set industry record Collected 1 million job profiles in 3 months 5 million job profiles in 2 years Provide job search both on SimplyHired.com and on 3rd party sites LinkedIn MySpace
Demo Scenario – Competitive Intelligence Web Based Pricing Data
Questions? www.kapowtech.com www.openkapow.com More information at the Kapow Technologies booth

Enterprise Mashup Infrastructure Kapow Mashup Server

  • 1.
    Enterprise Mashup InfrastructureAndreas Krohn Product Manager Kapow Technologies
  • 2.
    Agenda What isa Mashup The Business Value of Mashups Mashup Infrastructure Enterprise Mashup Examples Demo
  • 3.
    Introduction Why isExcel so popular inside Enterprises? It allows the business users to look at data they way they want to It give the business users the ability to build mini-applications as needed without support from central IT Why is data integration and SOA so important inside Enterprises? Sharing data between systems increases the overall value of that data Together several systems can solve more problems than the systems can individually Mashups combines and simplifies all this!
  • 4.
    Wikipedia Mashup (or mash it up) is a Jamaican Creole term meaning to destroy. Mashup (web application hybrid), a web application that combines data and/or functionality from more than one source Mashup (music), a musical genre of songs that consist entirely of parts of other songs Mashup (video), a video that is edited from more than one source to appear as one Mashup , in parts of the UK means a mash or pot of tea (colloq. Yorkshire), other areas brew or stand tea
  • 5.
    Mashup: Web ApplicationHybrid Combines content from several sources via services Focus on services not software Often a Mashup is built without support from the providers of the services The Web as a Plattform Most mashups on the web today are simple A website + Google Maps Enterprise mashups Intranet apps + Internet sites + … You are already using Mashups! Excel spreadsheets Ad-hoc reporting Data Migration Content Aggregation Management Dashboards
  • 6.
    Data: The EmergingProductivity Driver Transformational Tasks (20%) Transactional Tasks (40%) Tacit-Oriented Tasks (40%) Individualized Collaborative Structured Unstructured Task-Intensive Data-Intensive IT-Controlled Do-it-Yourself Productivity Potential Low High
  • 7.
    Mashups: The NextIT Productivity Wave Huge untapped productivity potential in knowledge workers Strategic corporate data not being used effectively Application controls & restricts data generation and flow Central IT only supports key business processes, not ad-hoc information resources There is a need for lightweight access to corporate data The right data, to the right people, at the right time Logical extension of SOA investment to the web tier
  • 8.
    The Long Tailof Projects Using traditional approaches just the most important projects can be implemented With Mashups the long tail of projects can be implemented # Users # Projects
  • 9.
    Enterprise Mashup InfrastructureEnable Enterprise Mashups Enterprise – up time, maintainance, security, load balancing etc Leverage existing infrastructure Existing databases, servers etc Fit into existing choice of technology Extend existing infrastructure Complementing SOA Improved access to existing data sources (web sites for example) Major components Mashup Builders Mashup Enablers
  • 10.
    Mashup Builders Producethe user interface of a Mashup Today this is mostly done by programming Java, C#, Ruby on Rails. etc The trend is towards doing this by assembling Mashups by non-developers Connecting widgets to create composite applications Examples Excel BEA AquaLogic Pages IBM QEDWiki Yahoo! Pipes Need access to data!
  • 11.
    So, Where IsThe Data? Web Based Content & Data MS Office Doc’s, PDF, email RDMS, Flat files Examples Web Based (Unstructured) File system (Unstructured) Database (Structured) Page Views Document Retrieval SQL Queries Data Access Method Full Page Document Level Only Data Element Access Granularity High Med Low Knowledge Worker Productivity Potential No Proprietary Yes Programmatic API’s Exploding Exploding Modest Growth Trend 80%-85% of data 15%-20% of data Enterprise Scope
  • 12.
    Access the Data:Mashup Enablers Access the unstructured data Webscraping – access all internal and external websites Use HTML as an API! Serve functionality to Mashup Builders REST, RSS, Atom etc Makes internal and external resources available Examples: Kapow Mashup Server Openkapow Feed43
  • 13.
    Inside a MashupMashup Mashup Builder REST Mashup Enabler Atom RSS API Web Scraping HTML AJAX API SQL Users Data Sources RPC JMS API WS* WS* DB Web Apps SQL WS* Widget Widget Widget
  • 14.
    Enabling Mashups bygiving access to unstructured web data Handles unstructured HTML Handles JavaScript Handles cookies, sessions, authentication, HTTPs etc Making data & functionality from HTML available as needed REST Service, RSS/Atom feed, Java API, C# API, Portlets, SOAP WS, Written to a database etc Automating what a person can do in a browser = Robot 250+ customers Audi, Bank of America, Simply Hired etc Openkapow.com Free Mashup Community Build robots and share with the Community Create REST services and RSS/Atom feeds from web sites Kapow Mashup Server
  • 15.
    Access the functionalityof a web app without an API Handles multi-page navigation Does it with great stability and robustness Create RSS/Atom feeds, REST services from almost any web site. Advanced JavaScript handling and execution. Handles logins to protected sites using cookies, HTTP & form authentication, HTTPs. Very powerful data extraction and HTML interaction using regular expressions, converters and patterns. Flexible error handling and debugging. Graphical point-and-click development environment with 1-click deployment. Full control over the process flow in a robot, including conditions and loops Highlighted Features
  • 16.
    Types of EnterpriseMashups Opportunistic Applications Situational Applications Mashups-on-the-fly Repurposing Portal Enablement Mobilising Applications Web Automation Content Migration Swivel-chair automation Data Collection Competative Intelligence Reputation Management
  • 17.
    Opportunistic Applications Addressingthe Long Tail Allowing individuals, small groups or departments to build there on applications Solving small problems not worth the investment from central IT Taken together these small apps increase productivity Short lifespan Throw-away apps Apps owned by business users, infrastructure owned by central IT Self-Service IT
  • 18.
    Opportunistic Applications: AudiMany integration problems Built a platform to be able to quickly solve new problems Applications driven by the Business Side Central IT provides the infrastructure Using Kapow Mashup Server to provide lightweight SOA enablement (among other things) Aggregate content and services from internal and external sources
  • 19.
    Repurposing Taking existingapplications and repurposing them for different platforms and/or use cases No need to change the code in the existing applications Avoid costly implementation projects Avoid risking new security issues Interact with existing applications via Mashup Enablers & Existing Infrastructure Service enable Portlet generation for Enterprise Portals Common use cases Portal enablement Mobilization
  • 20.
    Repurposing: Deutsche PostWorld Net 20+ business units, each with it’s own unique portal DHL Mailingfactory eFiliale etc New Deutsche Post portal with features from all business units Mydeutschepost.de Built by repurposing the existing portals Take the features from each portal as needed Package it into one united portal The original portals are still up and running
  • 21.
    Web Automation Automatejobs that a person could perform Improved efficiency Improved quality Improved reliability Common use cases Automated Content Migration (as oppose to ”bio-migration”) Swivel chair automation
  • 22.
    Web Automation: GMXGMX.net Major hosting company, 1&1 hosting Free Email provider Automating integration with other online email providers Via the web interface check email Synchronize address books etc
  • 23.
    Data Collection Collectand structure large amounts of unstructured data Data sources both on internet and intranet Add value by adding custom analyzis or visualizastion to the data Common use cases: Competative Intelligence Information Syndication Reputation Management
  • 24.
    Data Collection: SimplyHired Job search engine Collects jobs from other job sites (Monster.com) as well as from individual companies pages No need for the job sites to provide an API, all data collected via web pages Set industry record Collected 1 million job profiles in 3 months 5 million job profiles in 2 years Provide job search both on SimplyHired.com and on 3rd party sites LinkedIn MySpace
  • 25.
    Demo Scenario –Competitive Intelligence Web Based Pricing Data
  • 26.
    Questions? www.kapowtech.com www.openkapow.comMore information at the Kapow Technologies booth