On Demand Int
Upcoming SlideShare
Loading in...5
×
 

On Demand Int

on

  • 161 views

 

Statistics

Views

Total Views
161
Views on SlideShare
161
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

On Demand Int On Demand Int Document Transcript

  • iWay Unites On-Demand Integration, Big Data forReal-Time AnalyticsiWay Software, a unit of Information Builders, is bringing together the worlds ofon-demand integration with Big Data. IDN explores how iWay has optimized itslong-standing any-to-any integration framework and connector architecture to make iteasier for IT to design and deploy solutions based on MapReduce, Hadoop and othernew data and analytics technologies.by Vance McCarthyTags: analytics, Big Data, Hadoop, integration, iWay, MapReduce, iWay Software, a unit ofInformation Builders, is bringing together the worlds of on-demand integration with Big Data.IDN speaks with iWay’s marketing director Vincent Lam to learn how iWay has optimized its long-standing any-to-anyintegration framework and connector architecture to make it easier for IT to design and deploy solutions based onMapReduce, Hadoop and other new data and analytics technologies.“We’re definitely seeing among our customers a trend where unstructured data with technologies such as Hadoop andMongoDB are becoming a bigger component of what companies are using, and these customers need those to worktogether with their RDMBS and other resources,” Lam told IDN. “Supporting these solutions within iWay was importantbecause to us, the whole crux of integration is to get all the things a customer needs to work together to worktogether.” “Our support for MapReduce allows iWay to federate the query [across distributed data
  • sources] very quickly.” Vincent Lam Marketing Director iWay SoftwareSupporting Hadoop, MongoDB and other new unstructured data sources was not a major problem for iWay’s corearchitecture, he added.“iWay has always used an agnostic approach to integration, where endpoints were always separated at the functionallayer. So, integrating with these new types of data only meant that we needed to make sure we had new adapters thatwould talk to these sources,” Lam said.That said, iWay also wanted to go one step further and make it easier for various IT professionals (from both the dataand the integration side) to design and deploy these types of solutions.“Because MapReduce and other Hadoop and ‘Big Data’ tools are so new, IT often needs to learn specialized skills.Even beyond training. Sometimes it’s also not clear how IT should address a project or even who should takeresponsibility,” Lam added. “That’s why we added MapReduce-caliber capabilities to its iWay Parallel ServiceManager.”The iWay Parallel Service Manager 6.1 adds MapReduce-style functionality to provide simplified tooling andout-of-the-box integration support to power Big Data solutions, such as “federated search” across unstructured datastores, as well as many popular ones now used by enterprises including Oracle, MUMPS, and HL7.Lam added, “For some customers, the ability to search across a large pool of physically distributed data sources andobtain near-real time results can be a monumental issue,” adding MapReduce allows iWay to federate the query veryquickly, supported by agents in the iWay Parallel Service Manager.The latest Parallel Service Manager also extends ETL with “parallel” extracts, transforms, and loads, as well tobatch-oriented managed file transfer (MFT) where the processing and distribution of files takes place in parallelUnder the covers, the iWay Parallel Service Manager will accelerate the processing of complex queries and/orcomputations by breaking them up and distributing them across multiple technology assets.It will perform these computing functions in parallel (rather than sequentially) because inside the iWay Parallel ServiceManager, “map” and “reduce” functions are done in a process flow, which is executed in parallel for each item in thelist. So, when a user makes a query that needs to access multiple data stores simultaneously, iWay launches a“parallel control agent,” which will simultaneously execute process flows configured with information for whatconnections are required based on information in the list. The outcome of each query can then be amassed into afinal results document.This approach, Lam said, will deliver key MapReduce-optimized benefits, without the need for IT complexity. Amongthe benefits are:● Reduce complexity of setting up and executing MapReduce-type federated queries● Seamless integration with the Hadoop Distributed File System● No need to write custom Java code to divide large queries across multiple systems (and then collect and aggregate the results)● A drag-and-drop interface to set up and execute parallel processing jobs● Pre-packaged integration components providing native access to 300+ backend source / target systemsThis iWay approach to bringing together Big Data and integration comes as Gartner predicts that real-time analytics
  • will explode for structured and unstructured data across distributed systems.“Analytics has become a major driving application for data warehousing, with the use of MapReduce outside andinside the DBMS, and the use of self-service data marts, according to Gartner’s just-rleeased report on the Top 10Technologies for 2012. “One major implication of big data is that in the future users will not be able to put all usefulinformation into a single data warehouse. Logical data warehouses bringing together information from multiplesources as needed will replace the single data warehouse model.“