Rapidly Evolving IT Trends Make Open, Agile Integration More Important than Ever
Rapidly Evolving IT Trends Make Open, Agile IntegrationMore Important than EverTranscript of a sponsored BrieﬁngsDirect podcast on the maturing of open source software andits role in making enterprises responsive to a changing landscape.Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Sponsor:FuseSourceDana Gardner: Hi. This is Dana Gardner, Principal Analyst at Interarbor Solutions, and youre listening to BrieﬁngsDirect. Today, we present a sponsored podcast discussion on how enterprise integration requirements are rapidly shifting to accommodate such trends as cloud computing, mobile devices explosion, and increased demand for extended enterprise business processes. [Disclosure: FuseSource is a sponsor of BrieﬁngsDirect podcasts.]Application-to-application integration inside an enterprises four walls is well understood, butvery quickly the demands placed on integration are spanning multiple enterprises, multiple typesof applications, and varieties of service providers.Software as a service (SaaS) and cloud computing are joining with legacy systems to form newand varied hybrid models that require whole new sets of integration needs and challenges.Once these newer breeds of integrations are set up, can the old, brittle management and upkeepof them sufﬁce or will agility and rapid upgrades and innovations require new tools to makeintegration a lifecycle function with ongoing management and more automated governance?In this discussion, well examine how open-source integration projects like Apache Camel andlightweight integration implementations and graphical tools are making developers and architectsmore agile. At the same time, these open-source approaches are proving less vulnerable to thecomplexity, fragility, and cost that often plague aging commercial middleware integrationproducts.More inclusive integration built on open-source infrastructure should encourage, rather thanhobble, the rapid innovation now needed of applications and data services, ecosystems, amongand between multiple businesses and from a variety of business service providers.Here to examine the new need for open and agile integration capabilities is Rob Davies. He is theChief Technology Ofﬁcer at FuseSource. Welcome, Rob.Rob Davies: Hi, Dana. Good to be here.
Gardner: Were also here with Debbie Moynihan. She is the Vice President of Marketing atFuseSource. Hello, Debbie.Debbie Moynihan: Hi, Dana.Gardner: Debbie, whats going on out there? Why are things happening so rapidly? Why dopeople need to rethink integration?Many challengesMoynihan: Dana, there are so many things happening out there -- integration, in particular. There are so many challenges right now. The business models are changing and people are being asked to do more with less. Teams and applications are more distributed than they have ever been. There are a lot of new technologies coming out that people are struggling to learn about, and ﬁguring out how to incorporate them into their infrastructure: cloud, mobile, the explosion of the huge amounts of data that enterprises are trying to understand and make sense out of. Not to mention the social media technologies that people are being asked about and wonderinghow to incorporate into their enterprise infrastructure.There are a lot of different skills that people are looking to have that theyve never been asked tohave before. More and more people are being asked to perform IT tasks. It isn’t just highlyskilled developers, but also business analysts and people who have never done integration beforeare being asked to do integration activities.A lot of people are looking for solutions and ideas. Theyre not sure how to keep up with all ofthese changes. Costs are a problem because essentially everyone has the same or smaller budgetgoing forward and a lot of people have fewer people to do what theyve been doing before.People are challenged. I think open source is a great solution. At FuseSource, weve seen a lotpeople looking more and more to open source to solve some of these problems.The reason why open source is a good solution is that with open source theres a lot of ﬂexibility.When the environment changes and new technologies come out, you need to integrate newthings into your environment.The community people, when they see a problem or new technology, just make it happen. Theycan add, expand, and modify whats involved in the various open-source integration projectswithout the overhead and bureaucracy of some of the traditional software developmentenvironments.Gardner: Debbie, in the past, when we had a shift in computing, wed bring in a new set ofapplications, wed update our platforms, and then think about integrating them based on the
needs. It was a sequential process and it could take three to ﬁve years to go through somethinglike that.We don’t really have that luxury anymore. Now things are happening in a simultaneous fashion.So integration really cant be an afterthought, but needs to be part and parcel with how you goabout designing and implementing your applications. Doesn’t open source, in a sense, allow for acompression of the time that we’ve traditionally taken with commercial product?Moynihan: Absolutely. Open-source is a componentized lightweight approach. As peopledevelop their applications, they develop them in such a way that they can be broken apart in newand different ways down the road, and its very transparent. It makes it easier over time to furtherintegrate what you’ve built and to make changes as you need to.Gardner: Rob Davies, lets dive a little deeper into this notion of open and agile integrationcapabilities. Whats wrong with simply going into traditional, commercial integration capabilitiesand somehow broadening them into this new domain? Is it not something that can be extended?The pace of changeDavies: I think it is, but I think the real crux of the problem goes back to what Debbie was talking about earlier -- the pace of change. If you’ve got an open-source framework, you can actually have an insight into how project works. After we launched Apache Camel at the Apache Software Foundation, we provided a number of default integration components for Camel. But, as soon as they got out there and the community started to use them and saw the beneﬁts of using them, we saw no end of contributions. People contributed adapters to weird and wonderful systems, and contributed them right back into the Apache project.Then we’ve got other components that people use to automate open-source but not at Apache. Anumber of components have grown rapidly since the inception of a project. When we started, wehad probably 20 components, and now its well over 100. Those are the ones we know about, theones that people have open-sourced.We know from our customers that they’ve got speciﬁc needs. They’ve got legacy applications.Because weve gone to the effort of making sure that its very easy to add a new component intoApache Camel, its very straightforward for someone to add in extra functionality.For example, if you want to write a component for legacy mainframe application, you could veryeasily do in a matter of hours. The old approach would take you weeks, months, maybe evenyears, especially if you don’t have access to the source code. So, you’ve got that addedﬂexibility.
The fact that its an open-source project at Apache means that there is a vibrant community ofusers and developers. You can get feedback instantly, if you’ve got issues and problems. Ofcourse, if you want professional help, there’s FuseSource as well. We have our own communityat fusesource.com. So, all these things combined means that you have more ﬂexibility and amuch more agile way of doing integration.Gardner: You know it strikes me that when we begin to talk about integration that I’dmentioned service-oriented architecture (SOA), but that was sort of yesterday’s buzzword. Werenow into cloud, hybrid, and mobile. But, from an architectural perspective, you cant really scaleand leverage these open components without that proper underpinning, typically an enterprise-service-bus (ESB) architecture.Rob, help me understand why doing this correctly from an architecture, not just an open-sourceperspective, is really important as well.Davies: You hit the core things about the SOA and the ESB architectures. We see where peopleare using, in particular, Apache Camel and some of our other open-source projects. They wantﬂexibility there. So, they want to leverage a service bus, put things on, expose them as service,and expose them over the service bus, which uses different transports to enable that bus, be thatmessaging, HTTP, or whatever other means you want to use.Application integrationAt the same time, you also want to have the ﬂexibility now to do it in application integration. You want to have that ﬂexibility for some services and you very much need that enterprise service bus in place. But for other cases, you want to be able to do that more locally, where the integration points are. The approach that we have is that we enable you to do both,because you can embed Apache Camel inside an application server, if you want it inside yourapplication itself. If you want to use it in a more traditional sense, you can deploy it intoServiceMix. You can deﬁne your apps easily, deploy them into ServiceMix, and use it to managethe container.Having that ﬂexibility as well means that you can have the right architecture for your particularsolution. If you look at how people would do the integration before, they’d have to get an ESB,and that would force the whole architecture of how they do things. When you’ve got moreﬂexibility, it means that you can make the right architecture choices that you need, and youre notconstrained to one particular style of integration.Gardner: Im facing a lot of questions more recently about how to cross the domains that wevementioned -- SaaS, cloud, on-premises, traditional architecture, and private cloud architecture.
Does the service-bus approach and the open-source approach also give us some sort of a path orvision for how to go about this? I think were just starting to enter into how to integrate mylegacy applications with cloud or SaaS applications in a meaningful way? What are yourthoughts about that, Rob?Davies: I completely agree. Having open source enables you to have the insight into how theintegration application works. But more importantly, those environments are changing veryrapidly. If you just look back just a couple of years, when people were starting to use the cloud,they weren’t even thinking about having hybrid clouds. Now, were seeing more and morepeople, more of our customers, looking to hybrid clouds and have a private cloud forapplications.When they need the capacity, obviously they can get that capacity in a public cloud. But, to haveall those PCs working together seamlessly, they need the agility that you get from an integrationsolution that can be deployed on a public cloud, locally, or a combination of both. That’ssomething that you can only get from software that has evolved at the same pace as the demandsof the environment.You can only really get that speed of innovation to keep up with the way the environment ischanging by choosing open source, because the open-source community itself is driving theprojects to keep up with the demands.So, you have to try to move outside of a traditional release cycle that you would get from atraditional product company. You don’t really have any other alternatives, if you want to keep up,than to look at open-source projects, the Apache ones in particular.Apache projects certainly hit the right notes in that youve got both very business-friendly licensefrom the Apache license and very active communities, and you’ve got diversity in thatcommunity. You know these projects are going to live beyond the lifetime of particularindividuals on the projects.Support and consultancyYou also have the beneﬁt of having companies like FuseSource, which created the projects inthe ﬁrst place, and who are there and able to provide support and consultancy if you need it. Youget the best of having a dynamic community, a dynamic project, and you also get the security ofhaving professional company to back it up.Gardner: Id like to revisit that thought about the traditional upgrade path in the product cycle.Many organizations have faced two stages of this. One is to wait for the commercial vendor, tocome out with the upgrade or often, an association with larger projects that they have acrossdifferent platforms, brings in various versions and iterations that theyve done.
Its a fairly complex undertaking for the vendor, but then there is the complexity of them bringingthat into your organization, and theres cost, because you have the upfront licensing cos. Manytimes, you’ll incur hardware cost and many times you want to have cohabitation of your olderdeployments, as well as new ones that come online. This is sometimes a three- to ﬁve-yearprocess.Tell me why an open-source approach and, from a cost perspective, that upgrade path is muchdifferent.Davies: Because it’s open source. The projects that we are involved in Apache are Apachelicensed. ActiveMQ, which is a message bus, Camel, ServiceMix, and CXF, are Apache licensed.It means that you dont have to pay the license costs upfront.Youre actually right about the time of the release cycle for a traditional product company. Theproblem that organizations are facing now is that the environments that they can deploy into andhave to interface with are changing and evolving so quickly. You just cant have a luxury waitingfor a three- to ﬁve-year release cycle.And what often happens is that the software you are trying to integrate with is really out of dateand people have moved onto something else. So, up front, you have to look at what you can useto integrate with these systems as they evolve. Things are evolving more quickly over time.There are different sorts of social networks that you have to interface with, and that market hasbeen very dynamic over the last few years.Twitter has been around for a few years, but we see people using Twitter as asynchronouscommunication within their organizations to give out real-time information updates. So, that’simportant. Who knows whats going to be just around the corner, because things have evolvedvery quickly.If you want your organization to keep pace with the changing environment were in, you have tolook for the right integration solutions right now, and choose the ones that will be able to keeppace.Dana Gardner How rapidly are the iterations within the Apache project, within Camel inparticular, happening? How rapidly is innovation taking place?Very fast paceDavies: It’s happening at a very fast pace. When we do release these out of Apache, itstypically every three months, but in that three month period there could be other components thathave gone into the Apache Camel Framework. Because its open source, people can actually lookabout, release their own components into an open-source environment, or develop themseparately without necessarily releasing to Apache, just to get the functionality out.
That pace of change is very fast and it’s near real time. When the need comes up, within a fewdays or a week, you would probably ﬁnd someone who has already written that integrationcomponent that you need and it’s available.Gardner: This is, of course, a global community. You have a great number of different inputsand parties involved, different locations that are supported, and different localizations, andlanguages.Davies: Absolutely. That’s another beneﬁt of having an open-source, and a well-known open-source, community to drive our innovation and to back it up.Gardner: Debbie, lets go look at whats happening in the community. I understand you have aconference that’s coming up, a ﬁrst of its kind. Why is this a good time to be pulling together theCamel Community, and what you’re going to do?Moynihan: We’re really excited. We have an event coming up in May. It’s called CamelOne andthe reason why we focused on Camel with the name of the event, is that it’s actually an event foropen-source integration and messaging overall. It’s because Camel is a really great way forpeople to get started, and it’s also a great way for more advanced integration developers as well.It brings together the entire community.Rob was talking about earlier about how there is always these new technologies coming up andpeople can add components. The nice thing about Camel is that it provides a basic foundationand a terminology of well-deﬁned patterns. The integration patterns themselves are very well-deﬁned, but whats happening is all the different ways in which you connect and what you areconnecting to have been changing and evolving over time.Camel is a great foundation and CamelOne is an event to bring together users of Camel and otheropen-source integration and messaging technologies to learn more about Camel, open-sourcemessaging like ActiveMQ, and ESBs like Apache ServiceMix.You were talking earlier about cost savings. More and more people are being asked to dointegration. The nice thing about Camel and about these other technologies is some peoplemaybe just only needed to do lightweight integration. They can just learn how to use Camel andlearn the basics.Other people are going to be doing more in-depth management of many integration patterns andthey may need to know all the nuances of an ESB platform. The focus of CamelOne is to bringpeople together to understand, learn about, and meet each other and to grow this community ofopen-source integration users.Gardner: So, this is CamelOne, May 24, in the Washington D.C. area. Why Washington D.C.?Is there a lot of this going on in the public sector?
Central locationMoynihan: Actually, we do have a lot of users in the Washington D.C. area. We also thoughtthat was a central location, where people could come from not only anywhere in the US but alsofrom other regions of the world as well There are a lot of direct ﬂights to that location. But, wedo have a lot of users in the area. For example, the Federal Aviation Administration (FAA) isgoing to be speaking and they have selected open-source integration for the next generation oftheir services infrastructure.Since they connect with a lot of other agencies, there is a lot of interest in learning morespeciﬁcally about that program and about the technologies that its built upon, because a lot ofother agencies need to connect.Gardner: One of the other aspects of this that Im seeing in the market is that more people needto take part in integration. It cant just go through a bottleneck of beard-and-sneaker guys in theback room who can do coding. Integration needs to be part and parcel with process innovation.That means we need to elevate it out to a wider group of individuals, maybe as many as possiblethat are on the front lines of process innovation and analysis.Whats being done about the integration that weve been describing? It’s wonderful that we havethe open source and we have the cost beneﬁts, but how about bringing this to a larger class ofindividuals, Debbie?Moynihan: On April 11, we announced the general availability of a new graphical tooling forApache Camel. The addition of graphical tooling makes it easier for more people to dointegration development. They dont have to write code. They can use a drag-and-dropenvironment to select the integration patterns that they want to implement, and the software willimplement them. They can test them and deploy them into production as well.The addition of tooling is going to help broaden how many people can do integration, and werereal excited. Weve been doing a Beta program since the end of January with over 500participants. Rob mentioned the breadth of all the components and how hot Apache Camel hasbeen. Were not surprised that more and more people want to use it. So, the idea of having toolingon top of it is really attractive to users.Gardner: So, whats the name and where do you go to ﬁnd out more about them?Moynihan: The Fuse IDE for Camel is the name. It plugs into an Eclipse environment and youcan get it at fusesource.com.Gardner: And how about more information on CamelOne? It’s simple, I suppose search onCamelOne will get you there.Moynihan: Yes, camelone.com is the website as well.
Gardner: Now, you guys have been involved with a series of books and you have somethingnew coming out in that series. Tell me about that.Camel in ActionMoynihan: There are a couple of books that recently have come out. One is Camel in Action,which is fantastic for people who want to get going with Camel and learn how to use and deployit. Rob is coauthor of the ActiveMQ in Action book, which has come out in print recently fromManning Publications.Davies: ‘ActiveMQ in Action’ is really a scripted book, which goes through all the different usecases of using ActiveMQ, right from getting started and what messaging is about. It walks youthrough different deployment options, all the way up through using clusters of ActiveMQbrokers, to using ActiveMQ as a wide area network, so you can connect geographically dispersedlocations.It shows you how to tune the performance of ActiveMQ and get the best out of it. So its verycomprehensive book about how to use ActiveMQ. Its somewhat complementary to Camel inAction as well. ‘Camel in Action’ goes through all the different patterns you can use.It doesnt talk about using Camel. It talks about integration patterns as well and then describeshow you can use those using Apache Camel, and you can use Apache Camel with ActiveMQ.ActiveMQ also can embed Apache Camel. So, you have routes running inside the broker fromCamel. The two of them are very complementary.Gardner: Lets step back for a wider perspective. Im seeing that the need for integration isincreasing. The things that need to be integrated are increasing, perhaps exponentially. The paceat which that needs to take place is very rapid and dramatic, compared to the history ofcomputing. Open source is well established We’ve seen many different organizations embracingthis. We saw Red Hat come out recently with some very strong growth ﬁgures.So, it seems to me that open source is a very mature approach now, not something that’s a newkid on the block, by any stretch. When you put these factors together and when you look at theneed for enterprise as a service within applications from the start, not something you bolt on orthink about after the fact but actually build applications for, of, and by integration capabilities,this perhaps spells a historic shift.Maybe we could riff on the future or even look at this from an abstract or even philosophicalperspective. Rob, are we at a shift here where the ability to integrate becomes an essentialcharacter of businesses?Davies: We probably are at that shift right now. Sometimes, its difﬁcult to see things happeninglike that, if you’re actually right inside in the middle of it. But, if you look at the way theenvironments change, you’ve got to actually be running your compute resources.
We’ve talked about cloud environment. Also there’s social network, SaaS, and mobile devices,and you need to link all those together. Its coming to the point where organizations won’t have achoice other than to use open source as a way to try to keep up with a pace of change.Were probably at a point now, where we’re going to see that the traditional model of providingsoftware is going to dwindle over time, probably pretty rapidly as well, as organizations realizethat they need the ﬂexibility and the ability to change what they’re doing very quickly.Future-prooﬁng applicationsIts a really good point that you made. You have to start thinking about how youre going tofuture-proof your applications right from the beginning to adapt to changes in theirenvironments. You have to architect in how you’re going to integrate and future-proof yourapplications, because it does get more costly if you do it as an afterthought.Gardner: Many of the SaaS providers are doing multitenancy and providing applications asservices on demand at a very attractive and aggressive price point. Theyre leveraging opensource on the back end, I have to imagine. Do you have any insight into what the serviceproviders themselves are building with?Davies: Most applications now, in particular on the cloud, are using open source at the back end.We cant give you any speciﬁc details of vendors that are doing that, but I know theyre usingopen-source projects, and not just the SaaS vendors, but some of the other existing productvendors use open-source as well to enable their product.We certainly see open-source as deﬁnitely mainstream now, and we’ve seen it has been the ﬁrstchoice that people use for building any kind of application or service they’re providing. Its morea case of people asking the questions now of not should we be using open source but whyshouldn’t we use open source? Its starting to become a ﬁrst choice for people to go to.Gardner: Lets look at some of the ways in which those people are making that choice. Debbie,you mentioned the FAA. Are there other organizations that you can point to and say, either byname or by use case scenario, that they’ve taken this leap, they’ve made those choices, they’veembraced some of the new requirements around integration, and they have some positive proofpoints? Any examples that we can look to?Moynihan: Sure. Sabre is one of our customers. Sabre Holdings is using the FuseSource open-source software, and they started using open-source software many years ago. Years ago, a lot ofpeople chose it because they were looking at cost and ﬂexibility. Now, theyre seeing that youactually were getting more features faster in open source than you were getting in traditionalsoftware. It’s more dynamic and ﬂexible, and being involved with the community is reallyexciting.One of the things that Sabre has done with open-source software is a travel gateway. Theyconnect to many different airline technologies and travel agencies and they have over one-and-a-
half billion transactions running through their infrastructure on any given day. Theyve beenusing FuseSource open-source software for over a couple of years with zero downtime.Being able to use open-source, they have that ﬂexibility, have the interaction with thecommunity, and also have high-performance and reliability.People are getting all of the traditional beneﬁts of high-quality software, but also that dynamicability to get new features, to get new technologies, to get bugs ﬁxed, for example, really quicklywith the community. With support vendors like FuseSource providing subscription so that theycan have access to the engineers directly who are working on these projects, they can quickly getturnaround and get what they need to make those dynamic changes in their business.Retail industryAnother area that we are seeing a lot of people look at open source is in the retail industry.Earlier on, people were looking for cost savings. If you think about retail, it’s common forretailers to have a lot of locations, whether it’s franchises or stores. Using open-source, you cansave a lot of cost on your IT footprint in those locations.Specsavers is one of our customers. Theyre deploying open-source to over 1,000 retail stores.Were seeing more and more retailers looking at open-source to be able to do that. Theyre goingto get all the ﬂexibility of being able to incorporate these new technologies as we incorporatethem into the open-source projects really quickly. But, right from the get-go, they have reducedcosts, ﬂexibility and the involvement within the open-source community and directly with thedevelopment teams through working with vendors like FuseSource that support the open-sourcecommunities.Gardner: For folks who are looking to ramp up their adoption of open-source integration, arethere some resources that they should be aware of in terms of getting started?Moynihan: I would encourage people, speciﬁcally if they are looking at open-source integrationand messaging, Apache Camel is a good place to get started. We have a productized distributionat fusesource.com as well.The reason why I suggest Apache Camel is that its is based on the enterprise integration patternsbook and provides a nice foundation and deﬁnition around some of the most commonly usedintegration patterns. It’s a great way to get started.People could obviously come to CamelOne, which is going to be really exciting, meet a lot of thepeople who are experts in the community, and meet other users of open-source integrationmessaging software.Also on our website fusesource.com, we have a lot of webinars, which are happening live on aregular basis. We have a lot of archived webinars, which actually walk you through technicaltutorials on how to get started with these various open-source projects.
So I’d highly recommend to check that out and to check out the books that weve mentioned andthe documentation on our site as well.Gardner: Very good. We have been talking about how enterprise integration requirements arerapidly shifting in order to accommodate such general and global trends as cloud computingmobile device use an explosion and the increase demand for extended enterprise businessprocesses.I want to thank our guests who have been here with us, Rob Davies, who is the Chief TechnologyOfﬁcer at FuseSource. Thanks a lot, Rob.Rob Davies: Thank you very much, Dana.Gardner: And Debbie Moynihan, the Vice President of Marketing at FuseSource. Thanks,Debbie.Moynihan: Thank you, Dana.Gardner: Youve been listening to a sponsored BrieﬁngsDirect podcast. This is Dana Gardner,Principal Analyst at Interarbor Solutions. Thanks for listening, and come back next time.Listen to the podcast. Find it on iTunes/iPod and Podcast.com. Download the transcript. Sponsor:FuseSourceTranscript of a sponsored BrieﬁngsDirect podcast on the maturing of open source software andits role in making enterprises responsive to a changing landscape. Copyright InterarborSolutions, LLC, 2005-2011. All rights reserved.You may also be interested in: • FuseSource Gains New Autonomy to Focus on OSS Infrastructure Model, Apache Community Innovation, Cloud Opportunities • Apache Camel Addresses Need for Discrete Infrastructure for Services Mediation and Routing • Apache CXF: Where its Been and What the Future Holds for Web Services Frameworks