Development of social media projects with Apache Camel, Fabric8 & Hawtio

2,121 views
1,918 views

Published on

This talk presented at Devnation 2014 - San Francisco let's you to discover how to develop social media projects with Apache Camel, Fabric8 & Hawtio and more precisely how to handle your data/metrics to inquiry them using Full Text features of Lucene/ElasticSearch with Kibana3, how to design dashboard, monitor your activities and create plugins for your business based on Hawtio Web Console. The code f the 3 demos is available here : https://github.com/cmoulliard/devnation-2014-camel and cover use cases :
- Real Time application (apache camel, twitter and websocket)
- Collect and store metrics to query them (elasticsearch, lucene & kibana3)
- Design dashboard, plugin to measure activities

Published in: Technology, Business

Development of social media projects with Apache Camel, Fabric8 & Hawtio

  1. 1. Powered by Asciidoctor, Hyla & RevealJS DEVELOPMENT OF SOCIAL MEDIA PROJECTS WITH CAMEL DevNation /14th of April-2014 Architect, Engineer &Committer Charles Moulliard
  2. 2. SPEAKER AgriculturalEngineer &Zoologist 19 years of experience in ITworld development Projectmanager in Bank, Financial, Telco world Specialized in new technologies Web &Integration Architect/Engineer @Red Hat Committer : Apache ServiceMix, Karaf (PMC), Camel(PMC), Fabric8, Drools, Hawtio, Asciidoctor
  3. 3. SPEAKER - CON’T Twitter : LinkedIn : Blog: Slideshare : http://twitter.com/cmoulliard http://www.linkedin.com/in/charlesmoulliard http://cmoulliard.github.io http://www.slideshare.net/cmoulliard
  4. 4. COUNTRY WHERE I LIVE
  5. 5. WHAT YOU SHOULD KNOW ABOUT BELGIUM
  6. 6. MY PASSION(S)
  7. 7. AGENDA Introduction SocialMediaHype Use cases covered Integration &Camel Technology WhatJBoss Fuse can offer Conclusion
  8. 8. INTRODUCTION Web 2.0 “Revolution”, User is an actor collaborating Consequences : HTTPs requests Volume of data exchanged
  9. 9. INTRODUCTION - CON’T TechnologyTransformation HTML5 JavaScript/JSon /NoSQL Push  Ajax -WebSocket
  10. 10. AGENDA Introduction Social Media Hype Use cases covered Integration &Camel Technology WhatJBoss Fuse can offer Conclusion
  11. 11. SOCIAL MEDIA HYPE Business - Companies MarketingStrategy, ProductPromotion, Launch or Campaign annoucement Seekingcandidates … Use Web2.0 butSocial Medias
  12. 12. SOCIAL MEDIA HYPE - CON’T Why  To create content thatattracts attention &encourages readers to shareitwith their socialnetworks
  13. 13. AGENDA Introduction SocialMediaHype Usecases covered Integration &Camel Technology WhatJBoss Fuse can offer Conclusion
  14. 14. USE CASES COVERED - DEMO1 Pattern: RealTime &Broadcasting Definition: Collectdatain realtime from differentproviders (twitter, facebook, …) and broadcast/multicastdatato subscribed channels Business : Event, conference, meetingroom
  15. 15. USE CASES COVERED - DEMO2 Pattern: CollectDataMetrics/Statistics Definition: Retrieve datafrom SocialMediaand store them in order to querythe datato analyze the results Business : Measure performances of acampaign, product launch, analyze datato design marketingstrategies
  16. 16. USE CASES COVERED - DEMO3 Pattern: Business ActivityMonitoring Definition: Monitor activities, operationaldays using reporting, dashboard tools. Business : Measure performances, rentability, return of investment
  17. 17. AGENDA Introduction SocialMediaHype Use cases covered Integration& Camel Technology WhatJBoss Fuse can offer Conclusion
  18. 18. INTEGRATION Integration is  reallyhard  protocols, standards, dataformats, systems Longstorysince Common ObjectRequestBroker Architecture
  19. 19. INTEGRATION - CON’T Concept  Decouple Producer from Consumer  Message transport information
  20. 20. INTEGRATION - CON’T  Layer to transport messages : BUS ESB, JBI, SCA
  21. 21. INTEGRATION - CON’T OpenSource JavaIntegrationFramework Designed around : Domain Specific Language ImplementEnterprise Integration Patterns
  22. 22. CAMEL implemented and more : Loadbalancer, Throttler, Delayer, … > 50 patterns
  23. 23. CAMEL - CON’T Keyfeatures Component Endpoint Consumer Producer
  24. 24. CAMEL - CON’T Keyfeatures : route, processor
  25. 25. CAMEL - CON’T Interceptor : trace, log, capture business events
  26. 26. CAMEL - CON’T Container for the routes  CamelContext
  27. 27. CAMEL - CON’T Cross communication notallowed using direct, seda
  28. 28. CAMEL - CON’T Butpossible Using BUS like NMR, Broker, Shared Component with direct-vm, vm
  29. 29. CAMEL - CON’T TypeConverter Strategy Allow to convert the bodypayloads from one type to another To and From these types File String byte[] and ByteBuffer InputStream and OutputStream Reader and Writer
  30. 30. CAMEL - CON’T Data Transformationfor complex use case packageorg.devnation.camel; importjava.io.InputStream; importjava.io.OutputStream; importorg.apache.camel.Exchange; publicinterfaceDataFormat{ voidmarshal(Exchangeexchange,Objectgraph,OutputStreamstream)throwsException; Objectunmarshal(Exchangeexchange,InputStreamstream)throwsException; } Marshalling : Object XML (JAXB) Unmarshalling : XML  Object(JAXB)
  31. 31. CAMEL - CON’T > 20 DataFormat
  32. 32. CAMEL - CON’T
  33. 33. CAMEL - CON’T
  34. 34. CAMEL - CON’T FluentAPI packageorg.devnation.camel; importorg.apache.camel.builder.RouteBuilder; publicclassExampleRouteBuilderextendsRouteBuilder{ @Override publicvoidconfigure()throwsException{ from("amq:queue:quotes") .filter().xpath("/quote/product/='widget") .bean("QuotesService","widget") .filter().xpath("/quote/product/='gadget") .bean("QuotesService","gadget"); } }
  35. 35. CAMEL - CON’T Alternative : Spring, BlueprintDSL <?xmlversion="1.0"encoding="UTF-8"?> <beansxmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beanshttp://www.springframework.org/schema/ http://camel.apache.org/schema/springhttp://camel.apache.org/schema/spring/camel- "> <beanid="quotesService"class="org.devnation.camel.QuotesService"/>" <camelContext xmlns="http://camel.apache.org/schema/spring"> <route> <fromuri="amq:queue:quotes"/> <filter> <xpath>"/quote/product/='widget"</xpath> </filter> <beanid="quotesService"method="widget"/> <filter> <xpath>"/quote/product/='gadget"</xpath> </filter> <beanid="quotesService"method="gadget"/> </route> </camelContext> </beans>
  36. 36. CAMEL - CON’T In memorybus /alternative to JBI, SCA, NMR Transportobjects : XML, File, Stream, Bytes Predicate &Expression language (xslt, xpath, …) Sync/Async exchanges Threads Management, Tx Architecture Error and exception handling Policydriven Container agnostic
  37. 37. CAMEL - CON’T
  38. 38. AGENDA Introduction SocialMediaHype Use cases covered Integration &Camel Technology WhatJBoss Fuse can offer Conclusion
  39. 39. TECHNOLOGY - CAMEL combined with camel-gmail camel-dropbox camel-ajax, camel-websocket, camel-sql, camel-jdbc, … camel-elasticsearch
  40. 40. TECHNOLOGY - COMMON Open standard for Authorization OAuth
  41. 41. TECHNOLOGY - CAMEL Component: FaceBook Info : GrantAccess : API : http://camel.apache.org/facebook.html https://developers.facebook.com/apps http://facebook4j.org/en/index.html //startwitha30daywindowforthefirstdelayedpoll Stringsince="RAW(" +newSimpleDateFormat(FACEBOOK_DATE_FORMAT).format( newDate(System.currentTimeMillis()-TimeUnit.MILLISECONDS.convert(30,T +")"; from("facebook://searchPosts?reading.limit=10&"+ "reading.locale=en.US&reading.since="+since+"&"+ "consumer.initialDelay=1000&"+ "consumer.sendEmptyMessageWhenIdle=true&" +getOauthParams());
  42. 42. TECHNOLOGY - CAMEL OAuth protectedStringgetOauthParams(){ return"oAuthAppId="+properties.get("oAuthAppId")+ "&oAuthAppSecret="+properties.get("oAuthAppSecret") +(properties.get("oAuthAccessToken")!=null ?("&oAuthAccessToken="+properties.get("oAuthAccessToken")):""); } Externalize tokens with PropertyPlaceHolderparameters consumer.key=rtPyyyyyyyyyyyyyyyyg consumer.secret=HyoWW3xxxxxxxxxxxxxxxxTuyEz8qrk access.token=7976eeeeeeeeeeeeeeeeeeeuRmG0IP access.token.secret=VNtttttttttttttttttg0B8jWGs
  43. 43. TECHNOLOGY - CAMEL Example : To create apostwithin your Facebook profile Send afacebook4j.PostUpdate bodyto acamelfacebook producer PostUpdatepost=newPostUpdate(newURL("http://facebook4j.org")) .picture(newURL("http://facebook4j.org/images/hero.png")) .name("Facebook4J-AJavalibraryfor"+ "theFacebookGraphAPI") .caption("facebook4j.org") .description("Facebook4JisaJavalibrary"+ "fortheFacebookGraphAPI."); from("direct:post") .setBody().constant(post) .to("facebook://postFeed/inBody=postUpdate");
  44. 44. TECHNOLOGY - CAMEL ComponentTwitter Info : GrantAccess : API : http://camel.apache.org/twitter.html https://dev.twitter.com/apps/new http://twitter4j.org/en/index.html Example from("twitter://search?type=polling&delay=" +delay+"&useSSL=true&keywords=" +keywords+"&"+getUriTokens()) .choice() .when().simple("${body.tweet.text}>'java'") .bean("Service","push") .otherwise() .to(">>Tweetsreceived");
  45. 45. TECHNOLOGY - CAMEL OAuth protectedStringgetUriTokens(){ return"consumerKey="+consumerKey +"&consumerSecret="+consumerSecret +"&accessToken="+accessToken +"&accessTokenSecret="+accessTokenSecret; } Advanced Query https://support.twitter.com/articles/71577-using-advanced- search
  46. 46. TECHNOLOGY - CAMEL Simple Example to postatweet Datenow=newDate(); Stringtweet="Demo:thisisatweetpostedon"+now.toString(); from("direct:post") .setBody().constant(tweet) .to("twitter://timeline/user?"+getUriTokens());
  47. 47. TECHNOLOGY - CAMEL Object/JSON Use utilitylike Google JSONAPI Marshall/Unmmarshall JSON
  48. 48. TWITTER - CON’T BAD -  Stringdata="{"+ """+"timestamp"+"":""+formatDate(generateTimeStamp())+"","+ """+"createdAt"+"":""+status.getCreatedAt().toString()+"","+ """+"id"+"":""+status.getId()+"","+ """+"text"+"":""+status.getText()+'''+"","+ """+"isFavorited"+"":""+status.isFavorited()+"","+ """+"isRetweeted"+"":""+status.isRetweeted()+"","+ """+"favoriteCount"+"":""+status.getFavoriteCount()+"","+ """+"inReplyToScreenName"+"":""+status.getInReplyToScreenName()+'''+ """+"geoLocation"+"":""+status.getGeoLocation()+"","+ """+"place"+"":""+status.getPlace()+"","+ """+"retweetCount"+"":""+status.getRetweetCount()+"","+ """+"isoLanguageCode"+"":""+status.getIsoLanguageCode()+"","+ """+"user"+"":""+status.getUser().getName()+"","+ """+"country"+"":""+country+"""+ "}";
  49. 49. TECHNOLOGY - CON’T BETTER-  privatevoidinit(JSONObjectjson)throwsTwitterException{ id=getLong("id",json); source=getUnescapedString("source",json); createdAt=getDate("created_at",json); isTruncated=getBoolean("truncated",json); inReplyToStatusId=getLong("in_reply_to_status_id",json); inReplyToUserId=getLong("in_reply_to_user_id",json); isFavorited=getBoolean("favorited",json);
  50. 50. TECHNOLOGY - CAMEL Routingengine Camel&EIP Patterns like  Aggregator Splitter Transform Multicast
  51. 51. TECHNOLOGY - REAL TIME Websocket Full-duplexsingle socketconnection HTTP requestfollowed byWebSocketdata Packets exchange ws:// and wss:// protocol Partof HTML5 initiative Specification rfc-6455 (Dec-2011) managed by IETF
  52. 52. TECHNOLOGY - REAL TIME - DEMO1 Integrated within ActiveMQ &Camel More info : Develop RealTime HTML5 applications usingWebSocket with Apache Camel&ActiveMQ -CamelOne 2012
  53. 53. TECHNOLOGY - DATA METRICS Kibana&ElasticSearch  InsightTechnology CollectLogs, Camelmetrics …
  54. 54. TECHNOLOGY - DATA METRICS
  55. 55. USE CASES COVERED - DATA METRICS
  56. 56. TECHNOLOGY - DATA METRICS Create abean recuperatingMessage/Exchangeusing @Header, @Body Store itusingorg.fusesource.insight.storage.StorageService publicclassStoreService{ privatestaticStringES_TYPE="insight-tweet"; privatestaticStorageServicestorageService; publicstaticvoidstore(@Header("tweet-full")Stringdata){ storageService.store(ES_TYPE,generateTimeStamp(),data); } publicvoidsetStorageService(StorageServicestorageService){ StoreService.storageService=storageService; }
  57. 57. TECHNOLOGY - DATA METRICS InjectStorage service <?xmlversion="1.0"encoding="UTF-8"standalone="no"?> <blueprintxmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd"> <!--ServiceServiceusedtosendtweetstoESStorage,parseJSON--> <beanid="helper"class="org.devnation.demo.camel.Service"> <propertyname="storageService"> <referenceinterface="org.fusesource.insight.storage.StorageService"/> </property> </bean> </blueprint>
  58. 58. TECHNOLOGY - DATA METRICS Callitfrom your Camelroute from("twitter://search?type=polling&delay="+delay +"&useSSL=true&keywords="+keywords +"&"+getUriTokens()) .routeId("Tweet-Store-WS") .delay(5000) //WereceivetheTwitter4JStatusthatwewilluse //toextractinfotext,isfavorited,isRetweeted,geolocation,isoLanguageCod //andcreateAJSONmessageusedlaterontostoreitinES .setHeader("tweet-full").method(Service.class,"getJSONTweet") //MessageisstoredusinginsightinElasticSearch .bean(Service.class,"store")
  59. 59. TECHNOLOGY - DATA METRICS - DEMO2
  60. 60. TECHNOLOGY - DASHBOARD Hawtio Lightweight&modular HTML5 web consolewith plugins for managingJavaMBeans Javascript/RESTfrontend  jolokia JMX translator   Heartof the newFuse ManagementConsole
  61. 61. TECHNOLOGY - DASHBOARD Extend Hawtio Dashboard
  62. 62. TECHNOLOGY - DASHBOARD Declare MBean Interface publicinterfaceSocialMediaMBean{ /*Attributes*/ voidsetTweetsCounter(Integerval); publicIntegergetTweetsCounter(); /*Operations*/ List<String>searchTweets(Stringkeywords)throwsTwitterException; StringuserInfo(Stringid)throwsTwitterException;
  63. 63. TECHNOLOGY - DASHBOARD Service Implementation &MBean registration publicList<String>searchTweets(Stringkeywords)throwsTwitterException{ returntwitterService.searchTweets(keywords); } publicvoidinit(){ try{ if(objectName==null){ objectName=newObjectName("hawtio:type=SocialMedia"); } if(mBeanServer==null){ mBeanServer=ManagementFactory.getPlatformMBeanServer(); } try{ mBeanServer.registerMBean(this,objectName); }catch(InstanceAlreadyExistsExceptioniaee){ //Trytoremoveandre-register LOG.info("Re-registeringSchemaLookupMBean"); mBeanServer.unregisterMBean(objectName); mBeanServer.registerMBean(this,objectName); } }catch(Exceptione){ LOG.warn("Exceptionduringinitialization:",e); thrownewRuntimeException(e); } }
  64. 64. TECHNOLOGY - DASHBOARD - DEMO3 InjectService &initbeans <?xmlversion="1.0"encoding="UTF-8"standalone="no"?> <blueprintxmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"> <cm:property-placeholderid="twitterConfig"persistent-id="twitter" update-strategy="reload"/> <beanid="twitterFactory"class="org.devnation.demo.service.TwitterFactory"> <propertyname="consumerKey"value="${consumer.key}"/> <propertyname="consumerSecret"value="${consumer.secret}"/> <propertyname="accessToken"value="${access.token}"/> <propertyname="accessTokenSecret"value="${access.token.Secret}"/> </bean> <beanid="twitterService"class="org.devnation.demo.service.TwitterService"> <propertyname="twitterFactory"ref="twitterFactory"/> </bean> <beanid="socialDataMBean"class="org.devnation.demo.service.SocialMedia" init-method="init"destroy-method="destroy"scope="singleton"> <propertyname="twitterService"ref="twitterService"/> </bean> </blueprint>
  65. 65. TECHNOLOGY - PLUGIN Create your own hawtio plugin Web project(war, bundle) Require to register io.hawt.web.plugin.HawtioPluginwith parameters Context JS Scripts location Name and domain (used bycontroller)
  66. 66. TECHNOLOGY - PLUGIN <?xmlversion="1.0"encoding="UTF-8"standalone="no"?> <blueprintxmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"> <beanid="plugin"class="io.hawt.web.plugin.HawtioPlugin"init-method="init"destroy- <propertyname="name"value="${plugin-name}"/> <propertyname="context"value="${plugin-context}"/> <propertyname="domain"value="${plugin-domain}"/> <propertyname="scripts"value="${plugin-scripts}"/> </bean> </blueprint>
  67. 67. TECHNOLOGY - PLUGIN Design Frontend (AngularJS) Send RESTrequestto callService $scope.searchTweets=function(){ if(Core.isBlank($scope.keywords)){ return; } jolokia.request({ type:'exec', mbean:SOCIAL.mbean, operation:'searchTweets', arguments:[$scope.keywords] },{ method:'POST', success:function(response){ /*SimpleTable*/ $scope.tweets=response.value.map(function(val){ return{tweet:val}; }); Core.$apply($scope); }, error:function(response){ SOCIAL.log.warn("FailedtosearchforTweets:",response.error); SOCIAL.log.info("Stacktrace:",response.stacktrace); Core.$apply($scope); }
  68. 68. TECHNOLOGY - PLUGIN <divclass="row-fluidgridStyle"ng-controller="SOCIAL.FormController"> <h3>Twitter</h3> <divclass="span12"> <div> <formname="inputForm"ng-submit="searchTweets()"> <div> Keyword(s):<inputtype="text"class="entry-widget"ng-model="keywor </div> <pclass="span4centered"> <inputtype="submit"class="btnbtn-success"ng-disabled="!keywords" </p> </form> </div> </div> <divclass="span8"> <divclass="gridStyle"ng-grid="tweetsGrid"/> </div> </div>
  69. 69. TECHNOLOGY - PLUGIN - DEMO3
  70. 70. AGENDA Introduction SocialMediaHype Use cases covered Integration &Camel Technology What JBoss Fusecanoffer Conclusion
  71. 71. WHAT JBOSS FUSE CAN OFFER
  72. 72. WHAT JBOSS FUSE CAN OFFER - CON’T
  73. 73. WHAT JBOSS FUSE CAN OFFER - CON’T
  74. 74. TECHNOLOGY - CON’T
  75. 75. AGENDA Introduction SocialMediaHype Integration &Camel Use cases covered Technology WhatJBoss Fuse can offer Conclusion
  76. 76. CONCLUSION Fuse Technologyready  design Social Media Projects To cover use cases  RealTime, Datacollection, Fuse Analytics, BAM …
  77. 77. QUESTIONS Twitter : More info : , , @cmoulliard Apache Camel Hawtio fabric8 Code of demos : https://github.com/cmoulliard/devnation-2014-camel

×