Best Practices for Architecting High Volume, High Performance Publishing for Data Intensive Website

4,487 views

Published on

Published in: Technology, Education

Best Practices for Architecting High Volume, High Performance Publishing for Data Intensive Website

  1. 1. Copyright Edmunds Inc. (the “Company”). All rights reserved. Edmunds®, Edmunds.com®, the Edmunds.com car design, Inside Linesm , CarSpacesm and AutoObserver® are proprietary trademarks of the Company. This document contains proprietary and/or confidential information of the Company. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Company, and any such disclosure requires the express approval of the Company. Best Practices for Architecting High Volume, High Performance Publishing for Data Intensive Web Site October 23th 2010 Greg Rokita Director, Sr. Architect Edmunds.com
  2. 2. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Assumptions o Knowledge of Java o Basic understanding of Spring o Basic knowledge of JMS
  3. 3. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Agenda o Common Enterprise Problems o Layered Architecture o ActiveMQ and Virtual Topics o Camel o Thrift & Versioning o Retry and Throttling mechanism o Monitoring o Q&A
  4. 4. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Common Enterprise Problems o Multiple: o Environments (Prod, Test, Dev, etc) o Data Centers (Los Angeles, New York, Amazon EC2, etc) o Sites o Applications (Solr, Coherence, etc) o Data Sets (inventory, user data, pricing data, etc) o Data Format Changes o Components Fail
  5. 5. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. What evolved from the efforts o Message o Delivery o Routing o Persistence o Durability o Retries o Throttling o Versioning o Monitoring ActiveMQ Camel Thrift
  6. 6. 6 ActiveMq Broker
  7. 7. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Publish Subscribe o Producers decoupled from consumers –– cool idea o JMS durable topics suck o message consumer is created with a JMS client ID and durable subscriber name o only one consumer can be active for a client ID and subscriber name o CAN’’T failover of the subscriber if that one process running that one consumer thread dies o CAN’’T load balancing of messages.
  8. 8. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Virtual Topics Virtual Topic: VirtualTopic.Vehicle Queues: Consumer.Queue1.VirtualTopic.Vehicle Consumer.Queue2.VirtualTopic.Vehicle
  9. 9. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Virtual Topics public class QueueDestinationInterceptor implements DestinationInterceptor { public synchronized Destination intercept(final Destination destination) { return new DestinationFilter(destination) { public void send(ProducerBrokerExchange context, Message message) throws Exception { if (applyFilterBasedOnMessageProperties(destination)) { return; } destination.send(context, message); } }; } } o Message is always send to ALL the queues o Solution: Destination Interceptor ActiveMQ plug in
  10. 10. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Camel from(A).filter(header(““type").isEqualTo(““Widget")).to(B) Endpoint A Endpoint B Filter
  11. 11. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Camel Cont. activemq.queue.A activemq.queue.B Filter RouteBuilder builder = new RouteBuilder() { public void configure() { from(“activemq.queue.A”) .filter(header(“type”).isEqualTo(“Widget”)) .to(“activemq.queue.B”); } };
  12. 12. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Camel Cont. activemq.queue.A activemq.queue.B Filter <camelContext errorHandlerRef="errorHandler“ xmlns="http://camel.apache.org/schema/spring"> <route> <from uri=“activemq.queue.A"/> <filter> <xpath>/foo:person[@name='James']</xpath> <to uri="activemq.queue.B"/> </filter> </route> </camelContext>
  13. 13. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Camel Cont. <route> <from uri="timer://foo?fixedRate=true&amp;period=1000"/> <to uri="bean:myBean?method=someMethodName"/> </route> o Example endpoints o Queue o Topic o Timer o Email o Log o Javabean o FTP o HDFS o HTTP o XSLT
  14. 14. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Source/Target Selectors 14 Field Example Values Purpose Environment PROD, TEST, DEV The staging environment in the promotional cycle Data Center LAX, EC2 The data center where the environment is located Site Edmunds, InsideLine Defines the site as a set of services Application Digital Asset Manager, Inventory Application Deployment Unit
  15. 15. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Topic Selectors 15 Field Example Values Purpose Type Publish, Audit, Control Defines the type of the message Service Inventory, Pricing Type of data being send
  16. 16. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Producer / Consumer matching Producer Consumer Prod Lax Edmunds Inventory I am Prod, Test Lax, EC2 Edmunds Dealer Send To Prod Lax, EC2 Edmunds Inventory I am Test EC2 Edmunds Dealer Receive From Broker Destination Interceptor Publish Inventory Publish Inventory Virtual Topic Name Queue Name Match!
  17. 17. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Thrift: Data+Service+Strong Typing+ Versioning namespace java com.edmunds.inventory.thrift.gen struct Product { 1: string productType = "NCI", 2: map<string, string> vehicleDisplayInfo, } struct Inventory { 1: string id, 2: string vin, 3: string franchiseId, 4: map<string, string> edmundsAttributes, 5: list<Product> products, } service InventoryService { oneway void removeInventory(1:Inventory inventory), oneway void updateInventory(1:Inventory inventory), }
  18. 18. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Thrift –– Camel Integration Camel Thrift
  19. 19. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Thrift –– Camel Integration: Sender import org.apache.thrift.transport.TTransport; public class ClientTransportImpl extends TTransport { private SenderInternal senderInternal; public void flush() throws TTransportException { byte[] buf = writeBuffer.toByteArray(); writeBuffer.reset(); senderInternal.sendThrift(buf); } public class SenderImpl extends AbstractEndpoint implements Sender, SenderInternal, InitializingBean { public void sendThrift(Object object) throws Exception { Map<String, Object> headers = initializeMessageHeaders(); headers.put("CamelBeanMethodName", "executeThrift"); headers.put("CamelJmsMessageType", getContext().getProtocol().getJmsMessageType()); doSend(object, headers, ReceiverImpl.getEntryEndpointName(getTopicSelectors().topicName())); } private void doSend(Object object, Map<String, Object> headers, String entryPointName) { producerTemplate .sendBodyAndHeaders(entryPointName, ExchangePattern.InOnly, object, headers); } }
  20. 20. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Thrift –– Camel Integration: Receiver public class ReceiverService { public void executeThrift(@Body byte[] byteArray, @Headers Map<String, String> headers) { enterMessageDeck.addHeaders(headers); ReceiverInternal receiverInternal = (ReceiverInternal) callable.getReceiver(); receiverInternal.getContext().initialize(headers); ProcessorTransportImpl processorTransport = new ProcessorTransportImpl(); TProcessor processor = findProcessor(); processorTransport.messageReceived(byteArray, processor, receiverInternal }
  21. 21. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Creating Consumer @Component(“inventoryConsumer") public class InventoryConsumer extends AbstractDataHandler implements InventoryService.Iface { @Override public void updateInventory(com.edmunds.inventory.thrift.gen.Inventory inventory) { // perform your business logic here } <bean id="receiver" class="com.edmunds.eps.endpoint.impl.ReceiverImpl"> <property name="service" value=“inventory"/> <property name="messageType" value=“publish"/> <property name=“dataCenter" value=“lax"/> <property name=“environment" value=“prod"/> <property name=“site" value=“Edmunds"/> <property name="application" value=“Search"/> </bean>
  22. 22. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Creating Producer // Inventory.Client is generated by Thrift Inventory.Client inventoryClient = new Inventory.Client(sender.getProtocol()); inventoryClient.updateInventory( /*inventory object generated by Thrift*/ ); <bean id=“sender" class=“com.edmunds.eps.endpoint.impl.SenderImpl "> <property name="service" value=“inventory"/> <property name="messageType" value=“publish"/> <property name=“dataCenter" value=“lax"/> <property name=“environment" value=“prod"/> <property name=“site" value=“Edmunds"/> <property name="application" value=“Inventory-Source"/> </bean>
  23. 23. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Throttling getReceiver().getThrottler().setEnabled(true); getReceiver().getThrottler().setMaximumRequestsPerPeriod(10); getReceiver().getThrottler().setTimePeriodInMilliseconds(2000); Camel: Dynamically in Java: receiver.throttler.enabled=true receiver.throttler.maximumRequestsPerPeriod=10 receiver.throttler.timePeriodInMilliseconds=2000 Statically in property file using Spring PropertyOverrideConfigurer: from(…).throttle(10).timePeriodMillis(2000).to(…)
  24. 24. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Retries / Error Handling: free gift from Camel getReceiver().getErrorHandler().setUseCollisionAvoidance(true); getReceiver().getErrorHandler().setUseExponentialBackOff(true); getReceiver().getErrorHandler().setDelayPattern("5:1000;10:5000;20:20000"); getReceiver().getErrorHandler().setUri("jms:queue:dead"); ExceptionHandler exceptionHandler = new ExceptionHandlerImpl(RegionException.class); exceptionHandler.setHandled(true); exceptionHandler.addException(ExceptionA.class); exceptionHandler.setStop(true); getReceiver().addExceptionHandler(exceptionHandler);
  25. 25. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Control System 25 Topic Producer A Producer B Producer C Control Message o Control Message o Initiates producer activity o Bulk, Single and Multiple loads o Indicates targets systems for publishing o Decouples Producer logic form Clients of the publishing system o Allows to initiate all publishing activity form a single point o Can be sent from JMX or HTTP
  26. 26. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Heartbeat 26
  27. 27. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. High Level View
  28. 28. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Summary o Simple to develop producers and consumers (library takes care of the plumbing) o Can deploy producers and consumers “anywhere” o Can match producers and consumers at any level o Handle error conditions, throttling o Type safety o Versioning o HA & scalability o Consumers: Virtual Topics, Queues o Producers: Control System o Broker: Network of Brokers o Monitoring
  29. 29. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. We are hiring! http://www.edmunds.com/help/about/jobs
  30. 30. No part of this document or the information it contains may be used, or disclosed to any person or entity, for any purpose other than advancing the best interests of the Edmunds Inc., and any such disclosure requires the express approval of Edmunds Inc. Q&A Greg Rokita grokita@edmunds.com

×