Sophisticated customers wanted to take truck positions, run algorithms to optimize routes, and change the driver’s job orders based on location and skill. To do this, Lucid Logistics needed to open access to the raw data so that companies could run their own business rules. With RabbitMQ, the development team is now in a position to connect incoming data to all different types of micro-services, push data into Hadoop, or integrate with other clouds like Amazon Web Services. To learn more, visit pivotal.io/platform-as-a-service/pivotal-cf.

1. Lucid Logistics, a subsidiary of Quake Global, provides extremely fast, accurate asset tracking, fleet management, and telematics for fixed and mobile assets like trucks, ships, and equipment. Their system provides hardware for GPS monitoring, a satellite feed infrastructure, a cloud-based data store, and a web-based business intelligence application. Together, the solution allows customers to visualize via maps, capture sensor data, set-up geo-fences, configure email alerts, generate reports, and more. Customers cross a variety of industries such as oil, gas, energy, mining, heavy equipment, trucking, maritime, industrial control (SCADA), and government. Ultimately, Lucid Logistics helps companies run more effective business operations and improve safety, efficiency, compliance, and downtime.
CHALLENGE
Improving a Software Architecture to Satisfy Customers
Sophisticated customers wanted to take truck positions, run algorithms to optimize routes, and change the driver’s job orders based on location and skill. To do this, Lucid Logistics needed to open access to the raw data so that companies could run their own business rules.
The development team initially extended the architecture with a SOAP-based web service interface. As initial customers began to frequently poll large data sets via this API, system resources were quickly drained, taking minutes or not running to completion. The architecture couldn’t scale or update in real-time. Without current information, the raw data customers could not effectively run business rules. In addition, the resource drains impacted all users, creating broad customer dissatisfaction.
SOLUTION
Scaling with a Different Architecture and the RabbitMQ Ecosystem
To shift the architecture and achieve real-time data feeds, the engineering team agreed that a publish/subscribe mechanism was needed—a way to push all the data outside of the main application database instead of allowing polls against it. The
AT-A-GLANCE
Challenges
•
Poor performing queries could take 20+ minutes or fail, leading to unsatisfied customers
•
Finding an alternative to scaling a traditional RDBMS
•
Shifting an architecture from delayed polling via SOAP to real-time pushing
•
Getting a technical solution to market quickly
Solution
•
RabbitMQ
Key Benefits
•
Improved business agility, launched to market in weeks, and satisfied customers
•
Scaled real-time data feeds to millisecond responses instead of minutes per query
•
Yielded effortless integration effort to expose a cloud- based API
•
Provided a cost effective solution with wide, open source community support
CASE STUDY
Lucid Logistics
IMPROVED BUSINESS AGILITY, SPEED TO MARKET, AND SCALE
OVERVIEW
“ After about three weeks of development effort, we had the entire pub/sub scheme up and running on development servers where messages were being pushed to subscribers in JSON format.”
—Natalya Arbit, Senior Web Developer, Lucid Logistics
pivotal.io

2. Pivotal is a registered trademark or trademark of Pivotal Software, Inc. in the United States and other countries. All other trademarks used herein are the property of their respective owners. © Copyright 2014 Pivotal Software, Inc.
All rights reserved. Published in the USA. PVTL-CS-362-03/14
At Pivotal our mission is to enable customers to build a new class of applications, leveraging big and fast data, and do all of this with the power of cloud independence.
Uniting selected technology, people and programs from EMC and VMware, the following products and services are now part of Pivotal: Greenplum, Cloud Foundry, Spring,
GemFire and other products from the VMware vFabric Suite, Cetas and Pivotal Labs.
CASE STUDY LUCID LOGISTICS
Pivotal 3495 Deer Creek Road Palo Alto, CA 94304 pivotal.io
team looked at the open protocol, PubSubHubbub, and liked
the approach; however, the original implementation was geared
towards Atom/RSS with an XML format and heavy footprint. After
looking further, they realized an open source project, RabbitHub,
implemented the protocol with an HTTP-based interface to
RabbitMQ and allowed a flexible message format. The approach
made sense.
According to Natalya Arbit, Senior Web Developer at Lucid
Logistics, “Because RabbitHub was written as a plugin to
RabbitMQ, we immediately benefited from a myriad of RabbitMQ
features such as low memory footprint, built-in security, SSL
support, message and message queue persistence, and flexible
routing topologies via exchanges. After about three weeks of
development effort, we had the entire pub/sub scheme up and
running on development servers where messages were being
pushed to subscribers in JSON format.”
BUSINESS BENEFITS
Improved Business Agility, Speed to Market, and Customer
Satisfaction
RabbitMQ provided a valuable and performant set of capabilities
to Lucid Logistics’ customers and helped improve the real-time,
reliable delivery of data used to make their logistics decisions.
The development team was able to have a working solution in a
development environment very quickly, and the product evolved
from a completely database-driven application with a user
interface to a scalable, “public cloud API” in weeks. Frustrated
customers had issues resolved quickly, and a new, reliable
product capability was available to help bring in revenue and
grow the business.
Delivered a Scalable, Real-Time Data Publishing Mechanism
Together, RabbitMQ, RabbitHub, and PubSubHubbub delivered a
fast, scalable solution. Beyond the architecture of pushing data
instead of polling, the messaging model allows the new service
to be decoupled from the existing database and application.
This way, a number of raw data customers can be added to the
system without impacting the performance of the application
and user interface. In addition, RabbitMQ is lightweight—it has a
small disk and memory footprint. It is designed for reliability, high
availability, and scale out via clustering multiple nodes and other
distributed models.
Yielded Almost Effortless Integration for Development Teams
and Customers
While development teams can use RabbitMQ clients in Java,
Groovy, Ruby, Python, .NET, C++, COBOL, and other languages,
Lucid Logistics’ approach targeted JSON over HTTP POSTs for
internal and external interfaces.
Arbit shared, “It was a simple task when it came to integrating into
the existing Lucid Logistics message delivery stack. RabbitHub
places a pub-sub layer on top of plain old HTTP POST and
provides two URLs for every AMQP exchange and queue. One URL
is used to deliver messages to the exchange or queue. The other
is used to subscribe to messages forwarded by the exchange or
queue. So, whenever new data comes in, it is pushed to the hub
and HTTP POSTs go out to subscribers’ callback URLs. With this
approach, all the overhead of polling goes away while message
latencies go from minutes to milliseconds.”
RabbitMQ also supports open protocols like AMQP, HTTPS,
WebSockets, MQTT, and STOMP and a wide range of messaging
patterns such as persistent queues, durable subscribers, multicast,
topic routing, content routing, RPC, and others. Developers can
connect with just about any language via any integration pattern
and virtually any type of application, component, or service.
RabbitMQ also has an extensible plug-in architecture.
Provided Low Costs and a Significant Support Community
Lucid Logistics found an enterprise-level solution with all the
capital and operating expense benefits of open source software.
In addition to being easy to set-up, use, extend, and integrate,
RabbitMQ greatly reduces the cost of using traditional, enterprise
middleware licenses and hardware. As well, the open source model
has a large support community in the web server, app server, and
data store arenas as well as middleware, and the Lucid Logistics
team found that RabbitMQ had an active, extensive community to
lean on. In fact, RabbitMQ has approximately 50,000 production
sites and is bundled into several widespread open source
distributions like Chef, Debian, Ubuntu, and Fedora.
CONCLUSION
In looking at the bigger picture, Arbit concluded, “RabbitMQ, on
its own, opens many additional potential uses within the Lucid
Logistics message processing ecosystem. One such application
could be routing messages to another queue prior to them being
written into a database.” With RabbitMQ, the development team
is now in a position to connect incoming data to all different types
of micro-services, push data into Hadoop, or integrate with other
clouds like Amazon Web Services.
LEARN MORE
To learn more about our products, services and solutions, visit us
at pivotal.io.