On October 23rd, 2014, we updated our
By continuing to use LinkedIn’s SlideShare service, you agree to the revised terms, so please take a few minutes to review them.
Successful Stories• "Wissbi is an easy-to-use tool for distributedevent processing"-- Scott Wang, 奇群科技• "Wissbi provides great utilities for logging,debugging, and monitoring your distributedsystem" -- 前趨勢科技，王姓工程師• "Wissbi lets you easily manage your dataworkflow in a pipe and filter style with intuitivecommands" -- lunastorm, Open Source Developer
Wissbi 的前世今生• Wissbi is a highly available andscalable distributed messagerouting framework like ZeroMQthat took a different path fromtraditional MQ middlewares.• The toolkits supports an elegantand intuitive integration model,allowing writing event drivenapplications as easy as "$ tail -flog | grep error > error.log".• Scott Wang a.k.a. lunastorm, Sr.Engineer at Zillians• Trend Message Exchange (TME) isa highly available and scalabledistributed message routingframework that took a different pathfrom traditional MQ middlewares.• TMEs client toolkit, MIST, supportsan elegant and intuitive integrationmodel, allowing writing event drivenapplications as easy as "$ tail -f log |grep error > error.log".• Scott Wang a.k.a. lunastorm, Sr.Engineer at Trend Micro
I need amessaging system
Rethink Your Problem
XXDo I reallyneed a broker?Do I really needsynchronization?
What Do I Need?• A message publisher• A message subscriber• A directory service for publishers to knowwhere the subscribers are
Name Foo BarAddressDirectoryServiceMessage Subscriber
Name Foo BarAddress192.168.0.1:5566DirectoryServiceMessage SubscriberMessage PublisherGet all subscribersSend Message
Directory Services• DNS• First attempt• LDAP• $&@?!&@!?• ...
What ifI use directory fordirectory service?
Using Filesystem forDirectory Service• Store metadata on the filesystem• Follows the philosophy "Everything is a file"• Use standard Unix commands to manage it• mkdir, touch, rm, ln, ...• Just like /proc and /sys
Using Filesystem forDirectory Service• You get authorization for free• chown, chmod, ...• Good for testing• Launch any number of cluster just withdifferent metadata directories
Name FooAddress192.168.0.1:5566DirectoryServiceMessage SubscriberMessage PublisherNFS
Plumber (Program)• The plumber, in the Plan 9 from Bell Labs and Inferno operating systems, is amechanism for reliable uni- or multicast inter-process communication offormatted textual messages. It uses the Plan 9 network file protocol, 9p, ratherthan a special-purpose IPC mechanism.• Any number of clients may listen on a named port (a file) for messages. Portsand port routing are defined by plumbing rules. These rules are dynamic. Eachlistening program receives a copy of matching messages. For example, if thedata /sys/lib/plumb/basic is plumbed with the standard rules, it is sent to theedit port. The port will write a copy of the message to each listener. In this case,all running editors will interpret this message as a file name, and open the file.• The plumber is the 9P file server that provides this service. Clients may uselibplumb to format messages. Since the messages are 9P, they are networktransparent.http://en.wikipedia.org/wiki/Plumber_(program)
Minimum Dependency$ ldd wissbi-publinux-vdso.so.1 libpthread.so.0 libstdc++.so.6 libm.so.6 libgcc_s.so.1 libc.so.6 /lib64/ld-linux-x86-64.so.2The only dependency is a compilerwhich supports C++11!
<?xml version="1.0"?><!DOCTYPE Conﬁgure PUBLIC "-//Mort Bay Consulting//DTD Conﬁgure//EN" "http://jetty.mortbay.org/conﬁgure.dtd"><!-- =============================================================== --><!-- Conﬁgure the Jetty Server --><!-- --><!-- Documentation of this ﬁle format can be found at: --><!-- http://docs.codehaus.org/display/JETTY/jetty.xml --><!-- --><!-- =============================================================== --><Conﬁgure id="Server" class="org.mortbay.jetty.Server"><!-- =========================================================== --><!-- Server Thread Pool --><!-- =========================================================== --><Set name="ThreadPool"><!-- Default bounded blocking threadpool --><New class="org.mortbay.thread.BoundedThreadPool"><Set name="minThreads">10<Set name="maxThreads">50<Set name="lowThreads">25</New><!-- New queued blocking threadpool : better scalability<New class="org.mortbay.thread.QueuedThreadPool"><Set name="minThreads">10<Set name="maxThreads">25<Set name="lowThreads">5<Set name="SpawnOrShrinkAt">2</New>用 Config寫程式之Java 篇
# How many filter instances will be run in parallelWISSBI_FILTER_COUNT="1"# How to run the filterWISSBI_FILTER_CMD="sed --unbuffered -e "s/^/[ / ; s/$/ ]/""# If you run multiple instances in parallel, you can use the instance id $iin the command# WISSBI_FILTER_CMD="sed --unbuffered -e "s/^/$i: [ / ; s/$/ ]/""# The message sources name, leave it empty if the filter is a messagegeneratorWISSBI_FILTER_SOURCE="test.in"# The message sinks name, leave it empty if the filter is a messageterminalWISSBI_FILTER_SINK="test.out"WISSBI_FILTER_LOG_PREFIX="/tmp/filter-example"WISSBI_FILTER_PID_PREFIX="/tmp/filter-example"# If WISSBI_DEBUG_DUMP is set, message recording will be enabled, and 50messages# before the filter is terminated is dumped to the specified file.# If WISSBI_DEBUG_DUMP is set to empty, then a random dump filename will beused#WISSBI_DEBUG_DUMP=""wissbi filterexample
The easiest way to builda distributed pipeline
Live DemoA Calculator Service http://ascii.io/a/2914