This document provides tips about DirectFN's integration platform (DIP). It discusses how DIP addresses integration problems through an enterprise integration patterns approach. Some key points: - DIP views problems from an enterprise application integration (EAI) architect perspective rather than traditional enterprise architecture. - DIP follows design philosophies inspired by observing human society, such as modeling actors like people and incorporating concepts of concurrency, asynchrony, and soft real-time responses. - DIP incorporates both defensive programming and "let it crash" philosophies through mixed approaches, being more fault-tolerant at lower layers. - Actors in DIP have supervisor hierarchies to handle exceptional situations through restarting,

1. TIP OF THE DAY SERIES ABOUT ONE OF MY EXCELLENT CREATION
TIP 1 – Introduction
DirectFN / Mubashertheirnatural businessis75% relatedtointegrations,inone side theyconnectedto
financial exchangestogetPrice informationwhileinotherside to manage tradingtransactionsthrough
FIX,and laterwithapplicationgrowththeyhave beenintegratedwithmultiple exchangesandvarious
otherenterprise informationsystemsandmore andmore complex integrationsyettobe come.
Thoughthe problemnature iscloselyrelatedtointegrationsDFN have notseenthe problemthroughits
natural angle inits initial stages, DirectFnintegrationplatformhasbeendevelopedwithbetter
understandingtoaddressthese problems.
DIP isnothingbutlookingatproblemin anotherangle,ratherthanlookingatthe probleminEnterprise
Architectseye Icouldsay it’sa seenthroughproblemof EAIArchitectseye.
(EAI – Enterprise ApplicationIntegrations).
TIP 2 – Enterprise Integration Patterns
BuildinganIntegrationsolutionwhileaddressingitsownspecificproblemsisaverycomplex task,if you
do notuse righttool at right time youcan neverbuildasuccessful integrationsolution.Mostof early
day EAIArchitectsdevelopedthistechnologysenseafterspending verylongtime intheirproblem
domainswithlotof painful experiences.
Luckilytoour generationthose earlydayheroes(suchasGregorHohpe and BobbyWoolf fromIBM)
have beenidentifieddocumentedmostcommonandfrequentlyoccurringproblemsandpossible and
efficientwaystosolve those problemswhichwe call todayas Enterprise IntegrationPatterns.
Enterprise IntegrationPatternsletmodern-dayArchitectstosee problemsthroughwell identifiedsetof
patterns,there are over50 well identified patternsandbycombiningthemwe canbuildastrong
solutionsforintegrationproblems.
Where can we foundpatternimplementations?there are three majorcategories,Integration
frameworks,EnterpriseService Buses,IntegrationPlatforms.
IntegrationFrameworks - facilitatedeveloperstoprogramand buildtheirownsolutions,Integration
patternshas beenexposedassetof APIsand librariestoreuse.
Ex – Apache Camel,SpringIntegrationKit.
Enterprise Service Bus –more complex thana Integrationframework,Icouldsayit’sa solutionbuild
usingan IntegrationframeworkbyexposingIntegrationPatternsinconfigurationlevel instead
of programminglevel,mostopensource ESBsusesapache camel asitsunderliningframework.
Ex – Mule ESB, JBossESB, Tibco
IntegrationPlatform –thisisthe nextlevel solution,whichfacilitate muchmore richfeaturestobuild
entire problemdomainontopof it.
Ex- Ws02 Platform- combiningESB,Identityserver,businessrule processor
and APIManager we couldconsideritas full featuredIntegrationplatform.
Kapow - http://kapowsoftware.com/

2. DIP – inveryhighlevel,apartfromclassicESB featureswe alsoprovide our
ownway of custom applicationstomodel specific domainandalsowe have richsetof persistentAPISto
getdata servicesfrombothSQL and NoSQL data basesand more importantlyreal time business
intelligence asa buildinfeature tothe platform.
TIP 3 – Design Philosophy
There are perfectlydesignedandwell-engineeredsystemsinnatural world,suchashumanbody,forests
systemsandevenhumansociety.If somebodyopenhiseyestothe surroundingthenit’spossible to
figure outhowdoesnature isaddressinghighlycomplex problemsinsuchasimple way,withsuch
understandingaparticularcreatormay applythose theoriesintohissolutions.
In DIPthere are suchfewprincipleshasbeenfollowedspeciallybyobservinghumansociety.
1. Personisthe basic entityinsociety –inDIP thisconceptrepresentedasanactor, actors has
much similarpropertiesaspeople dosuchasitsown behaviors,itsownproperties,alanguage
to talkand interactwithotheractors and reactionstoenvironmental andinternal events.
2. SocietyisConcurrent – such as people canact concurrently,actorsalsorun inparallel andthey
have no interestsof whatotheractorsare doingingiveninstance of time,inDIPoperationsare
concurrentby nature.
3. People notwillingtoshare theirownproperties –actors has itslocal memoryspace whichisnot
sharedamongotheractors, whichwill facilitatelockfree operations.Thinkaboutyourself thatif
youshare something(maybe yourcar) withsetof yourfriends,thenyoumightneedtospend
considerable waitingtime until othersfinishwithsharedresource.If youdon’tshare nowaiting
time or no more resource locksrequired.
4. A personmayneedotherspropertiessometimes –it’strue thatI mightneedyourcar some
times,the matterishowthisprocessis happen.If Idirectlygoto your car withsome spare key
and usingitwithoutlettingyouknow mightendupbothof us inpolice station,insteadif I
requestyouthatI needyourcar for 30 ministhenprobablyyouwill lettingme use it.Likewise in
DIP if one actor needtouse some otheractors propertythenthatactor must requestitbyusing
some kindof messaging,thensecondactorknow how to react to the messagingeventby
consideringitspresentstate.
5. SocietyisAsynchronousbynature - whenpeople requestsomethingfromotherstheywill not
waitand stopdoingothertasks until theirrequestgettingfullfilled,inJavaorany otherknown
language a methodcallsare synchronousoperations,inDIPactorsare non-blockingmeansthey
couldrequestsomethingandexecute theirothertasks.
6. SocietyisSoftReal Time – there are some situationswherenature of the problemis
synchronous,evenitsblockingbynature the concernishow longshouldI wait?Nobodyisgoing
to waitforeversomeone other,inDIPeverysyncoperationhasfinite amountof time andin
otherwordsthe blockingoperationmustcompletewithinexpectedamountof time.
Q) WhyHuman societyasa role model whilehumanbodyisfarsmarterengineeringsolution? it’sup
to youto thinkabout,your commentsare highlywelcome.

3. Tip 4 - Let it Crash
In computerscience there are twomajorparadigmsto deal withexceptionalsituations,inone hand
Defensive Programmingsurelythisiswhatall of us thoughtinUniversitylevelwithbasicprogramming
coursesand keeppracticingtill today,inprogrammertermsyouwrite yourcode withintrycatch block
isalmostdefensive programming.
indefensiveprogrammingitis
somethinglikeseeingthe worldas
permanentandconstantthingin
otherwordswe expectprogrammer
to know all exceptionalsituationin
the time of writingthe programor
hide the exceptional situationby
puttingsome notificationintothe
console logforall othercases thatwe
don’tknow.The nature actuallyisnot
a permanentorconstant thingwhere
thingscouldgo wrongas worstas we
neverexpectedbefore,butin
defensive programmingwe believe
that thingscan go wrongbut within
the limitswe know.
In Let it Crash whichis inotherside seeingthe worldassomethingwill gowrongmore oftenthanwe
expected,inthismethodwe letsystemstocrashand go wrong.
Q).What ? yesevensystemsinlive,we letthemcrashwithouthesitation.
The pointis howsoonwe able to recoverwhatwentwrong,againLet itcrash philosophyissomething
that letthingsgowrong bycomingclosure and touchingnatural behaviorsbutasreaderof thistip don’t
discourage bythinkingthatnature isa verycruel thingbecause itletbeautiful andwell-functioning
thingsto crash.
In otherside nature hasdevelopedmuchsmarterwaystorecoverfromerrors and exceptional
situations,asituationwhere yousufferedfromaviral feverisone of a bestexample forit,firstyour
bodyletsome cellsandsome functionstocrash while activatingthe recoveryprocedures,withthis
recoverysame viruswill notattackhumanbodyagain as itknow the virusreallywellenough.
More furtherletitcrash is simplylettingthingsgowrongwhile developingsmarterwaystorecoverfrom
exceptionalsituations.
In DIPwe use mixedapproach,it’snotall defensiveor100% Let it Crash,I couldsay inDIP Core level its
followingLetitCrash andhigherlayersare its almostdefensive.

4. TIP 5 – Supervision
As youalreadyknowdesignphilosophybehindDIPisourownhumansocietyas a role model,inhuman
societywe are playingtwomajorroles;sometimeswe are workerswhoexecuteimportant
functionalitieswhile monitoredbysomebodyelse andsometimeswe become supervisorsand
monitoringotherworkers,thisfactistrue inour childhood,inyoungerage andevenoldage,regardless
of the time thatwe passedor whichisgoingto come infuture.
humanswere being
supervisedbyeither
parents,teachersorby
seniorsinourworking
places.A five yearsold
childprobablydon’tknow
how to act onexceptional
situationsunlesshis/her
supervisorwere watching
the childand deal with
exceptionalandspecial
situations,bythe time
thisinvolvementwill
become minimal aschild
learnhow to act upon
exceptionalsituationby
facinglotof such situations.
Likewise inDIPeveryactorhasa supervisorwhocouldwatchanddeal withexceptional situations,
actually actors are hierarchical ortree like structure thatcouldbe spawnlogicallyupto(n) numberof
levels.
Whenerroneoussituations,the exceptionwill be throwntothe parentof that particularactor and then
parentis usuallyknowshowtodeal withthe situation,parentwill watchthe exceptionandwill take
some action,usuallythere are fouractionsthatparticularparentcouldtake.
1. Tell childtocontinue,thiserrorwill notharmyour execution
2. Restartchild,75% computerbasedproblemssolvedaftersuccessful restart 
3. Stop– thisis a fail situation,there isnopointof yourexistence inthisexceptional situation.
4. Escalate – as a parentI don’tknow what to do,letme ask my parent(whocouldbe veryoldand
experienced)of howtohandle thissituation.
Nowby the time of writingthistipwhile sittinginthe office Ialsohave anexceptional situationaslotof
noise insurroundingwhichdisturbingmyfocus,sofollowingthe same techniqueIwill throw the
exceptiontomysupervisorlayerthoughI didnot getany exceptionsfromthe below layerwhomI
supervised. 

5. TIP 6 – Rich Customers
There wasa recentnewsthatone of DFN customerhas allocatedaHP-UX Superdoomjustfor SMS
Sendingfunctionality,superdoomissomethingbuildwithmassive computingpowerincluding16– 64
processorsperNode,withfourcore pereach processorand2 systemlevel threadspereachcore will
provide 128 individual pure processingentitiestoapplicationevenwithHP-UXminimalconfiguredsuper
doomserver, a superdoomserverissoldforhighprices(60000 USD onward) todayin the market,thatis
oil rich Saudi Arabiawhere whocouldallocate suchamachine justto route tradingsystemgenerated
SMS to the Gateway.These kindof serversare builtwithVertical Scalingfacility,verticalscalingis
somethingthatwhenparticularapplicationisrunningoutof computingresourcesthensystem
administratorsare facilitatedtoeasilyaddingadditional computingpowersuchasmore CPUs, memory
and hard disksinto the systemprobablywhile systemisfullyoperational.
addingmore
computingpower
inhardware level
doesnot
necessarilymean
that application
can fullyutilizedit
and workas we
expected.
Do youbelieve if
somebodytelling
that addingmore
core wouldcause
hisapplication
slowdownand
reducedthe
throughput,which
iscompletelyother
waythan
expected?
Yes,that couldbe true if that programoperatedinmultithreadingwithsharedmemory,the above
statementisnota hypothesisbuttrulyobservedphenomenafromengineersinourindustry.
In multi core environmentyouprobablyhave three level of cache memorylayersbeforethe main
memory,accessingmainmemorywillbe approximately100 timesslowerthanreadingfromCache.Now
thinkthat youhave created8 threadsinyourjava program andeach threadisreadingand modifying
some commonobject,asyou deal withveryhighlevel youhave noscene thatwhichthreadisrunning
on whichcore because it’stotallyupto JVMand underliningOStodeal withit.
Cache coherencyissomethingthatkeepingtrackof value of data as same inbothcache level andmain
memory,applyingthe same theorynowimagine followingsimple scenario.
1. Your java threadone runningincore one goesand write somethingtoshared object(Objectis
nothingbutsome space in memory)
2. Thenyour thread2 whichisrunningwithcore 2 wantto readthe object.

6. By thistime cache memoryof Core 1 andCore 2 are not equal because some threadfromcore one had
updatedthissharedmemorylocationandsystemmustreloaddatafrommainmemorytokeepcache
coherencybutbad thingaboutwhenreloadingisitsdeal withmemoryblocksratherthanoriginal sizeof
the object,meaningevenif yourobjectsize is48bitssystemmustreloadentire 64 bitmemorystructure
if underliningcomputerarchitectureis64 bits,andthis will surelyslow downyourprogramexecution
and speciallywithmore andmore coreswithmore parallel threadsthissituationwillbe gettingworst.
HonestlyinDIPwe can’t overcome thisproblemby100% as DIPdevelopedusingjavaandjavaisdesign
withsharedmemoryinmind,if we hada change to developDIPwithsome goodfunctional language
such as Erlangthenwe couldovercome thisproblem100% as language levelitsshare nothing.
But still we wantto serve andsatisfyourrich customersbysupportingvertical scalabilityasmuchas
possible andhere we use three majortechniquestoovercome above mentionedproblem.
1. In DIPwe use immutable objectsanddatastructureswhere threadsfromseparate core will not
be able to modifybutstill able toread,hence systemwillnotneedtoreloaddatafrommain
memoryas itcouldnot be modified.
2. If you rememberfromTIP3 (People notwillingtoshare theirownproperties),let’skeep data
local to actor but same time withjavait’snot 100% possible aswe needtoshare data between
actors and letthemdirectaccessdata, butwe can still minimize the harmif thatobjectis
immutable.
3. Actors are livinginsomewhere higherthanjavathreadshence we cancreate a threadand pin
that threadto an actor, where executionof thatgivenactoris alwaysthroughthatpinned
thread,withthatwe can avoidcore switchingwhile processingbythatgivenactor andin other
wordsthat actor will only processinCore – 1 if that pinnedthreadwasspawnedfromCore – 1.
Same time I musthighlightthatpinningathreadto an actor isnot goodalwaysas in thiscase that actor
will notbe able to use othercoreseventhose core are inidle state andfree fromprocessing,therefore I
mustsay in DIPwe are not 100% overcome thisproblembutupto80% we are doingfairto our rich
customerswhocouldstrongenoughto buysuperdoomsforevensimple computingproblems.
TIP 7 – Poor and average customers
MTR (MarketTrading Router) isone of applicationinDIPFamilytargetingof sellingtoaverage level
customerswithself-deploymentorwithminimal involvementof DFN Engineers,especiallywithMTRwe
do notask themto by veryhighendserverstodeploythe solution,if somebodywantstotrade withour
rich clientapplication(suchasmobile,pro,ria,ect..) while still usingtheirowntradingbackgroundthey
may probablystartit by
installingMTRinone single
commodityhardware
server.
as yesterdaywe discussed
how to serve richcustomers
whoown veryhigh-end
hardware usingvertical

7. scalingtodayI will explainhowDIPisfacilitate toaverage andpoorcustomerswhorunwithlowed
commodityhardware.
As thispicture showsaparticularusercan start theirbusinesswithone ortwoserversbyspendinga
minimal amountof price ashe couldand thencan increase servercountwhenbusinessismakingprofit
while growingthe userbase;technicallythisiscall Horizontal Scalingof the system.
DIP facilitate Horizontal Scalabilityreallywellwhereyoucanexpandthe systemupto 200 Exchanges
and similaramountof Endpointsingivenenvironmentandthisexpandabilityiscompletely plugand
play meaningadministratorswilldonothingotherthangettingupfew more DIPserversandthen
everythingwill be joinwitheachotherandstartprocessingfromnextfew messages.
A goodexample isanIPOday to explainthe importance of thisfeature,anIPO(Initialpublicoffering)
day isusuallycreatinghuge demandon tradingserversaslotof customersare activelyparticipatingon
same givenmoment.If somebodyfeelsthatexistingserverinfrastructure isnotenoughtoface this
situationwhilekeepingtradingsystemthroughput.
We don’taskusersto buy newhardware or restartentire environmentbefore the IPOdate toplug
those newmachinestorunningenvironment,insteadtheycanutilizedexistinghardware andrun
anotherDIPserver,thenitwill joinwithexistingCusterandwill be readytohandle the loadof IPO. After
the IPO eventwe againdon’taskadministratorstokeepthe serveronrunningbuttheycan just
shutdownDIPprocessandothermembersinthe clusterknowsthatno more such serverisexistingand
stopforwardingmessagestothatserver.
Note - I wouldlike if some of youraise some questionsoratleastsome commentsonwhatI am writing
inlast fewdays,Ihave no encouragementof furtherwritinganddonot feel itlive withoutyour
involvement.
Tip 8 – Wave particle duality (Functional or OOP / Nouns or Verbs)
From the earlydaysof humancivilizationpeoplehada unsolvedmysteriousquestionthatwhetherthe
worldisthinglike orstuff like;inotherwordshow couldsomebodyinterpretthe atomiclevel,whether
it’sas thingswhichiscountable like sandorstuffswhichisequal tosomethinglikewaterthatcanbe
uncountable?Withmore simplificationare atomslookslike objectswhichcarrysome massor theyjust
waves?
withthe growth
of physics,it
createda
separate branch
of physics call
quantum
mechanics
where inthe last
century
scientistsgot
realizedthat
atomiclevel is
not thinglike or
stuff like neither

8. but theywere bothinthe same time!yesactuallytheyare botha wave anda particle at same moment
of time.
The integratingthingisthatthoughwe knew the fact, throughhumansenseswhichwe usedtoobserve
the worldwill onlybe able toobserve one side atgivenmomentandwe couldneverbe able torealize
actual realityaswe can’t see bothsidesinsame time of a given phenomenon.
Thisis notonlyin natural worldwhere physicistsdeal with,itstrue incomputerscience aswell where
those computerscientistswhowere feltthe worldasthingscreatedaprogrammingparadigmcall
ObjectOrientedprogramming while Otherswhohave seensame worldasstuffsprobablycreated
functional programming paradigm.
Each paradigmhas itsown developedwaystounderstandaproblemandsolvingit,inOOPyoumostly
deal withnounsinhigherlevel andthenmapverbsinsidenouns,infunctional waywe directlydeal with
verbsor operations.There are muchmore differencesbetweentwoparadigmsandif youinterestingto
learnmore I couldprovide some goodlearningmaterialsaswell.
So whatis the paradigmusedinDIP ? obviouslyfromoutsiderviewingthe code willsee itsOOP,butits
mainframework(AKKA) waswrittencompletelyonScalawhichisa functional language whileother
usedframeworkssuchasCamel and Springwere fullyobjectoriented,how couldwe glue thosetwo
paradigms?
We somehowovercomethisproblemasseenproblemsasreactive thingswhilenotstickintoboth
paradigmsfully,alsothere are some placeswe appliedOOPprinciplesreallywell suchasinheritance
while there are otherplaceswe appliedfunctional principlesreallywell,conceptssuchasMonads in
functional programminghasbeenappliedtodataservice APIinESBand itsworksreallywell.
The summaryof thisTIP isas designersif yournatural view of worldisthingslike thengowithOOP
whichiswell suitable foryoubutif yousee everythingaroundyouassomethingoperational andstuff
like orif you coulddescribe problemsthroughmathematicswell,thenfunctionalstyle isagoodchoice
for you.
Note – C++, Java,C# are good example forlanguagesfor OOPbutwithJava 8 its movingtowards
functional principlessuchaslambdaexpressions,higherorderfunctionsect.
Erlang,Hascal, Clojure are goodexamplesforfunctional programminglanguages.
Java Scriptsand Rubycan alsobe consideredasfunctional languagesbutare notpure
functional languages.
Scala playa middle role,itsletprogrammertochoose hisparadigmwhere languagesupport
both.
Tip 9 – Files everywhere
If you had a change to go in the documentstore roomof departmentof motortraffic,thisiswhat it
usuallylookslikeeventodayinthisdigital era,withlotof fileseverywheretokeepinformationnotonly
aboutvehiclesthathave beenregisteredwith the departmentbutalsotheyhave lotmore other

9. informationtokeep.Mostof those fileswere traveledwithsome othergovernmentdepartmentssuch
as Police,Customsect..
thisishow service
orientedarchitecture
lookslike inearlydays,
everythingmanual and
exposedasservicesand
each department
providingitsownspecific
services,aparticularfile
mightneededtogoin
circularjourneysthrough
separate departmentsin
orderto full fill total
requirement.
Those daysthe only
integrationmechanism
between
interdepartmental
informationflow wasfiles,
and inotherwordsa
particularfile will have all the informationandstatusabouta particularvehicle registeredwith
departmentof motortraffic.
If you ask me that whatis the mostcommon integrationmechanismtointegrate betweensystemsfor
today’sworld,thenmyanswerwouldbe files.Actuallyfilebasedintegrationisthe mostcommonand
simplestformusedinITsystems.Infilesbasedintegrationssomebodyhastoproduce file whoisthe
source systemor producer,andthere mustbe some othersystemto consume those informationand
getparticularaction onthat.
DIP supportsfile handlingreallywell,itsletyoucreate fileconsumersandproducersthroughsome easy
fewcommands,andalsoletyoucontrol those consumersandproducersinruntime throughCLI, there
are 100s of settingtocontrol those file endpointssuchasthe locationof file,where tobackupect.
Eventhoughit’ssimple wayof systemsintegrationthere are some specificproblemsinvolvedwiththis
method.If updatedoccurinfrequentlythensystemsmaybe outof synchronization,asanexample if
customermanagementsystemchangedthe addressof particularcustomerandnotupdate the billing
systemontime thenthe deliveryof goodsprobablygotosome wrongplace.
FilesalsonotsupportingXA transactions,whereyoucan’tuse fileswithtransactionalsystemssuchas
J2EE or TUXIDO, alsofile consumersmightblocksome of systemthreadsif youdonotmanage it
properly.
So keepinmindthose factswhenyouplanwithfile basedintegrationbetweentwoormore systems,it’s
verysimple methodof systemsintegrationandrequire nospecificprogrammingknowledge toget
thingsdone butinthe same time ithas some itsownspecific problems.