Presentation by our CTO Marius Storsten at Integration Day in Belgium June 4th 2015

2. We’re gonna talk about...

3. • How is BAM working? Details....
• Challenges
• AIMS BAM setup....in theory
• Practial implementation, bugs and recommendations
• Live experiences
• Conclusion
Agenda

4. • Data from ports & orchestrations
• Realtime data (close to)
• Not many prerequisites
• Performance data
• Low impact - > tracking / pipelines is not an option
• Automated deployment
• Autmated maintenance / updates
• Supports BTS2006R2 and newer
Challenges

5. • Integrated BizTalk functionality
• Easy to enable, few prerequisites
• Selectable tracking
• Low overhead compared to tracking (unless you use it the same
way as tracking)
• Global tracking can be disabled
• No big changes in different BizTalk versions
• Tracking profiles can be deployed «in flight», adapts to changes
We ended up with BAM

6. How is BAM working?
MsgBox
BAMPrimary
(activities)
Asynch
TDDS
Synch
Eventstreams
Tracking
Tracking

7. • Scan BizTalk MgmtDb for artifacts & dependencies
• Scan DTADb for orchestration XML to find shapes / orch ports
• Dynamically deploy new activities / tracking based on changes
How to automate the monitoring
setup with BAM?
That was the theory......sounds simple right?

8. • One activity for all orchestrations (ports & call / start shapes)
• One activity for all receiveports
• One activity for all sendports
• Three trackingprofiles; receiveports / sendports / orchestrations
• Used Microsoft.BizTalk.Bam.TrackingCompiler.TrackingCompiler
/ bm.exe
ISSUES
• Limit in number of fields in an activity definition
• We had to create one unique field per shape per orchestration
AIMS BAM – take 1

9. • One activity per each common orchestration shape
• One activity for all receiveports
• One activity for all sendports
• One tracking profile receiveports, one for sendports, separate
tracking profiles for each orchestration common shape activity
• Used Microsoft.BizTalk.Bam.TrackingCompiler.TrackingCompiler
/ bm.exe
ISSUES
• Win32Exceptions on the TrackingCompiler (limit on open handles
exceeded) due to large amount of ports in orchestrations
• GUI handles in the TrackingCompiler, probably for TPE
• Timeouts
AIMS BAM – take 2

10. • One activity per each common orchestration shape
• One activity for all receiveports
• One activity for all sendports
• One tracking profile receiveports, one for sendports, separate
tracking profiles for each orchestration common shape activity
• Used Microsoft.BizTalk.Bam.TrackingCompiler.TrackingCompiler
/ bm.exe
ISSUES
• Win32Exceptions on the TrackingCompiler (limit on open handles
exceeded) due to large amount of ports in orchestrations
• GUI handles in the TrackingCompiler, probably for TPE
• Timeouts
AIMS BAM – take 2

11. • One activity per each common orchestration shape
• One activity for all receiveports
• One activity for all sendports
• One tracking profile receiveports, one for sendports, separate
tracking profiles for each orchestration common shape activity
• Bttdeploy.exe/ bm.exe
ISSUES
• Win32Exceptions on the bttdeploy.exe due to large amount of
ports in orchestrations
• GUI handles in the TrackingCompiler, probably for TPE. TPE
dependencies in bttdeploy.exe (no logic, should been the other
way around)
• Timeouts
AIMS BAM – take 3

12. • One activity per each common orchestration shape
• One activity for all receiveports
• One activity for all sendports
• One tracking profile receiveports, one for sendports, separate
tracking profiles for each orchestration common shape activity
• Bttdeploy.exe/ bm.exe but this time more & smaller tracking
profiles
ISSUES
• Win32Exceptions on the TrackingCompiler due to large amount of
ports in orchestrations
• GUI handles in the TrackingCompiler, probably for TPE
• Timeouts
AIMS BAM – take 4

13. • One activity for each orchestration shape
• Multiple activities for all receiveports (100 per activity)
• Multiple activities for all sendports (100 per activity)
• Multiple tracking profiles for receiveports, multiple for sendports,
separate tracking profiles for each orchestration shape activity
• Bttdeploy.exe/ bm.exe
ISSUES
• Time consuming, thousands of trackingprofiles / activities
• No changes to the BizTalk possible while deploying
• Orphaned trackingprofiles
AIMS BAM – take 5

14. • Good up to certain configs (6000+ components) /
high throughput
• TDDS not «cleaning» MsgBox quickly enough, leading to
throttling
• Condition worsening if tracking hosts & BAM are not properly
scaled
• Messaging has priority over TDDS if running TDDS on shared host
(which still happens!)
• Tested on multi MsgBox setups, scaled out BAM (dedicated
server), single box setup, multi server setup, different clustering
of BizTalk servers and SQL servers
Performance

15. • Test & dev environments are subject to
rapid changes
• New TP deployments / removals
necessary for each change
• Takes time......and in the middle of this
someone runs a deploy / undeploy
• In many cases lead to orphaned
tracking profiles
Test / Dev environments

16. • BTS 2009 / 2010 with BAM and passthrough pipelines -> stopped
processing of messages
• No existing fixes for BTS2009
• BTS2010 CU 2-5 solved issue, CU 6-7 breaks support
• DB locking during long deploys, caused uncomplete setup
• XML encoding issues in tracking profiles (unknown chars)
• Orchestrations without XML in DTADb
Other bugs & issues
XML declaration is removed at a receive location that uses
BAM tracking and the PassThruReceive pipeline in BizTalk
Server 2009 / 2010

17. • Use in stable environments with low frequency on changes and
make sure you update your BAM tracking accordingly
• Make sure you scale TDDS hosts, use dedicated tracking hosts
• Use synch BAM if you want to bypass the MsgBox. However,
synch «eats» cycles of the processing and can only be done
from code (orchestrations, pipelines, externals etc)
• Use asynch if you care about performance. Only option when
using bttdeploy.exe. MsgBox is used for caching so monitor its
size carefully
• Ensure that your BAM SSIS packages are running
• Know how to clean out orphaned tracking
• Know when to use tracking instead of BAM. BAM is not
necessary or good for everything
Recommendations

18. AIMS conclusion