This document discusses setting up automated monitoring of a BizTalk environment using BAM (Business Activity Monitoring). It outlines several challenges faced with different implementation attempts, including issues with tools timing out or exceeding limits. Recommendations provided include using BAM in stable environments, scaling infrastructure appropriately, and knowing when to use tracking instead of BAM.

2. We’re gonna talk about...

3. • How is BAM working? Details....
• Challenges
• AIMS BAM setup....in theory
• Practial implementation, bugs and recommendations
• Live experiences
• Conclusion
Agenda

4. • Data from ports & orchestrations
• Realtime data (close to)
• Not many prerequisites
• Performance data
• Low impact - > tracking / pipelines is not an option
• Automated deployment
• Autmated maintenance / updates
• Supports BTS2006R2 and newer
Challenges

5. • Integrated BizTalk functionality
• Easy to enable, few prerequisites
• Selectable tracking
• Low overhead compared to tracking (unless you use it the same
way as tracking)
• Global tracking can be disabled
• No big changes in different BizTalk versions
• Tracking profiles can be deployed «in flight», adapts to changes
We ended up with BAM

6. How is BAM working?
MsgBox
BAMPrimary
(activities)
Asynch
TDDS
Synch
Eventstreams
Tracking
Tracking

7. • Scan BizTalk MgmtDb for artifacts & dependencies
• Scan DTADb for orchestration XML to find shapes / orch ports
• Dynamically deploy new activities / tracking based on changes
How to automate the monitoring
setup with BAM?
That was the theory......sounds simple right?

8. • One activity for all orchestrations (ports & call / start shapes)
• One activity for all receiveports
• One activity for all sendports
• Three trackingprofiles; receiveports / sendports / orchestrations
• Used Microsoft.BizTalk.Bam.TrackingCompiler.TrackingCompiler
/ bm.exe
ISSUES
• Limit in number of fields in an activity definition
• We had to create one unique field per shape per orchestration
AIMS BAM – take 1

9. • One activity per each common orchestration shape
• One activity for all receiveports
• One activity for all sendports
• One tracking profile receiveports, one for sendports, separate
tracking profiles for each orchestration common shape activity
• Used Microsoft.BizTalk.Bam.TrackingCompiler.TrackingCompiler
/ bm.exe
ISSUES
• Win32Exceptions on the TrackingCompiler (limit on open handles
exceeded) due to large amount of ports in orchestrations
• GUI handles in the TrackingCompiler, probably for TPE
• Timeouts
AIMS BAM – take 2

10. • One activity per each common orchestration shape
• One activity for all receiveports
• One activity for all sendports
• One tracking profile receiveports, one for sendports, separate
tracking profiles for each orchestration common shape activity
• Used Microsoft.BizTalk.Bam.TrackingCompiler.TrackingCompiler
/ bm.exe
ISSUES
• Win32Exceptions on the TrackingCompiler (limit on open handles
exceeded) due to large amount of ports in orchestrations
• GUI handles in the TrackingCompiler, probably for TPE
• Timeouts
AIMS BAM – take 2

11. • One activity per each common orchestration shape
• One activity for all receiveports
• One activity for all sendports
• One tracking profile receiveports, one for sendports, separate
tracking profiles for each orchestration common shape activity
• Bttdeploy.exe/ bm.exe
ISSUES
• Win32Exceptions on the bttdeploy.exe due to large amount of
ports in orchestrations
• GUI handles in the TrackingCompiler, probably for TPE. TPE
dependencies in bttdeploy.exe (no logic, should been the other
way around)
• Timeouts
AIMS BAM – take 3

12. • One activity per each common orchestration shape
• One activity for all receiveports
• One activity for all sendports
• One tracking profile receiveports, one for sendports, separate
tracking profiles for each orchestration common shape activity
• Bttdeploy.exe/ bm.exe but this time more & smaller tracking
profiles
ISSUES
• Win32Exceptions on the TrackingCompiler due to large amount of
ports in orchestrations
• GUI handles in the TrackingCompiler, probably for TPE
• Timeouts
AIMS BAM – take 4

13. • One activity for each orchestration shape
• Multiple activities for all receiveports (100 per activity)
• Multiple activities for all sendports (100 per activity)
• Multiple tracking profiles for receiveports, multiple for sendports,
separate tracking profiles for each orchestration shape activity
• Bttdeploy.exe/ bm.exe
ISSUES
• Time consuming, thousands of trackingprofiles / activities
• No changes to the BizTalk possible while deploying
• Orphaned trackingprofiles
AIMS BAM – take 5

14. • Good up to certain configs (6000+ components) /
high throughput
• TDDS not «cleaning» MsgBox quickly enough, leading to
throttling
• Condition worsening if tracking hosts & BAM are not properly
scaled
• Messaging has priority over TDDS if running TDDS on shared host
(which still happens!)
• Tested on multi MsgBox setups, scaled out BAM (dedicated
server), single box setup, multi server setup, different clustering
of BizTalk servers and SQL servers
Performance

15. • Test & dev environments are subject to
rapid changes
• New TP deployments / removals
necessary for each change
• Takes time......and in the middle of this
someone runs a deploy / undeploy
• In many cases lead to orphaned
tracking profiles
Test / Dev environments

16. • BTS 2009 / 2010 with BAM and passthrough pipelines -> stopped
processing of messages
• No existing fixes for BTS2009
• BTS2010 CU 2-5 solved issue, CU 6-7 breaks support
• DB locking during long deploys, caused uncomplete setup
• XML encoding issues in tracking profiles (unknown chars)
• Orchestrations without XML in DTADb
Other bugs & issues
XML declaration is removed at a receive location that uses
BAM tracking and the PassThruReceive pipeline in BizTalk
Server 2009 / 2010

17. • Use in stable environments with low frequency on changes and
make sure you update your BAM tracking accordingly
• Make sure you scale TDDS hosts, use dedicated tracking hosts
• Use synch BAM if you want to bypass the MsgBox. However,
synch «eats» cycles of the processing and can only be done
from code (orchestrations, pipelines, externals etc)
• Use asynch if you care about performance. Only option when
using bttdeploy.exe. MsgBox is used for caching so monitor its
size carefully
• Ensure that your BAM SSIS packages are running
• Know how to clean out orphaned tracking
• Know when to use tracking instead of BAM. BAM is not
necessary or good for everything
Recommendations

18. AIMS conclusion