Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to BizTalk Custom Adapters Toronto Code Camp Presentation
Similar to BizTalk Custom Adapters Toronto Code Camp Presentation (20)
BizTalk Custom Adapters Toronto Code Camp Presentation
- 1. Developing BizTalk Custom Adapters Moustafa Refaat Principle Software Consultant, Genetic Thought mrefaat@GeneticThought.com www.GeneticThought.com www.MoustafaRefaat.com
- 2. What and Why BizTalk? BizTalk Adapters. Native Custom Adapters WCF Adapters WCF LOB Adapter Introduction 2 www.MoustafaRefaat.com
- 3. Business Process Management. Service Oriented Architecture (SOA). Enterprise Services Bus (ESB) platform. Tools and application services that facilitate the rapid creation of integration solutions. Exchange of data in a standard, consistent and reliable way. What is BizTalk 3 www.MoustafaRefaat.com
- 4. Why BizTalk? 4 www.MoustafaRefaat.com
- 6. Messaging Orchestration engine Business Rule Engine (BRE) Health and Activity Tracking (HAT) Enterprise Single Sign-On (SSO) Business Activity Monitoring (BAM) Business Activity Services (BAS) BizTalk Tools 6 www.MoustafaRefaat.com
- 7. How BizTalk Works? 7 www.MoustafaRefaat.com
- 8. Messages received Contextual properties. Matching subscriptions. Application-specific queue, instance of a service. Queued messages de-queued designated instances of services. The service handles the message. Publish/Subscribe mechanism 8 www.MoustafaRefaat.com
- 9. BizTalk Adapters 9 www.MoustafaRefaat.com
- 10. COM or .NET Framework components Transfer messages to and from business end points (such as file systems, databases, and custom business applications) by using various communication protocols (TCP/IP, HTTP, File System, MSMQ, etc). 10 What is a BizTalk Adapter www.MoustafaRefaat.com
- 11. Types of BizTalk Adapters Native Adapters FTP, File, SOAP, HTTP WCF Adapters WCF-BASIC, WCF-HTTP .. WCF LOB Adapters www. MoustafaRefaat.com
- 12. BizTalk and Service Orientation Expose all different types of adapters through WSDL and XSD metadata documents that describe the adapter’s messaging contracts. With BizTalk adapter framework create a custom adapter that can be consumed in BizTalk as existing BizTalk adapters through a consistent, service-oriented, model that relies on WSDL and XSD metadata artifacts www. MoustafaRefaat.com 12
- 13. BizTalk Adapters "native" or "integrated" adapters FILE, FTP, HTTP, MQSeries, MSMQ, POP3, SMTP, SOAP, SQL, Windows Sharepoint Services WCF Adapters WCF-WSHttp, WCF-BasicHttp, WCF-NetTcp, WCF-NetMsmq, WCF-NetNamedPipe, WCF-Custom, and WCF-CustomIsolated www. MoustafaRefaat.com
- 14. Adapters Function Associated with a receive location designed to listen for messages from a certain transport at a certain address. Message is received by the receive location, it is passed to the adapter. The adapter attaches the data stream to the message, adds any metadata pertaining to the endpoint that the data was received from, submits that message into the BizTalk Messaging Engine www. MoustafaRefaat.com
- 15. Message Exchange Patterns One-Way (Asynchronous) Request-Response Style Protocols Solicit-Response Style Protocols. Request-Multiresponse. Loop-Back. 15 www.MoustafaRefaat.com
- 16. Hosting the adapter In-process (BTSNTSrvc.exe). Send and Receive. Instantiate the adapter Transport prox. Services the adapter's requests. Terminates the adapter. Adapter and endpoint configuration. Isolated adapters (out of Process) Receive only. 16 www.MoustafaRefaat.com
- 17. Why write a custom Adapter Specific custom application Use a protocol for which a native adapter does not exist Writing a custom adapter is challenging exercise. Adapter Framework simplifies custom adapter development Sample adapter source code in the BizTalk Server SDK. 17 www.MoustafaRefaat.com
- 18. Types of Custom Adapters Native Custom Adapter WCF Custom Adapter WCF LOB Custom Adapter www. MoustafaRefaat.com
- 19. The Adapter Framework Microsoft.BizTalk.Adapter.Framework interfaces enable adapters to: Provide a means to modify configuration property pages. Import services and schemas into the BizTalk project. 19 www.MoustafaRefaat.com
- 20. Adapter Framework Tools Administration console BizTalk Explorer 20 www.MoustafaRefaat.com
- 21. Adapter Framework Tools Metadata Wizard Property Pages 21 www.MoustafaRefaat.com
- 22. Adapter Framework Components IBaseMessage Interface Create a message to pass to the Messaging Engine. One or more message parts represented by the IBaseMessagePart interface. Each message part has an IStream interface pointer. Context of a message is IBaseMessageContext interface www. MoustafaRefaat.com
- 23. Adapter Framework Components Messaging Engine Interfaces IBTTransportProxy Transport proxy, creating batches, getting the message factory, and registering isolated receivers with the engine. IBTTransportBatch defines methods that adapters can use to operate on batches of messages. The Messaging Engine processes batches asynchronously. IBTDTCCommitConfirm for MSDTC transactions. www. MoustafaRefaat.com 23
- 24. Adapter Framework Components Adapter Interfaces IBaseComponent: Name, Version, and Description of the adapter. IBTTransport: Transport Type and ClassID of the adapter. IBTBatchCallback: callback interface through which the adapter receives status and error information for a batch of messages it submits to the Messaging Engine. IPersistPropertyBag(optional): configuration interface through which handler configuration is delivered to the adapter. IBTTransportControl(Optional): used to initialize and terminate an adapter not required for isolated adapters. www. MoustafaRefaat.com 24
- 25. IPropertyBag void Read ([InAttribute] string propName, out Object ptrVar, interrorLog) void Write ( [InAttribute] string propName, [InAttribute] ref Object ptrVar ) www. MoustafaRefaat.com
- 26. Adapter Components AdapterRegistryWizard.exe Design-Time Component Run-Time Component www.MoustafaRefaat.com 26
- 27. Adapter Instantiation www.MoustafaRefaat.com 27
- 28. Practical Adapter Development Program Files (x86)icrosoft BizTalk Server 2009DKamplesdaptersDevelopment BizTalk Adapter Wizard http://biztalkadapterwizard.codeplex.com/ www. MoustafaRefaat.com 28
- 29. The Unzip Adapter www.MoustafaRefaat.com 29
- 32. ProcessMessage Method ///<summary> /// Implementation for AsyncTransmitterEndpoint::ProcessMessage /// Transmit the message and optionally return the response message (for Request-Response support) ///</summary> public override IBaseMessageProcessMessage(IBaseMessage message) { IBaseMessage result = null; Stream source = message.BodyPart.Data; // build url UnzipTransmitProperties props = new UnzipTransmitProperties(message, propertyNamespace); // string filePath = FaxAdapterTransmitProperties.CreateFileName(message, props.Uri); UnzipManager um = new UnzipManager(props.TargetDirectory); um.UnzipStream(source); return result; } www. MoustafaRefaat.com
- 33. Transmst Handler Send Adapter www. MoustafaRefaat.com
- 34. Transmit Location Send Port www. MoustafaRefaat.com
- 35. The Adapter Management Part public string GetConfigSchema(ConfigType type) { switch (type) { case ConfigType.ReceiveHandler: return null; case ConfigType.ReceiveLocation: return null; case ConfigType.TransmitHandler: return GetResource("MoustafaRefaat.UnzipAdapterManagement.TransmitHandler.xsd"); case ConfigType.TransmitLocation: return GetResource("MoustafaRefaat.UnzipAdapterManagement.TransmitLocation.xsd"); default: return null; } } www. MoustafaRefaat.com
- 36. How to Deploy a Custom Adapter C:rogram Files (x86)icrosoft BizTalk Server 2009DKtilitiesdapterRegistryWizard www. MoustafaRefaat.com
- 37. Windows Registry www. MoustafaRefaat.com
- 38. Adapters Framework Final Notes BizTalk adapter framework Tremendous business value in complex environments Adapters to be consumed within BizTalk Server Only. You cannot consume a BizTalk adapter from traditional .NET code. www.MoustafaRefaat.com 38
- 39. WCF Adapters WCF-WSHttp adapter. WS-* standards WS-Transaction WS-Security WCF-BasicHttp adapter. ASMX WS-I Basic Profile 1.1. WCF-NetTcp adapter. WS-* standards support over the TCP transport. WCF-NetMsmq adapter. Microsoft Message Queuing (MSMQ) WCF-NetNamedPipe adapter. on-machine cross-process communication. WCF-Custom adapter. WCF extensibility features. select and configure a WCF binding and the behavior information for the receive location and send port. WCF-CustomIsolated adapter. WCF extensibility features over the HTTP transport. select and configure a WCF binding and the behavior receive location running in an isolated host. www. MoustafaRefaat.com
- 40. WCF bindings A binding in WCF is responsible for “binding” the logical messages used within the service layer to the underlying channel layer. A binding defines all of the details around how a message will be processed between your .NET service code and the underlying network or communication medium that’s used to transmit the message. Specify security processing, message encoding, a transport channel details. www. MoustafaRefaat.com
- 41. WCF Adapters Benefits Support for WS-* Richer “pipeline” model for manipulating messages as they move in and out of the server. WCF services can be consumed from any .NET application (NOT restricted to BizTalk). www. MoustafaRefaat.com
- 43. Service-enablement Issues Consume any LOB application that has been service-enabled. A service façade that provides WSDL and XSD metadata. How to effectively expose metadata to the consumer. Some applications expose a vast underlying data source impossible to serve up a single WSDL The consumer “what data” they’re interested in up front and generate specialized metadata files for that particular usage. Impact the performance and throughput of the system. HTTP and XML negative effect WCF LOB Adapter SDK provide a simplified and unified model for creating custom BizTalk adapters that can be used from any WCF client application including BizTalk Server. The SDK makes it much easier to provide metadata-driven adapters that can embrace native LOB protocols and formats when necessary. www. MoustafaRefaat.com
- 44. Consuming WCF LOB Adapters Add Adapter Service Reference plug-in Consume Adapter Service Reference plug-in www. MoustafaRefaat.com
- 45. When Should I Write a Custom LOB Adapter? Do not if A Web service façade already exists An adapter already exists transport such as FTP or SMTP Do if LOB application expose multiple operations and a wide variety of data Different consumers need to consume your LOB application differently? LOB application have native programming interfaces or protocols www. MoustafaRefaat.com
- 46. Building Custom Adapters with the WCF LOB Adapter SDK metadata support connection management message handling www. MoustafaRefaat.com
- 47. Steps for Building a Custom Adapter Choose a protocol scheme for the custom adapter (e.g., “myhttp://…”) Decide on the directionality (outbound vs. inbound) and whether it needs to be asynchronous. Decide on the metadata options your custom adapter will provide. Identify any adapter properties you’ll want to expose for end-user customization. Run the WCF LOB Adapter Development Wizard and specify your choices. Familiarize yourself with the generated Visual Studio solution. Finish implementing the provided connection, metadata, and runtime classes. www. MoustafaRefaat.com
- 48. Using the WCF LOB Adapter Development Wizard www. MoustafaRefaat.com
- 49. WCF LOB Adapter Wizard www. MoustafaRefaat.com
- 50. WCF LOB Adapter Wizard www. MoustafaRefaat.com
- 51. Handler and Endpoint schemas www. MoustafaRefaat.com
- 52. Generated Project www.MoustafaRefaat.com
- 53. Implementing the Connection and Binding Classes The connection factory class has a default implementation that will create connections when requested by the Adapter SDK runtime The connection URI class requires you implement Uri parsing logic write the LOB-specific connection code. manage the opening and closing of connections to the LOB system. ClearContext method to prepare a connection object to be put back in the pool. IsValid method to return a status indicating whether the open connection is in a valid state or not. The four binding-related classes provide the WCF packaging that makes the adapter easy to use. A binding class, A binding element, and A binding element extension supplied in the new project wizard all the way from the configuration up to the adapter class itself www. MoustafaRefaat.com
- 54. Implementing the Metadata Classes The browse and search classes One method return array of MetadataRetrievalNode objects representing the operations that can be performed on the adapter. Definitions of named items that can be shown to a developer using the adapter You’ll need to do your mapping to the LOB system. The metadata resolver is responsible for resolving metadata identifiers to operation descriptions. www. MoustafaRefaat.com
- 55. Implementing the Remaining Runtime Classes The outbound handler execute method receives a WCF message object and is expected to return the same. Where the real communication happens. Handler might need: a connection and metadata to provide information about a specific operation. Base class for the handler provides the connection and a MetadataLookup object that can be used to query for metadata. You to decide how to translate that information into an LOB API call, and how to translate the result back into a WCF Message object. www. MoustafaRefaat.com
- 56. Implementing the Remaining Runtime Classes The inbound handler is complicated Begin listening for messages get them into the system when they arrive. Implement StartListener and StopListener to control when the adapter should start or stop whatever specific mechanism it has for receiving messages. WaitForMessage TryReceive. www. MoustafaRefaat.com
- 58. WCF LOB Custom Adapters www.MoustafaRefaat.com
- 60. Conclusion Writing Custom adapters is not an easy task but not hard You can write custom Native BizTalk Adapters WCF extensions in WCF-CUSTOM adapter Custom WCF LOB Adapter www. MoustafaRefaat.com
- 61. Thanks to our contributors
Editor's Notes
- * I would like to get to know you, can we go through and in 2 min everyone introduce himself
- BizTalk is a Business Process Management; Service Oriented Architecture (SOA), or an Enterprise Services Bus (ESB) platform. You can think of BizTalk as a group of tools and application services that facilitate the rapid creation of integration solutions. Most enterprises are looking for BizTalk to enable unrelated and disconnected systems to exchange data in a standard, consistent, and reliable way. The tools that BizTalk provides allow you to design reliable and robust solutions faster than is often achievable by custom coding a solution from scratch using standard development languages and tools. With this in mind, it is important to understand what BizTalk achieves well and what is does not in order to fully realize any efficiency what a BizTalk solution can bring. If you were to look at two BizTalk projects that address identical problems, designed by two different groups, you would see two completely different solutions. Both solutions would solve the problem they were designed to address; however, you will find that one solution utilizes more BizTalk tools. This solution is the better one because it will be more robust and easier to maintain. Think of BizTalk as a toolkit, and within this toolkit, you find tools to help you build your application.
- If two applications need to establish some form of communication between them, this can be done in many ways. For example, application “A” needs information from application “B”, which was developed prior to “A”. This means “B” isn’t even aware of “A”’s existence. Luckily, the developers of “B” have exposed an API to integrate with. Application “A” could thus call “B”’s API to gather the information it needs. No problem, works fine, life’s easy isn’t it? Not really, integrating applications is usually about making assumptions. And in the case of our example: we’ve made many assumptions, such as: “B” is running while “A” makes the call. Communication links between “B” box and “A” box are stable, secure, well established, etc.“B” runs on a similar platform as “A” does, i.e. same data representation. “A” is programmed in a language that can consume “B”’s API.“A” will never need another application to provide the information.Now, imagine that you have four or six applications that you need to integrate. Even if you only have two applications as above imagine following scenarios for a moment: What if we wanted to replace “B” by another, better application? What if we wanted to phase-out “B” and move it to another box or cluster it? What if application “B” is down while “A” needs it?Because “A” makes a few assumptions about “B”, your integration solution might run into serious trouble. Loose coupling of applications is exactly about preventing these kinds of problems. The fewer assumptions an application makes when integrating with another application, the more loosely coupled they are.Messaging is a very popular way to reduce coupling between applications. Messaging relies on middle-ware to transmit chunks of data (“messages”) from one application to another. Usually this is done using a kind of “channel” called “message queues”. The nice thing about queuing is that it works asynchronously. This means that the sending application can put messages in the queue while the receiving application can pull the messages out of the queue as soon as it has time for it.This “spaghetti” kind of integration is likely to become unmanageable. Having a lot of point to point connections acting separately from each other is not only unmanageable; it also makes “business overviews” impossible. What do I mean with “business overviews” is, if you succeeded in connecting each and every system in an insurance organization claim processing systems, you’d expect to be able to make queries like: “How many claims were approved today?”.
- Architecturally, a Hub and Spoke Integration services are centralized processing machinery “the hub”, that accept messages from multiple applications, the Spokes. Applications interact with the hub through adapters deployed on the hub, so the spokes (applications) do not require any modifications. The hub manages routing, mapping and tracking messages between applications. The hub and spoke mode provide strong total cost of ownership by reducing the cost to add and remove connections and through centralized management.The hub and spoke integration services lead to the “publish/subscribe” pattern. In the publish/subscribe pattern, two parties communicate without any shared knowledge other than the agreement on the message format. These two parties are often referred to as the producer and the consumer or – more appropriately in the context of publish subscribe – the “publisher” and the “subscriber”. The publisher publishes its output using messaging to the pub/sub engine while the subscriber subscribes itself with the engine using a filter. The filter, based upon message content criteria, specifies which messages are of interest to the subscriber. BizTalk is Microsoft implementation of the Hub and Spoke architecture. BizTalk is the Hub that receives messages from the spokes and routes these messages to their destinations.
- The design of a BizTalk solution begins with selecting which of the BizTalk tools to use for each of these three simple yet all-important tasks (receive, process, and send a message). I'll introduce these tools in this chapter and continue to explore them further throughout the rest of the book. First, let’s start with a simpler overview: Messaging is a component that provides the ability to communicate with a range of other software. BizTalk introduces the concept of Ports and adapters. Ports and adapters provide the logical abstraction for sending and receiving messages to and from BizTalk. They allow you to code your application in a generic fashion and not worry about the implementation details of how these messages will be consumed and delivered. A port is a logical construct that can receive and send messages to/from the BizTalk MessageBox. A receive port must be associated with a specific receive location to accept the message. The receive location is tied to a specific adapter, which provides the implementations of the Transport protocol, and a pipeline. A Pipeline provides a way to examine, verify, and potentially modify messages as they are received and sent out from BizTalk. I will discuss Messaging in more details in chapters 2, 3, and 4. Orchestration engine supports the creation and running of graphically-defined processes. Orchestration is the most powerful tool within the BizTalk Server toolbox. Think of orchestrations as a special form of workflows similar to the Windows Workflows Foundations. Orchestrations are used to graphically model process workflow and provide the primary mechanism to implement business process automation within BizTalk based project. Orchestrations are created within Visual Studio and compiled into .NET assemblies that are registered in the BizTalk Management Database. Assemblies deployed to this database must also be installed into the Global Assembly Cache (GAC). I will discuss Orchestrations in more detail in Chapter 6. A Business Rule Engine (BRE) that evaluates complex sets of rules. The BRE allows you to model business rules using a simple GUI. It is designed to allow for versioning and modification of implemented business rules without the need to change the processes within BizTalk that uses them. You use the BRE to implement logic that requires frequent updates such as a discount policy percentage or calculations that are updated frequently as a result of legal or government regulations. I will discuss the Business Rule Engine in more detail in Chapter 7. The Health and Activity Tracking (HAT) tool lets you monitor and manage the engine services. HAT will be presented in more details in section 1.2.3 later in this chapter. Enterprise Single Sign-On (SSO) facility which provides the ability to map authentication information between Windows and non-Windows systems. Business Activity Monitoring (BAM) framework you use to build tracking and monitoring into you BizTalk based solutions.Information workers will use it to monitor the state of the running business processes. Business Activity Monitoring (BAM) and Business Activity Services (BAS) provide the infrastructure to perform application instrumentation and metric reporting. BAM provides the ability to instrument your application to provide business-level data that is much more relevant than the default system-level information which is available in the base product. I will discuss BAM in chapter 9. Business Activity Services (BAS), which is used by information workers to set up and manage interactions with transacting partners. BAS provides a simple yet powerful way to display metric data from BAM and other system-level subservices using Microsoft SharePoint Portal Services. Most organizations will integrate the BAS portal into an existing SharePoint infrastructure rather than build an entire SharePoint site around BAS itself.
- BizTalk is designed to receive inbound messages, pass them through some form of logical processing, and then deliver the result of that processing to an outbound location or subsystem. BizTalk views all data and events as “Messages”. A messageis a finite entity within the BizTalk MessageBox. Messages are immutable once they are published to the MessageBox. Messages have context properties and zero-to-many message parts. BizTalk extends the idea of TCP/IP Port to include other protocols such as file, SQL , etc. A Port is either a “receive location” (i.e. BizTalk would listen on this port for incoming messages) or a send port that BizTalk will send through messages. BizTalk groups receive locations into what it calls a “receive port” and groups send ports into send port group (I know the naming is a little bit confusing). A receive location receives messages using a component called “BizTalk Adapter”. A BizTalk Adapter is a component that understands the Transport Protocol like HTTP, FTP etc. The received message is processed by a pipeline associated with the receive location. A pipeline is a component that performs transformations on messages to prepare them to enter or leave BizTalk Server MessageBox database. In the receive locations, the main task of a receive pipeline is to convert input data into proper XML. Likewise, in send ports, the main task of a send pipeline is to convert input data into the proper data format for the transport protocol (Adapter).
- Messages are received (via receive handlers or from orchestration Send ports) and handed to the BizTalk engine. The engine stores data about the message called contextual properties. The engine queries a rule store to determine a set of matching subscriptions. For each matched subscription, a record is entered into an application-specific queue, and is associated with a specific instance of a service. Queued messages are then de-queued using a number of separate worker threads, and routed to the designated instances of services.The service (either an orchestration or a send port) handles the message.
- BizTalk is designed to receive inbound messages, pass them through some form of logical processing, and then deliver the result of that processing to an outbound location or subsystem. BizTalk views all data and events as “Messages”. A messageis a finite entity within the BizTalk MessageBox. Messages are immutable once they are published to the MessageBox. Messages have context properties and zero-to-many message parts. BizTalk extends the idea of TCP/IP Port to include other protocols such as file, SQL , etc. A Port is either a “receive location” (i.e. BizTalk would listen on this port for incoming messages) or a send port that BizTalk will send through messages. BizTalk groups receive locations into what it calls a “receive port” and groups send ports into send port group (I know the naming is a little bit confusing). A receive location receives messages using a component called “BizTalk Adapter”. A BizTalk Adapter is a component that understands the Transport Protocol like HTTP, FTP etc. The received message is processed by a pipeline associated with the receive location. A pipeline is a component that performs transformations on messages to prepare them to enter or leave BizTalk Server MessageBox database. In the receive locations, the main task of a receive pipeline is to convert input data into proper XML. Likewise, in send ports, the main task of a send pipeline is to convert input data into the proper data format for the transport protocol (Adapter).