Contents
Once, just half a day before a proof-of-concept with a major client, Marty was handed a complex BizTalk® solution to fix. The main component was an orchestration that integrated several back-end systems, making multiple calls out to each. The Orchestration Designer screen he was staring at consisted of mostly black lines connecting dozens of Receive and Send shapes to over 40 inbound and outbound ports—a nearly impossible design to debug. His solution: start over using Multi-Part Message Types and Direct-Bound Ports (Tips #1 and #2 below), and perform unit testing along the way. And guess what? He made it!
Marty’s success can be at least partially attributed to the design of BizTalk Server, which is built to meet the special challenges inherent in programming connected systems without, in some cases, even having to write code. But don’t let the pretty drag-and-drop flowcharts or the apparent lack of code fool you. BizTalk is a broad and powerful product that takes years to master. Along life’s path to becoming a BizTalk guru, you’ll encounter many situations that call for writing code based on the Microsoft® .NET Framework to supplement the sophisticated built-in message processing in BizTalk.
Until recently, there have been few opportunities for BizTalk newbies to accelerate their learning with secrets from the masters. (See the sidebar "Learning BizTalk Server 2006 Programming" for a few resources.) I got my first intensive introduction to BizTalk while attending one of Marty’s week-long BizTalk bootcamp classes, during which he continuously exhorted us to design systems properly and to test them under realistic load conditions—with PerfMon—before calling it a day. From those talks, we extracted a bucket of tips that can help you become a more effective developer. The purpose of this article is to share those useful tips with you, so let’s get started.
1. Always Use Multi-Part Message Types
BizTalk is all about messages, which, by the way, are not only for e-mail. Documents, InfoPath forms, large binary files, SQL records, flat files, and anything XML can be processed as messages, and you get the benefit of asynchronous communication. Once you get used to it, message-oriented programming is as cool and useful a paradigm as object-oriented programming.
Messages in BizTalk are data, and each message must be of a selected message type. The most common message type in BizTalk is a schema, which is to say the message is based on an .XSD file that specifies the record and field structure in the message. Please see
msdn2.microsoft.com/en-us/library/ms942182.aspx for a full discussion of schemas and maps in BizTalk.
Some day when programming in BizTalk, you’ll surely decide you want to change the schema on which a message is based. The problem is that if you have already selected that message for a Send or Receive shape and connected it to an orchestration port, you’ll get this error when you try to change the Message Type:
Property value is not valid: One or more Send or Receive actions are connected to Ports and are using this Message. Please disconnect the actions before changing the Message Type.
Before jumping in to do as the error message suggests, let’s think for a moment about what that entails. First, you’ll have to inspect every Receive and Send shape to determine if it uses a Message variable that’s associated (has its Message Type set) with the schema you want to change. Today, there is no facility in BizTalk to provide an audit of this or to generate a dependency map; it just wouldn’t be a good practice to create a single orchestration with so many Receive/Send shapes that you would need such a feature. In the horrific orchestration mentioned earlier, most of the Receive and Send shapes were using different Message variables, but all were pointing to the same schema definition (all the same Message Type)!
Second, once you find all the Receive/Send shapes, you’ll have to delete their Port connections. Third, you’ll have to change the Message variable so that the Message Type property is set to the new schema and then reassociate the Message variable with each Receive/Send shape.
Fourth (and you thought you were done), you’ll have to find all the Port Types that are associated with the Ports you disconnected from Receive/Send shapes and reset their Operation’s Message Type properties as well. You see, when you originally associate a Receive or Send shape with a Port, BizTalk automatically sets the Message Type property in the Port type to match the Message Type of the Message variable for that Receive/Send shape. When you delete the connection between the Port and Receive/Send shape, BizTalk loses the Message Type of the Port.
Fortunately, there’s a better way. An axiom in computer science says that many problems can be solved by adding a level of indirection. That’s true once again in this case; all we need to do is use a Multi-Part Message Type to wrap the underlying schema. This is slightly more work initially, but it’s much more flexible and will save you time in the long run. Here’s how to do it.
When you right-click Messages in the Orchestration View tab to create a new message, you have four choices for the Message Type property (see the lower-right corner in Figure 1). It’s common to choose a schema for the message type, but for this tip let’s try something radically different. Click the + sign to expand Multi-part Message Types and then choose Create New Multi-part Mes sage Type. Name your multi-part message type then click the + sign to expand it so you can see its MessagePart_1 member (that’s the BizTalk suggested name for the first part of your new multi-part message type). Now change the Identifier property from "MessagePart_1" to something better, like "Body" (just don’t use lower-case "body" as it’s a reserved word). Don’t forget to set the Message Body Part property to True (so it can act just like a regular message).
Figure 1 Setting the Message Type Property in Orchestration View (Click the image for a larger view)
That’s the end of the extra work. Finally, set the Type property for your new message part (such as Body) to be the same schema you would have used for a regular schema-based message type. Now your logical port and Receive/Send shape are using your multi-part message type, and you can easily change the underlying schema in the future without having to disconnect the port from the receive shape.
2. Always Try to Design Orchestrations with Direct-Bound Ports
Take a look at Figure 2, which shows the choices you have when configuring a port in Orchestration Designer. As you work through the Port Configuration Wizard, you are moving from top to bottom in this diagram. Notice the three Direct options in the diagram. Though the BizTalk UI offers "Direct" as the port type, we’ll use the vernacular here and refer to them as "Direct-Bound."
Figure 2 Port Options in the Orchestration Designer Port Configuration Wizard (Click the image for a larger view)
The key to understanding port terminology in BizTalk is to understand the notions of logical ports (also called orchestration ports) and physical ports. To oversimplify, it’s the difference between creating ports in Orchestration Designer (logical), and using BizTalk Explorer or BizTalk Administration Console (physical).
When a developer creates a Specify Later port in Orchestration Designer, he’s configuring a logical port, leaving the corresponding physical port properties to be configured later by the BizTalk administrator. Giving the administrator the flexibility to configure the physical port in the production environment is a key reason why Specify Later is the most frequently used option.
With the Specify Now option, the developer does it all, both logical and physical configuration, and the solution is hardwired. This is not recommended, except for quick experiments.
With Dynamic ports, the final configuration of the physical port properties is again done by the administrator. However, in this case the developer can also write code to set certain outbound message properties at run time. This is useful for SMTP or HTTP, where the orchestration logic includes an Expression shape with a statement like this:
DynamicSendPort(Microsoft.XLANGs.BaseTypes.Address)= “mailto:john@contoso.com”;
While there are many options in Figure 2, the figure only shows the options in Orchestration Designer. Where do Static ports fit in? That term comes up when you create a physical port in BizTalk Explorer or in BizTalk Administration Console. If the developer creates a Specify Later or Dynamic port in Orchestration Designer (a logical port), then the administrator will choose among the four options shown in Figure 3when creating the corresponding physical port in BizTalk Explorer or BizTalk Administration Console.
Figure 3 Physical and Logical Port Types
Just remember, Dynamic physical ports are for Dynamic logical ports (easy enough), but Static physical ports are for Specify Later logical ports. Also, note the terminology mismatch between Solicit-Response and Request-Response. Oh well.
With those preliminaries out of the way, we can finally talk about the advantages of Direct-Bound ports. Direct-Bound ports are great for sending a message from BizTalk to BizTalk. By the way, this is why it wouldn’t make sense for BizTalk to have a Direct option for Web ports. Web ports are for BizTalk to the Web, not BizTalk to BizTalk.
Direct-Bound ports are self-configured, without sacrificing flexibility or performance. A handy benefit of Direct-Bound ports is that neither the developer nor the administrator needs to create corresponding physical ports in BizTalk Explorer or BizTalk Administration Console. So Direct-Bound ports yield orchestrations that are more self-contained, and therefore more reusable and easier to redeploy independently.
You’ve got three Direct-Bound port types: Message Box Routing, Self-Correlating, and Orchestration-to-Orchestration (also called Partner Ports). The actual text of the three radio buttons in Orchestration Designer is much wordier, but when you have a look you’ll easily be able to map our shortened names to the UI.
Of these three port types, the Message Box variety is the one that comes up most often. It’s the one that permits your orchestration to be completely independent of other orchestrations, even though you might intend for it to communicate with itself. Ah yes, that does imply the possibility of an infinite loop. We’ll get to that in just a second.
When a Receive shape with the Activate property set to True is configured for Message Box Routing, it simply means that a new instance of the orchestration will fire up when a message arrives in the Message Box that matches the given subscription filter, regardless of where the message came from or who sent it. This could help you build a reusable orchestration, like a generic function in a utility library. Marty built an elegant message-based exception handling system using this technique, but that’s another story.
Suppose you’d like to start your orchestration in two ways: first, if the operator drops a "kickoff" message in a folder on a Static One-way FILE (physical) port; or second, if the orchestration sends itself a kickoff message to run another instance. Your first thought might be to use a Listen shape (like an OR operator for orchestrations) with two Receive shapes underneath it (a Specify Later for the FILE port, and a Direct for the Message Box). But the neat thing about Direct-bound Message Box ports is that, again, they don’t care where the message comes from. So you only need one Receive shape to start this orchestration!
Self-Correlating Direct-Bound ports are usually used when you want to send messages from one orchestration to the next, asynchronously. Say you have two orchestrations, A and B. Just go into Orchestration B, and create a logical Send Port under the Parameters section and set it to use the Port Type created in Orchestration A. Now, any messages you send from that port in Orchestration B will be immediately routed to the logical Receive port you created in Orchestration A. We’re basically leveraging the publish/subscribe nature of BizTalk to not only forward messages, but also stamp them with hidden unique correlation tokens, hence "Self-Correlating." In this case, the orchestrations do have some knowledge of each other because you’re sharing a port type between them. You can reduce the dependency somewhat by setting the Message Type of the Operation on the Port Type to Sys tem.Xml.XmlDocument. This is a great way to have orchestrations run asynchronously, while having some kind of broker be able to receive Acks or Nacks from other orchestrations.
Orchestration-to-Orchestration Direct-Bound (Partner) ports are for sending messages asynchronously. You’re adding a broader restriction to the subscription than you have with Message Box Direct-Bound ports. In this case, you’re saying that you’ll accept any message from the partner orchestration. One orchestration specifies the Direct-Bound port, and the other orchestration (the partner) specifies itself. Either the sender or the receiver can be the one that refers to the Direct-Bound port. The key here is that you’ll be wiring up two known solutions to communicate, whereas with Message Box Direct-Bound ports the sender and receiver are more loosely coupled.
Now for the fun part. A common pitfall with Direct-Bound ports, particularly the Message Box variety, is creating an infinite loop. Imagine a simple orchestration consisting of just two shapes, an Activate=True Receive shape (Direct-Bound, of course) and a Send shape that merely forwards the message to a FILE port. When this orchestration sends the message out, where does it go? As always, first to the Message Box. Whenever a message arrives in the Message Box, BizTalk searches for any subscriptions that match. And how will this message differ from the message that activated the orchestration in the first place? It won’t, so BizTalk will gladly fire off another instance of your orchestration to process it, and so on, until you run out of memory. One way to fix this is to copy the inbound message to a newly constructed outbound message and change the value of at least one promoted property so the subscription filter on your Receive shape will fail to match the new message. Run Administration Console and go to Group Hub | New Query | Open Query. The next step is to open BTSSubscription Viewer.btq in your BizTalk\SDK\Utilities folder. You’ll see that BizTalk automatically creates a predicate when you set the message type for the Receive shape. In Orchestration Designer, add a predicate to the Filter Expression property so the full subscription will look like this:
BTS.MessageType == “http://MyInternalSchemas.MyProject#MyRootNode” AND inbound_message(status) != 1
Or if you prefer not to change the schema to add a custom field like that, you can also add a clause to the subscription filter on the Receive shape to test that the BTS.Operation name is not the same as the one in your orchestration.
Remember when we said BizTalk is all about messages? Well, scratch that. It’s really all about subscriptions. Nothing happens to a message in BizTalk without a subscription on a Send port or an Orchestration.
3. Always Use Separate Internal and External Schemas
You should always transform messages you receive into your own canonical schema—regardless of how simple the schema is or who the source is.
For a small amount of processing, this buys you a lot of flexibility in case (or more appropriately, when) the sender changes the schema. You don’t want your orchestration logic to depend on any properties or fields in a schema that is controlled by someone else. If the sending party changes its schema, you only need to change your map, not your orchestration. It’s the indirection axiom again.
And by following the best practice of creating separate Visual Studio projects for maps, orchestrations, internal schemas, and external schemas, it’s easier to redeploy your updated solution. This becomes really crucial for Web services generated by the BizTalk Web Services Publishing Wizard.
If you’re lazy, you might be able to get by without inventing your own schema with new field names. Just copy the inbound schema you’ve been given and map all the fields from the source schema as is, but at least save the copy into your internal schemas project and call it your own.
This can also reduce the number of maps you need. Suppose you need to map three types of inbound messages to three types of outbound messages (perhaps not today, but maybe in the future). Applying this technique, you create a map to your canonical schema for each inbound schema and then a map from your canonical schema to each outbound schema. That’s only six maps to build and maintain, not nine.
4. Never Expose Your Internal Schemas Directly in WSDL
This means you should never click the first radio button in the BizTalk Web Services Publishing Wizard; always click the second button instead (see Figure 4). The first radio button is designed for publishing orchestrations and the second is designed for pure messaging solutions, but it’s a good idea to use the second option for orchestrations too.
Figure 4 Publish Your External Schema, Not Your Orchestration (Click the image for a larger view)
Yes, this is more complex, so we know you’re expecting a good reason why you’d want to do this. In a word, interface. Or if you prefer something more verbose, loose coupling! This buys you more freedom to change your orchestration without breaking the caller. Think of this as a specialization of the concept in Tip #3.
Let’s talk about how you can do this. Here’s one thing to remember about the wizard: when you get to Step 2, the Web Service dialog (seeFigure 5), you need to right-click on practically everything. That’s how you’ll name the Web service and its methods that you’re creating.
Figure 5 Edit Your WSDL in the Web Service Dialog
You need to compile your schemas project in BizTalk before you get to this step in the Web Services Publishing Wizard, so you’ll be able to browse to the assembly. Actually, as you’ll see below, it makes sense to go ahead and deploy your solution before you run the wizard. One more aside; When you publish an orchestration as a Web service, you must change the Type Modifier property on your orchestration port type from Internal to Public before you build and deploy. If you didn’t do it when you first created the port, it takes some real digging to find this (highlight the port type in the Types pane in Orchestration View—not by clicking on the port on the Port Surface, which displays a different set of properties). But another benefit of publishing schemas as Web services is that you won’t have to do that anymore!
OK, let’s get back to the wizard, specifically Figure 5. You’ll need to right-click the Request and Response nodes to choose the schema on which the web messages are based. Orchestrations you publish will typically have a logical Receive shape and a Send shape for communicating with the caller of the Web service. Those shapes will have associated messages, and those messages will have a schema type (a multi-part message type is not required, just recommended). So you need to tell the wizard to use your external-facing schema (remember Tip #3). Then you’ll use maps in the Receive port (inbound and outbound maps) to transform the Web service request and response schemas to the internal schemas expected by your orchestration. When the client adds a reference to your Web service, he’ll get a proxy class that includes a type for your external-facing schema. The client will use that type to create and populate a request message to send to your Web service. And he’ll snag the response from your Web service into an object that is based on the external response schema you specify in Figure 5.
You’ll also have to remember to choose the XML Pipeline in the Receive location, otherwise BizTalk won’t create the BTS Message Context properties, one of which is the Message Type of the incoming Web service message. BizTalk knows the Message Type because it parsed it via the XML Disassembler in the pipeline and promoted the property, and it uses that type to execute the map to transform the request into the type the orchestration is expecting.
What’s really important is that the operation name you select in the wizard must exactly match the name of the operation you specify in the logical port in the orchestration. Simply accepting the defaults for everything won’t work because the default operation name in Orchestration Designer is "Operation_1", while the default name in the wizard is "WebMethod1" (see Figure 5). It’s a good practice to set them to a meaningful name in verb-noun format, such as UpdateInventory.
Remember, messages, even incoming Web service request messages, get routed to an orchestration based on a subscription. When you bind a logical port to a physical port, which is a requirement with Request-Response ports published as Web services (even when publishing via schemas), BizTalk creates an automatic subscription for you. You can view the subscription from the BizTalk Administration Console by selecting "Subscriptions" as the value in the query window. The subscription is slightly different for SOAP than what’s usually created for other logical-to-physical ports that aren’t Direct-Bound. Regular subscriptions that BizTalk creates for orchestrations will look like the following:
http://schemas.microsoft.com/BizTalk/2003/ system-properties.ReceivePortID == {some Guid} AND http://schemas.microsoft.com/BizTalk/2003/system-properties.MessageType == http://schemas.someschema#Root AND http://schemas.microsoft.com/BizTalk/2003/ system-properties.InboundTransportType != SOAP OR http://schemas.microsoft.com/BizTalk/2003/ system-properties.ReceivePortID == {some Guid} AND http://schemas.microsoft.com/BizTalk/2003/soap-properties.MethodName == {Inbound Operation Name on Logical Port} You can see that the name you type into the wizard as the MethodName must match the Operation name of the logical port within the orchestration, or the mapped Web request message will never get routed to the orchestration.
When you get to the Web Service Project dialog in the wizard, enter a URL for your Web service, for example: http://localhost/MyBizTalkWebService. Also, you may as well let BizTalk create the receive locations in your application for you. Since you already deployed your application, you can choose it here from the dropdown list. For more details about all this, be sure to see the article titled "How to Use the BizTalk Web Services Publishing Wizard to Publish Schemas as a Web service" in the BizTalk help file.
I know we said BizTalk is all about subscriptions, but that’s not our final answer. As you can tell from the last two tips, it’s really all about schemas. Every message and every map depends on a schema (or two) so mastering schemas is vital to BizTalk proficiency.
5. Always Optimize the BizTalk Registry for Web Services
If your BizTalk orchestration is published as a Web service, you’ll definitely want to tweak some of the default ASP.NET parameters used by BizTalk. Actually, these parameters affect any BizTalk artifact that uses the CLR including XLANG. BizTalk automatically multiplies these recommended values by the number of CPUs you have. Here are the steps:
1. Before modifying the Registry, back it up.
2. Create a file in Notepad with the following code.
Windows Registry Editor version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$BTSHOST\CLR Hosting] “MaxIOThreads”=dword:00000064 “MaxWorkerThreads”=dword:00000064 “MinIOThreads”=dword:00000019 “MinWorkerThreads”=dword:00000019
By creating the file in this manner, you can easily reuse it on other machines running BizTalk. The DWORD values are in hexadecimal (Registry scripts do not use the standard 0x prefix for hexadecimal values).
3. Replace the string "BTSHOST" with the name of the BizTalk host for your machine. You can run RegEdit first to examine the actual key name on your box.
4. If you save the file with a suffix of .REG, you can double-click it to run it on your BizTalk installation.
5. Run RegEdit and navigate to the CLR Hosting key to verify that the values have been added correctly.
See
msdn2.microsoft.com/en-us/library/aa561380.aspx for more details about this and other optimizations and
msdn2.microsoft.com/en-us/library/aa475435.aspx for an excellent article about performance tuning for low latency messaging.
6. Always Set the Assembly Key File with a Relative Path
This tip will help you check out the solution from version control and build it on a different box. Some developers on your team may arrange their environment with different drive letters, or you may change yours in the future. This is really simple, but you may be wondering what’s going on with all those dots.
The recommendation is for the first developer on the solution to create a strong name key file in the same folder with the Visual Studio solution file (.sln). Then create subfolders of that solution folder to hold each Visual Studio project by artifact type, such as maps, pipelines, orchestrations, internal schemas, external schemas, and any .NET components you’re compiling. By the way, despite how many times you may have heard that advice, you don’t have to think of it as a strict rule. If you know certain artifacts are likely to be revised together in the future (such as a map and its schemas) then you should put them in the same project to reduce the number of components you’ll have to redeploy. During development we can share the same key file among the projects within the solution and among developers. (Creating secure key files for production is another matter.) Just check the key file into version control along with the other artifacts in your solution. Now indicate the path to the key file for each assembly in the solution. The way to do that for BizTalk projects is "..\..\..\Key.snk", as shown inFigure 6. This works because BizTalk builds your binaries in a subfolder called something like [YourSolution]\[YourProject]\bin\Development. So three parent directory hops gets BizTalk back to your solution folder with the key file.
Figure 6 Configuring the Assembly Key File for a BizTalk Project (Click the image for a larger view)
Note that if you add .NET Framework Class Library projects to your solution, they’re compiled with a slightly different directory structure than BizTalk projects, and the Visual Studio UI to set the key file path doesn’t really allow you to enter a relative path. One solution is to edit the .csproj (for C#) or .vbproj (for Visual Basic®) as a text file. Search for the name of your key file that you browsed to in the Visual Studio project properties UI, and replace it with a relative path that goes up only one parent, like so: ..\Key.snk. Don’t forget to add your .NET Framework assembly to the GAC after each build. You can do it automatically with a post-build step in Visual Studio. It just requires full paths and quotes around strings with spaces, like this monster (all on one line):
“C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin\gacutil.exe” /i “$(TargetPath)” /F
7. Never Overlook Free Sample Code
Here’s a quick tip worth an ounce of gold. The BizTalk help file documents over 50 sample applications and scripts (installed in the SDK\Samples folder) that you can learn from, adapt, and reuse. You’ll also find over 30 useful BizTalk applications on this page:
msdn2.microsoft.com/en-us/biztalk/aa937647.aspx. Finally, the Bloggers Guide to BizTalk (available on
GotDotNet.com) is a compendium of more than 400 technical articles, including many useful code snippets and samples. I keep a shortcut to the Bloggers Guide on my desktop and it’s one of the first places I look when venturing into new BizTalk territory.
OK, we admit it, if you already knew this, then you might think this is a lame tip. But considering that it is otherwise difficult to find professional BizTalk sample code, we thought it worth reminding that this cache is always right at your fingertips. The thing is, these samples aren’t just useful when you’re getting started with BizTalk, they’re useful references again and again.
By the way, don’t forget to check back on MSDN for the BizTalk help file that’s updated periodically. You can download it in PDF or zipped .chm format, or browse it online.
8. Debug XSLT in Visual Studio
This could be the least well-known feature in Visual Studio, and it doesn’t require BizTalk to be installed. The XSLT debugger allows you to see the results of a transform while it’s running, set breakpoints, and examine local variables and the call stack. You can even step into your XSLT script from C# or Visual Basic.
Figure 7 shows a running example of a transform. I just created a blank solution and loaded this tiny XML data file to be transformed into HTML:
Figure 7 The XSLT Debugger in Visual Studio (Click the image for a larger view)
Then I entered the XSLT file you see in the left pane in Figure 7. It will create a Hello World document, and if the node element has an attr attribute equal to “debug”, then it will spit that out too.
Now with your cursor in the XSLT file (that’s important), set the Input property to be the path of the XML data file you’re going to transform. Be aware that right-clicking the XSLT filename itself in the Solution Explorer window will display a different set of properties, the File properties, not the Document properties. I wish that were more intuitive, but once you learn it, you’ve got it. And if there’s one thing that separates pros from average blokes, it’s the desire to explore. You can set the Output property too, or enjoy the show as the transformation come to life in the middle pane.
When the active document in Visual Studio is an XSLT file, the dropdown XML menu (on the main menubar) includes a Debug XSLT command. Just click that and you can step through your transformation code, even stepping into subroutines. Figure 7 is showing the value of @attr in the Watch window as the program has just performed the xsl:if test. As you can see from the output in the middle window, the test evaluated true, and the
node was output.
