Eric's BizTalk 2004/2006 blog

Enhancements to the EDI Logging tool (more)

2008-09-30T03:35:35-06:00:00

This seems to be a big complaint, there are many companies that would like to have access to the envelope information in the map. There are a few VERY KLUDGY ways of getting data from the message into the map. All of them are VERY BRITTLE. The EDI logger now will create an additional message part called 'envelope' and send it to the message box.

If you are familiar with the multi part message that the BTAHL7 accelerator creates, this will be a breeze. For those of you not in the provider world, I will walk through the instructions on how to create a process to access the envelope information (you will see that there is a LOT more than just the envelope information).

To start off with, there are now two message parts to the message that shows up in the message box. The body part is the transaction itself (so you can still use the message in a send port map if necessary).

Here is a screen shot of the multi part message:

The schema that is deployed to the GAC that you can reference (C:\WINDOWS\assembly\GAC_MSIL\EDIArchiveProperties\1.0.0.0__e7dd178931a8d66e\EDIArchiveProperties.dll) in your project that will host your orchestration has this structure (click to enlarge):

I normally create at least two different projects for each solution. One that represents the schemas, and the other that represents the mapping/orchestrations. For simplicities sake, I have called the schema project Schemas, and the location that I am going to put my maps and orchestrations will be called Logic. Here is a snapshot of the Solution Explorer:

I added the reference to the EDIArchiveProperties by adding a reference and pasting the dll mentioned above:

Let's create an orchestration, here you will want to create a multi-part message, making sure you define the body segment first and then the envelope segment second (click to enlarge):

Next you will create the necessary port and message from this multi-part message. I will jump to the creation of the map. You have a message defined from this multi-part type and you are mapping it to an output. You open up the map configuration dialog window and you fill in two lines, one that represents the envelope and the other that represents the EDI transaction on the input and the output message on the destination window.

And when pressing <OK>, this is the map that is created (click to enlarge), notice that the transaction is at the bottom of the source:

This allows you to access both the envelope and context data for that transaction, giving you the ability to map everything that would need to without going through a unnecessary hoops to get data injected into the message.

For pricing on this component, please refer to our services page.

Enhancements to the EDI Logging Tool

2008-09-30T02:38:32-06:00:00

By popular demand I have made some significant changes to the EDI logger. None I don't think are more important than the rest, but some do address a REAL thorn in a lot of peoples sides. I will get to that in the next entry about the change to the message. This one deals with the pipeline itself. Below is a picture of the receive pipeline configuration:

The send pipeline configuration:

And the following table on how to configure the values:

Row	Type	Meaning
Database	string	BAM Database to store data
Server	string	BAM Server to store data
Active	boolean	Activates the logging mechanism, if not, it is simply a pass thru pipeline component
Count1	string	Either XPath or Regex.Match (string count)
Count2	string	Either XPath or Regex.Match (string count)
Count3	string	Either XPath or Regex.Match (string count)
FromAddress	string	email address representing BizTalk as source
FromName	string	Friendly Name representing FromAddress (ex: BizTalk Prod biztalkprod@company.com)
Notify	boolean	Activate flag for email notification
NotifyOnlyOnError	boolean	Only send out email if there are validation errors
SMTPHost	string	SMTP Host address
Secure	boolean	True: Do not include ISA01/02/03/04 as components to be used False: Include ISA01/02/03/04 as components to be used
SubjectLine	string	Macros and regular text to create customized email subject line
ToAddress	string	Allows multiple email addresses to be included in email notification (semicolon separates email addresses)

Error details: Unable to read the stream produced by the pipeline. Details: Cannot find definition for the input: {Record, Element, or Attribute}

2008-09-27T00:27:45-06:00:00

I have come across this a lot and finally was sick of it enough to write about it.

The situation is I have an input that has a bunch of optional fields, and the output is normally a fixed length record file. In my case it is normally an EDI file of some kind that I am mapping to a flat file layout. When I start testing the map, I get the following error produced by the flat file assembler:

Error details: Unable to read the stream produced by the pipeline. Details: Cannot find definition for the input: Address

The reason is that there was no Address in the source data, but the output required it to be there to conform to the flat file layout. If you look at this page from msdn on how to force the creation of elements that are optional. Yes, this is possible, but in my case, I would be creating a script for every element I am creating, which would amount to hundreds of scripting functiods.

Underlying logic: The mapper when dragging optional source items to a required destination item creates the following logic in XSLT:

  <xsl:template match="/s0:File">
    <ns0:File>
      <xsl:if test="@Name">
        <Name>
          <xsl:value-of select="@Name" />
        </Name>
      </xsl:if>
      <xsl:if test="@Address">
        <Address>
          <xsl:value-of select="@Address" />
        </Address>
      </xsl:if>
      <xsl:if test="@City">
        <City>
          <xsl:value-of select="@City" />
        </City>
      </xsl:if>
      <xsl:if test="@State">
        <State>
          <xsl:value-of select="@State" />
        </State>
      </xsl:if>
      <xsl:if test="@Zip">
        <Zip>
          <xsl:value-of select="@Zip" />
        </Zip>
      </xsl:if>
      <xsl:value-of select="./text()" />
    </ns0:File>
  </xsl:template>

The easy solution, create a second map that maps the destination input to the destination output (and easy way to do this is by following these autolink directions). This creates the following XSLT:

  <xsl:template match="/ns0:File">
    <ns0:File>
      <Name>
        <xsl:value-of select="Name/text()" />
      </Name>
      <Address>
        <xsl:value-of select="Address/text()" />
      </Address>
      <City>
        <xsl:value-of select="City/text()" />
      </City>
      <State>
        <xsl:value-of select="State/text()" />
      </State>
      <Zip>
        <xsl:value-of select="Zip/text()" />
      </Zip>
      <xsl:value-of select="./text()" />
    </ns0:File>
  </xsl:template>

Notice that there are not any if tests.

You can chain the maps together in an orchestration, or you can place the map in the receive port and the straight thru map in the send port.

What would be nice is to have a map setting called Force Optional Creation Mapping set default to No to leave the behavior as currently is, but if set to Yes, the underlying logic would remove the xsl:if from the map an force the creation of the destination elements regardless of the presence of the input.

EDI Logger Emailing functionality and Subject Line Macros

2008-09-24T15:49:03-06:00:00

To give the ability to customize the subject line in the new email component of the EDI logger, you can use following macros:

Macro	Description
%ISA01%	ISA01 Value
%ISA02%	ISA02 Value
%ISA03%	ISA03 Value
%ISA04%	ISA04 Value
%ISA05%	ISA05 Value
%ISA06%	ISA06 Value
%ISA07%	ISA07 Value
%ISA08%	ISA08 Value
%ISA09%	ISA09 Value
%ISA10%	ISA10 Value
%ISA11%	ISA11 Value
%ISA12%	ISA12 Value
%ISA13%	ISA13 Value
%ISA14%	ISA14 Value
%ISA15%	ISA15 Value
%GS01%	GS01 Value
%GS02%	GS02 Value
%GS03%	GS03 Value
%GS04%	GS04 Value
%GS05%	GS05 Value
%GS06%	GS06 Value
%GS07%	GS07 Value
%GS08%	GS08 Value
Other Values	Description
%Adapter Type%	Adapter Type of receive transmission
%Received Filename%	Received ftp or file name
%AdaptercReceive Completed Time%	When file was picked up
%Port Name%	Port Name of receive location
%Inbound Transport Type%	Inbound Transport Type
%Receive Location Name%	Receive Location Name
%Message Type%	Message Type (http://namspace#rootelement)
%ISA Segment%	Entire ISA Segment (beware that private information might be displayed)
%GS Segment%	Entire GS segment
%Party Name%	Party Name
%Transaction%	Transaction
%Transmission ID%	Tranmission ID (represents entire transmission)
%Error Flag	'Yes' or 'No' - if there were validation errors or not
%Count 1 Query%	Entire text contained in the Count 1 pipeline field
%Count 1%	Actual value of query 1
%Count 2 Query%	Entire text contained in the Count 2 pipeline field
%Count 2%	Actual value of query 2
%Count 3 Query%	Entire text contained in the Count 3 pipeline field
%Count 3%	Actual value of query 3
%Error Description%	Entire error description
%FileGUID%	Unique ID representing this transaction

Which will create an email that looks like this:

Upgrading BizTalk 2006 to BizTalk 2006 R2 - New EDI components not available

2008-09-23T10:00:48-06:00:00

An issue that I recently came across was when we upgraded our servers from BizTalk 2006 to BizTalk 2006 R2 is that we were unable to see the new EDI features. This is the context menu I saw when right clicking the parties:

Which is incorrect, I should be seeing this context menu:

The resolution is that after you upgrade the server, you need to need to modify the installation in the add/remove programs and choose Modify:

Check the BizTalk EDI/AS2 Runtime:

Afterwards, you need to go into the BizTalk Configuration and configure the new EDI components (I already did this):

Then you get all of the components you are looking for.

Extracting data from a sub loop - two ways to accomplish the task

2008-09-12T09:22:15-06:00:00

I have come across this issue eleventeen million times, so I figure I would write about it (finally).

The issue: I want to extract data from a repeating record, but I need the output records based on something that is higher up the food chain. If I simply use the logical equals and value mapping or value mapping (flattening) funtiods, if the qualifier is the first occurrence it will successfully extract the data, otherwise it will not pull the data out.

Here is a screen shot of an 837 map that shows the issue in the 2000 REF segment:

For simplicity's sake, I have created a smaller map:

Here is the input:

<ns0:Root xmlns:ns0="http://Schemas.Input">
  <Record>
      <SubRecord Qualifier="AA">
          <Data>Here is AA Data</Data>
      </SubRecord>
      <SubRecord Qualifier="ZZ">
          <Data>Here is ZZ Data</Data>
      </SubRecord>
  </Record>
</ns0:Root>

I want to extract the /Root/Record/SubRecord/Data where the Qualifier is ZZ, but I want to create it driven on the /Root/Record

By using the out of the box functiods, I get the following output:

<ns0:Root xmlns:ns0="http://Schemas.Output">
  <Record>
  </Record>
</ns0:Root>

Which is not what I wanted.

There are two ways to extract the data. The first is to use three functiods, a string concatenate functiod to a cumulative concatenate functiod, to a script functiod like this:

The string concatenate functiod's arguments are the following:

The cumulative concatenate functiod:

And the arguments to the script functiod:

And the C# script inside the script functiod:

public static string extractFromInnerLoop(string qualifier, string segmentTerminator, string data)
        {
            string actualdata="";
            int posOfDelimiter = qualifier.Length;
            if(posOfDelimiter>data.Length) goto end;
            string delimiterValue = data.Substring(posOfDelimiter, 1);
            string fullQualifier = qualifier + delimiterValue; 
            int locOfFullQualifier = data.IndexOf(fullQualifier);
            if (locOfFullQualifier < 0)
            {
                actualdata = "";
                goto end;
            }
            int posOfSegment = data.IndexOf(fullQualifier);
            int segmentEnd = data.IndexOf(segmentTerminator, posOfSegment);
            int elementLength = segmentEnd-posOfSegment-fullQualifier.Length;
            actualdata = data.Substring(posOfSegment+fullQualifier.Length,elementLength);
            end:
            return actualdata;
        }

It works, but I thought that there must be a better approach. A second approach is to use Inline XSLT Call Template:

The arguments to the script:

and the Inline XSLT Call Template script:

<xsl:template name="SubLoopExtract">
 <xsl:param name="qual" />
 <xsl:element name="Data">
  <xsl:value-of select="SubRecord[@Qualifier = $qual]/Data/text()" />
 </xsl:element>
</xsl:template>

Both of them create the following output:

<ns0:Root xmlns:ns0="http://Schemas.Output">
  <Record>
     <Data>Here is ZZ Data</Data> 
  </Record>
</ns0:Root>

Add Gacutil to the Right Click Menu

2008-09-10T16:40:59-06:00:00

I've used this quite a bit on BTS 2004. Instead of deploying projects through Visual Studio, sometimes I want to just overwrite the DLL in the GAC, restart BizTalk, and start using it.

Here's one of the easier ways to do this:

Create a .reg file with the following text.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\dllfile\shell\gacutil\command]

@="c:\\WINDOWS\\Microsoft.NET\\Framework\\v1.1.4322\\gacutil.exe /i \"%1\" "

Double click the .reg file and right click on the strongly named .DLL to see the new menu item.

Split into transactions, suspend interchange on error: KB956051

2008-08-29T10:27:35-06:00:00

For those of you who have trading partners that deal with trading partners that send EDI in batch mode and can't fix/resend edi transactions individually, there was really no way to deal with this in the current R2 process, as there was no schema that could be created to consume the XML message that was created when you choose the Preserve Interchange - suspend Interchange on Error

The issue is that if the interchange was valid nothing but the EDI send pipeline can consume it:

So Microsoft has created a new processing option Preserve Interchange - suspend Transaction Sets on Error

Which can be downloaded from KB956051

Preserve Order while mapping

2008-08-28T14:07:12-06:00:00

There is the issue of the behavior of the mapping process creates invalid XML.

The input instance looks like this:

<xml>
 <loopA />
 <loopB />
 <loopA />
 <loopB />
</xml>

However, when using the mapper, you create your output and it ends up looking like this:

<xml>
 <loopA />
 <loopA />
 <loopB />
 <loopB />
</xml>

How to get this to work (only in R2) is to open up the btm file and change the following attribute from its default value of No to Yes in the mapsource element

PreserveSequenceOrder="No"

<mapsource Name="BizTalk Map" BizTalkServerMapperTool_Version="2.0" Version="2" XRange="100" YRange="420" OmitXmlDeclaration="Yes" TreatElementsAsRecords="No" OptimizeValueMapping="No" GenerateDefaultFixedNodes="Yes" PreserveSequenceOrder="No" CopyPIs="No" method="xml" xmlVersion="1.0" IgnoreNamespacesForLinks="Yes">

Here is the new code:

<mapsource Name="BizTalk Map" BizTalkServerMapperTool_Version="2.0" Version="2" XRange="100" YRange="420" OmitXmlDeclaration="Yes" TreatElementsAsRecords="No" OptimizeValueMapping="No" GenerateDefaultFixedNodes="Yes" PreserveSequenceOrder="Yes" CopyPIs="No" method="xml" xmlVersion="1.0" IgnoreNamespacesForLinks="Yes">

MSHUG HL7 Based Data Warehousing

2008-08-26T17:10:17-06:00:00

Eric Washburn CTO Athena Advanced Technologies

Very interesting method to warehouse all data components to the warehouse. There is a separate table that defines all of the complex data types, but each segment has its own table.

There is a unique key in the segment to represent the complex data type that then ties the segment to the complex type.

I would write more about it, but I was too interested in seeing how everything was done.

MSHUG - HealthVault Architecture Overview

2008-08-26T15:58:55-06:00:00

Sean Nolan - Chief Architect, Microsoft

Health Vault is like PayPal, they go to sites that use Health Vault to communicate data.

Why should you integrate with HealthVault:

Private and Secure Storage

Consistently log in using secure login

Application Interoperability - the ability to collect various data into one central repository

Device Connectivity - The ability for new devices to communicate directly to the health vault

Application and Device Discovery

Privacy and Security Focused - the customer is in control, the customer is in control of their own privacy

Industry Standards, it is an open platform, free and published SDK and APIs, (found on the HealthVault site) Community Promise, Easily Extensible Data Model, Strong Developer Community: MSDN Documentation

Modes:

Native: HealthVault is an online database

Copy: HealthVault is a synchronization resource to (imporpe cloyt/export/merge)

Per-type globalized transformation

html
tabular view
Standards and Device Exchange
Custom (custom xsl) on the Healthvault server 'dime'

Custodianship - allows to permissions to be granted to people (children for example) that can be granted on a 'sliding' scale, eventually, you can give it up when they move away.

Online and Offline processing mode, grants access instantly, and only for that session, or a longer term if necessary

MSHUG - Edifects using BizTalk Solution for end-to-end HIPAA transaction processing

2008-08-26T12:21:44-06:00:00

Herb Larsen Sr. Vice President of Alliances, Edifects Inc.

Not a lot of payors or providers have found that that ROI has been reached by implementing HIPAA.

A lot of the reasons is because the tasks of moving the edits from the back end system to the front end (BizTalk) is difficult.

Ramp Manager product - allows to test without being on the test without having to be on the phone with the other party and asking them to 'submit' the file and read back the errors.

Edifects XEngine - built on BizTalk Server 2006 R2

Because MS is not considered a covered entity, MS has not supplied a lot of HIPAA security standard documentation.

MSHUG Opening Keynote

2008-08-26T11:33:28-06:00:00

The next conference will be on April 4th 2009 in Chicago.

Steve Aylward - General Manager Health and Life Sciences Microsoft

Watched the future of Microsoft in healthcare video.

Microsoft is making a huge investment in healthcare.

It took 4 releases (10 years) to outsell MSDOS

It took 9 releases (11 years) to become most popular word processor

It took 5 releases (10 years) to become the leading spreadsheet

$8 billion in R&D, a large (no numbers as far as percentage directed directly at Healthcare, but a large portion)

In 2000, MS had 63 employees dedicated to Healthcare.

Amalga - Health Information System - not available in the US. Designed for the emerging markets. (used to be called Global Care)

Amalga - Unified Intelligence System - not limited to caregivers, but can be customized to end users

HealthVault - Internet health platform that enables third parties to create applications to stored personalized health care information

HTTP Post adapter

2008-08-26T00:54:01-06:00:00

I wrote my first two way adapter the other day. I have created it to communicate with Zirmed to send insurance verification request/responses via a HTTP Post.

My question, however, is how often are people using this method of transport, and would anyone else be interested in using this?

The REAL dilemma however is that there are specific items that are necessary to communicate with Zirmed, but any other post is going to look drastically different. How best to approach an enterprise solution though, do you walk thru a wizard, possibly pointing to the web page post page to 'interrogate' what needs to be passed?

Let me know if there is anyone out there interested...

BizTalk Message to System.String in BizTalk Server 2006

2008-08-22T18:10:05-06:00:00

I've needed to cast my BizTalk message as a System.String in a couple of occasions so I'm going to put it here to remember how to do it.
1. Create a Variable of type System.Xml.XmlDocument (I'm calling it xml_Temp here)
2. Create a Variable of Type string (I'm calling it str_MessageAsString here)
3. In the expression shape insert the following code:

//Instantiate your xml_Temp System.Xml.XmlDocument
xml_Temp = new System.Xml.XmlDocument();
//Assign the message you want as a string to your new xml_Temp System.Xml.XmlDocument
xml_Temp = msg_YourOriginalMessage;
//Assign the .OuterXml property of your xml_Temp to your str_MessageAsString System.String
str_MessageAsString = xml_Temp.OuterXml;

Module window not available

2008-08-21T18:46:54-06:00:00

If you are debugging pipelines or adapters in BizTalk, you can't just 'debug' the said component in Visual Studio (however, there are rumors of that changing in v.Next), you have to deploy and run it in the service.

You need to attach to the BTSNTSvc.exe and if you are lucky, all of the pdb files are in the correct place, and you can instantly start debugging.

However, if you are like me, and the dlls are in separate locations from the pdb files (GAC for example), you need to associate the dll with the pdb. This can only be done via the Module window.

According to the Module Window definition on MSDN you should "On the Debug menu, choose Windows, and then click Modules."

However, this is what I see:

The easiest you can bring it up is by pressing Ctrl - Alt - U

Event Log Monitor

2008-08-20T15:49:12-06:00:00

I've created a nice little tool I'm using more and more. As a BizTalk Administrator, I'm constantly needing to ensure nothing is going wrong on my servers. I need an alert process that could email me if/when there is a problem. I've created an event log monitoring tool that will email me depending on the search criteria I specify. I've included a feature that allows me to create multiple emails depending on the exact error. I use this quite a lot if a specific user needs to only know about their particular error.

The program is a single executable.

It is executed via the command prompt and takes the following parameters, which can be in any order.

-TOEMAIL: = ToEmail (Required) Can include multiple To Email addresses.
-FROMEMAIL: = FromEmail (Required)
-SMTPSERVER: = SmtpServer (Required)
-NUMBEROFDAYS: = Number of Days to look back (Optional) if not set to today's data
-NUMBEROFHOURS: = Number of Hours to look back (Optional) if not set will use -NUMBEROFDAYS
/n = Send Email if there are no events (Optional)
/e = Send Errors (Optional)
/w = Send Warnings (Optional)
/i = Send Information (Optional)
-EVENTSOURCE: = Event Source (Optional) Can include none or multiple Event Sources to include in the search
-EVENTCATEGORY: = Event Category (Optional) Can include none or multiple Event Categories to include in the search
-EVENTMESSAGESEARCHSTRING: = Event Message Search String (Optional) Can include none or multiple Event Message Strings to include in the search

I usually set up a batch file that is run on the Windows Scheduler and sends me an email every night for the previous day's information.

For example, I have the following event log:

I would like to receive an email for all the Events of Type Errors in the "BizTalk Server 2006" Category over the last 5 days. I would execute the following command at the command prompt:

"Generic Event Log Emailer.exe" -TOEMAIL:jstott@stottcreations.com -FROMEMAIL:MYBIZTALKERRORS@Biztalkerrors.com -SMTPSERVER:yoursmtpserver.com

-NUMBEROFDAYS:5 "-EVENTCATEGORY:BizTalk Server 2006" /e

Alternatively, I would just create a batch file with that string and schedule it to execute every 5 hours.

I then get an email every 5 hours with all the information I'm looking for.

Don't forget that you can also search using the -EVENTMESSAGESEARCHSTRING: parameter. Using this parameter, you can really narrow down your search to specific messages.

Here are my list of planned upgrades:

1. Add regular expressions to the -EVENTMESSAGESEARCHSTRING: parameter

2. Add -EXCLUDE parameters to allow you to exclude Source, Categories, or Messages instead of only including.

3. Add the other Event Logs besides the Application Event Log.

4. Create a real time monitoring tool that would push errors to you as they happen instead of pulling them at certain intervals.

If you have any other features you'd like to see in this product, please email me.

Examples of the EDI Reporting tool

2008-08-19T02:52:53-06:00:00

In the initial entry, and the follow up entry, I showed the table/view structure of the components that I built.

Here are some examples of what is available so you can see the real values for each transaction reported.

Here are the settings for the pipeline configuration: (notice I am tracking 3 different objects in the data, so I can monitor these values through the flow of the data so I have a point of reference as far as counts)

Because the table is so large, I am going to break it into chunks:

Failed to instantiate adapter

2008-08-15T11:15:23-06:00:00

I created an HTTP Post adapter and packaged it all up in a setup package. When I installed it, I opened up the management console and started to configure the adapter and I received the the following error:

Failed to instantiate adapter "{Adapter Name}".

Assembly file: "{dll source}"

Type name: "{Type}"

The problem was that the source was compiled with a dll that was on the development box, but not on the destination box. (In this case it was the Microsoft.Samples.BizTalk.Adapter.Common.dll) was not deployed to the GAC.

Once I put the assembly in the GAC, I didn't have a problem.

Retrieve the LoaderExceptions property for more information

2008-08-15T10:29:09-06:00:00

"There are no differences between BizTalk Server 2006 R2 and BizTalk Server 2006 except the enhanced EDI, WCF, and RFID functionality"

So I was creating a new HTTP Post adapter. I created it on a server that is R1.

When I copied the code to an R2 box and compiled the code and ran the Adapter Registry Wizard, I got the following error:

Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information.

The issue is that in the Transmit adapter project, it was referencing Microsoft.Samples.BizTalk.Adapter.Common that was not compiled on the R2 server.

The fix:

Simply Clean the project and Rebuild the project where it will tell you that a dll is missing.

EDI Counting functionality

2008-08-13T10:25:12-06:00:00

By popular demand, I have put the ability to count in the EDI reporting. This allows you to count up to three different objects in the data (via an XPATH statement). Here is the pipeline component:

And here is the updated view:

zombie resurrector

2008-08-12T16:07:41-06:00:00

On this BizTalk Glossary Page it states that a zombie ressurector is: An orchestration that listens at the MessageBox database for zombies and resurrects those that meet certain criteria.

The previous definition of a zombie is: A message that was not processed by an orchestration. It is not in the Suspended queue or anywhere else where it will be retried.

If there is an orchestration that is looking for 'zombies' - then they were never zombies were they?

Once you have answered that, then you can answer the question of which came first, the chicken or the egg?