Jul 022014
 

So in discussions with a lot of executives about the situations of integration, often times the question is asked: Why use BizTalk, it seems like a lot of work to get integration done when I could just use tools that Microsoft provides already.

I have struggled with coming up with a good answer to this question, because, yes Microsoft provides other integration tools packaged with other server products that solve the same problems that BizTalk Server solves.

Lets take SQL Server Integration Services (SSIS). It transforms data from one data type to another. You don’t even need SQL Server to be the source or destination. From outward appearances it can do all of the things that BizTalk can.

SSIS is great, but SSIS is akin to a machete, whereas BizTalk is akin to a Swiss army knife

Machete vs.swiss-army-knife 

 

They both have their uses, if I need to cut down a swath of weeds, or to clear a trail of underbrush, a machete is what I would use. If I needed to whittle away a piece of wood, I would use the Swiss army knife. Could I accomplish the same thing with the other tool? YES! Clearing under brush with a Swiss army knife, possible, but not the best, carving a wooden sculpture with a machete, I guess it can be done.

Can a Swiss army knife deal with a screw? Yes! Can I cut paper, can I open up a can, can I file my fingernails? All yes! Is it the best tool for the job? Probably not, but it is far more comprehensive than a lot of other tools.

So also is SSIS compared to BizTalk. If I wanted to do a mass update, without a lot of moving part SSIS is great, if I need to bulk move data from one place to another, SSIS is the job. If I need to design a workflow process, where there are multiple stops (along with different types of end points), BizTalk is the way to go.

Are there better screw drivers than the one provided with the Swiss army knife, how about can openers, how about scissors, yes, yes, and yes.

WCF exposed C# interfaces are much faster, and operate at a much granular level. However, you lose some of the functionality that comes out of the box with BizTalk, namely tracking, exception handling, etc,

Food for thought.

 

So I am on a project that takes existing BTS 2004 and converting it to BTS 2013.

They have a UNIX ftp server that has processes can’t lock files while being written. This causes a little challenge with the FTP adapter. The process writes the file, and then writes a trigger file.

Example:

Payload file: eric.samplefile1.txt

Trigger file: eric.samplefile1.txt.trg

What they did back in 2004, is create the standard ftp adapter that would look for the *.trg, and then the pipeline would re-connect and swap the payload of the (nearly) empty trigger file and replace the data with the actual payload of the file.

This is a ‘tricky’ way, but one I would never champion: it is getting the payload via .net code.

Now we are connecting via sFTP, and there is no publically available sFTP code to ‘backdoor’ connect the sFTP server. I needed to find a different way.

What I did was

  • create a trigger receive location, and the pipeline
  • which uses ExplorerOM to create a non primary receive location based on the trigger file name being picked up
  • The payload receive location knows it is not a trigger file based on the receive location name in combination of the filename extension
  • When the pipeline (which is the same one as the trigger file) gets the payload, it goes and deletes itself.

This allows for multiple files to be processed, all ports are always on, and it ‘self’ cleanses.

Here is what the receive location looks like:

image

When the trigger is picked up eric.samplefile1.txt.trg, the pipeline creates a new receive location with the same pipeline component

image

When the file payload file is picked up eric.samplefile1.txt, the pipeline runs and deletes itself, leaving the receive location looking like this…

image

 

Here is the pipeline code that accomplishes this…

using System; using System.Xml; using System.ComponentModel; using System.Collections; using Microsoft.BizTalk.Message.Interop; using Microsoft.BizTalk.Component.Interop; using Microsoft.Win32; using Microsoft.BizTalk.ExplorerOM; using System.Xml.Linq; using System.Web; using System.Diagnostics; using System.IO; using System.Threading; using System.Linq; using SSOUtility; namespace StottCreations.PipelineComponents.Trigger { [ComponentCategory(CategoryTypes.CATID_PipelineComponent)] [ComponentCategory(CategoryTypes.CATID_Decoder)] [System.Runtime.InteropServices.Guid("7F3DF154-3267-4154-ABC4-E163D0B79E39")] public class ReceiveLocationConfiguration : Microsoft.BizTalk.Component.Interop.IBaseComponent, Microsoft.BizTalk.Component.Interop.IComponent, Microsoft.BizTalk.Component.Interop.IPersistPropertyBag, Microsoft.BizTalk.Component.Interop.IComponentUI { #region Variables string mgmtDb, port, name, realFileName, transportDefinition, ftpServer, ftpFolderPath, originalFileName; XDocument doc = new XDocument(); #endregion #region PipelineProperties private string ssoAppName = null; public string SSOAppName { get { return ssoAppName; } set { ssoAppName = value; } } #endregion #region IBaseComponent [Browsable(false)] public string Name { get { return "Receive Location Configuration Component"; } } [Browsable(false)] public string Version { get { return "1.0"; } } [Browsable(false)] public string Description { get { return "Modifies an available receive location for the SFTP adapter based on the trigger file"; } } #endregion #region IComponent public IBaseMessage Execute(IPipelineContext pc, IBaseMessage inmsg) { getConnectionString(); getPortName(inmsg); getTransportDefinition(inmsg); getLocationName(inmsg); getServer(); getPath(); if (isTrigger(Convert.ToString(inmsg.Context.Read("ReceiveLocationName", "http://schemas.microsoft.com/BizTalk/2003/system-properties")))) { createLocation(); inmsg = null; } else { destroyLocation(); } return inmsg; } #endregion #region IPersistPropertyBag public void GetClassID(out Guid classid) { classid = new System.Guid("7F3DF154-3267-4154-ABC4-E163D0B79E39"); } public void InitNew() { } public void Load(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, Int32 errlog) { string val = (string)ReadPropertyBag(pb, "SSOAppName"); if (val != null) ssoAppName = val; } public void Save(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, Boolean fClearDirty, Boolean fSaveAllProperties) { object val = (object)ssoAppName; WritePropertyBag(pb, "SSOAppName", val); } private static object ReadPropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, string propName) { object val = null; try { pb.Read(propName, out val, 0); } catch (System.ArgumentException) { return val; } catch (Exception ex) { throw new ApplicationException(ex.Message); } return val; } private static void WritePropertyBag(Microsoft.BizTalk.Component.Interop.IPropertyBag pb, string propName, object val) { try { pb.Write(propName, ref val); } catch (Exception ex) { throw new ApplicationException(ex.Message); } } #endregion #region IComponentUI [Browsable(false)] public IntPtr Icon { get { return IntPtr.Zero; } } public IEnumerator Validate(object projectSystem) { if (projectSystem == null) throw new System.ArgumentNullException("No project system"); IEnumerator enumerator = null; ArrayList strList = new ArrayList(); try { } catch (Exception e) { strList.Add(e.Message); enumerator = strList.GetEnumerator(); } return enumerator; } #endregion #region Helper private bool isTrigger(string locationName) { bool returnValue = false; BtsCatalogExplorer root = new BtsCatalogExplorer(); try { root.ConnectionString = mgmtDb; ReceivePort receivePort = root.ReceivePorts[port]; returnValue = (receivePort.PrimaryReceiveLocation.Name == locationName) ? true : false; } catch (Exception e) { root.DiscardChanges(); throw e; } return returnValue; } private void createLocation() { string password = SSOClientHelper.Read(SSOAppName, "FTP_Password"); int iteration = 0; while (true) { BtsCatalogExplorer root = new BtsCatalogExplorer(); try { root.ConnectionString = mgmtDb; ReceivePort receivePort = root.ReceivePorts[port]; int nextPort = receivePort.ReceiveLocations.Count; receivePort.AddNewReceiveLocation(); XmlDocument transportData = new XmlDocument(); transportData.LoadXml(HttpUtility.HtmlDecode(receivePort.PrimaryReceiveLocation.TransportTypeData.Replace("<Password vt=\"1\" />", "<Password vt=\"8\"></Password>"))); transportData.SelectSingleNode("//FileMask").InnerText = realFileName; transportData.SelectSingleNode("//Password").InnerText = password; receivePort.ReceiveLocations[nextPort].Name = name; receivePort.ReceiveLocations[nextPort].Address = String.Format("sftp://{0}:22{1}/{2}", ftpServer, ftpFolderPath, realFileName); receivePort.ReceiveLocations[nextPort].TransportTypeData = transportData.OuterXml; receivePort.ReceiveLocations[nextPort].TransportType = receivePort.PrimaryReceiveLocation.TransportType; receivePort.ReceiveLocations[nextPort].ReceivePipeline = receivePort.PrimaryReceiveLocation.ReceivePipeline; receivePort.ReceiveLocations[nextPort].ReceivePipelineData = receivePort.PrimaryReceiveLocation.ReceivePipelineData; receivePort.ReceiveLocations[nextPort].ReceiveHandler = receivePort.PrimaryReceiveLocation.ReceiveHandler; receivePort.ReceiveLocations[nextPort].Enable = true; root.SaveChanges(); break; } catch (Exception e) { iteration++; root.DiscardChanges(); if (iteration < 30) { Random rd = new Random(); Thread.Sleep(rd.Next(100, 1000)); } else { throw e; } } } } private void destroyLocation() { BtsCatalogExplorer root = new BtsCatalogExplorer(); try { root.ConnectionString = mgmtDb; ReceivePort receivePort = root.ReceivePorts[port]; ReceiveLocation deleteableLocation = null; foreach (ReceiveLocation location in receivePort.ReceiveLocations) { if (Path.GetFileName(location.Address).EndsWith(originalFileName)) { deleteableLocation = location; break; } } receivePort.RemoveReceiveLocation(deleteableLocation); root.SaveChanges(); } catch (Exception e) { root.DiscardChanges(); throw e; } } private void getConnectionString() { string regEntry = @"HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\BizTalk Server\3.0\Administration"; string server = Registry.GetValue(regEntry, "MgmtDBServer", "BTSServer").ToString(); string database = Registry.GetValue(regEntry, "MgmtDBName", "BizTalkMgmtDb").ToString(); mgmtDb = String.Format("Server={0};Initial Catalog={1};Integrated Security=SSPI;", server, database); } private void getPortName(IBaseMessage inmsg) { port = (System.String)inmsg.Context.Read("ReceivePortName", "http://schemas.microsoft.com/BizTalk/2003/system-properties"); } private void getLocationName(IBaseMessage inmsg) { string initialLocation = (System.String)inmsg.Context.Read("ReceiveLocationName", "http://schemas.microsoft.com/BizTalk/2003/system-properties"); string triggerFileName = (System.String)inmsg.Context.Read("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties"); originalFileName = Path.GetFileName(triggerFileName); realFileName = Path.GetFileNameWithoutExtension(triggerFileName); name = String.Format("{0} - {1}", initialLocation, realFileName); } private void getTransportDefinition(IBaseMessage inmsg) { transportDefinition = (System.String)inmsg.Context.Read("InboundTransportLocation", "http://schemas.microsoft.com/BizTalk/2003/system-properties"); } private void getServer() { // sftp://mysftpserver:22/dev//eric.*.trg ftpServer = transportDefinition.Replace("sftp://", String.Empty); string[] splitTransport = ftpServer.Split(':'); ftpServer = splitTransport[0].ToString(); } private void getPath() { // sftp://mysftpserver:22/dev//eric.*.trg char[] splitting = ":22".ToCharArray(); string[] segments = transportDefinition.Split(splitting); string[] elements = segments[segments.Length - 1].Split('/'); string fileMask = elements[elements.Length - 1]; ftpFolderPath = segments[segments.Length - 1].Replace("/" + fileMask, string.Empty); } #endregion } }

Jun 282014
 

I have heard of re-tweeting, but re-blogging?

http://blogs.msdn.com/b/biztalkhealthmonitor/archive/2014/06/26/overview-of-biztalk-health-monitor-bhm.aspx

 

So Brian called today asking what was the apparatus for when you open a new BizTalk project, where does the BizTalk Management Database and Server come from.

We splunked around finally found it:

image

Another oddity: the btproj.user file doesn’t get updated until you close the solution.

 

Okay, so I needed to have a value to set the agreement. According to the documentation it states in step 3:

If step 2 does not succeed, resolves the agreement by matching the party name in the message context properties with the DestinationPartyName property, which is set as additional agreement resolver in the Identifiers tab of agreement properties.

Here is the correct Identifiers screen that needs to be populated.

image

 

So I feel I am gifted!

Truly GIFTED (oh and humble to boot)

I am gifted at discovering ways to break BizTalk.

Last night I had to re-design a process that is run in an itinerary. Now I have gotten into the mindset that executing the transform for all of the endpoints before consuming into BizTalk is far more efficient than executing 6 maps along the path through the business process. This means that I end up with a ‘collection’ of messages that I slice off the particular message I need and send it out.

My design pattern is that I have a generic xml document that runs through the ESB and then I extract the message I need, assign it to another xml document, and send it on its way.

The code that I am using in the message assignment is something like this:

RequestMsg = xpath(OriginalCanonicalMsg,”/Request/Part/SingleReqest”);

The problem is that when I change the message I am sending out from XmlDocument to a multipart message instead, the message assignment shape does not turn red.

I deploy it and it runs and shows up in the ESB Portal stating the error that it must have an implementation of Add(System.Object) at all levels of their inheritance hierarchy for RequestMsg.

It was just working!?

The code that I needed to change was:

RequestMsg.MessagePart = xpath(OriginalCanonicalMsg,”/Request/Part/SingleReqest”);

Viola, works now…

 

So I needed to fill a value up to 24 characters. I decided to do a bit of research so I could hold my own to my brother Brian (who currently holds the title at Stott Creations as the XSL king).

I have never used xsl:call-template nor recursion.

Here is an example of how to do this:

<?xml-stylesheet type="text/xsl" href="padding.xsl"?> <!--The most important is the <xsl:call-template>--> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:output indent="yes"/> <xsl:template match="/"> <XML> <Example> <xsl:attribute name="Instance"> <xsl:text>1</xsl:text> </xsl:attribute> <Title>Padding zeros at the end of A435 into a 6 byte field</Title> <Value> <xsl:call-template name="startpadding"> <xsl:with-param name="padChar" select="'0'"/> <xsl:with-param name="padCount" select="6"/> <xsl:with-param name="actualvalue" select="'A435'" /> <xsl:with-param name="justification" select="'left'" /> </xsl:call-template> </Value> </Example> <Example> <xsl:attribute name="Instance"> <xsl:text>2</xsl:text> </xsl:attribute> <xsl:variable name="pad_tmp_1"> <xsl:call-template name="startpadding"> <xsl:with-param name="padCount" select="40"/> <xsl:with-param name ="padChar" select="'#'" /> <xsl:with-param name="actualvalue" select="'Look ma, spaces:'" /> <xsl:with-param name="justification" select="'left'" /> </xsl:call-template> </xsl:variable> <xsl:variable name="pad1" select="translate($pad_tmp_1,'#',' ')"/> <Title>Dealing with spaces with a 40 character field (left justified)</Title> <Value> <xsl:value-of select="$pad1"/> </Value> </Example> <Example> <xsl:attribute name="Instance"> <xsl:text>3</xsl:text> </xsl:attribute> <xsl:variable name="pad_tmp_2"> <xsl:call-template name="startpadding"> <xsl:with-param name="padCount" select="40"/> <xsl:with-param name ="padChar" select="'#'" /> <xsl:with-param name="actualvalue" select="'...and then there is text'" /> <xsl:with-param name="justification" select="'right'" /> </xsl:call-template> </xsl:variable> <xsl:variable name="pad2" select="translate($pad_tmp_2,'#',' ')"/> <Title>Dealing with spaces with a 40 character field (right justified)</Title> <Value> <xsl:value-of select="$pad2"/> </Value> </Example> </XML> </xsl:template> <xsl:template name="startpadding"> <xsl:param name="padChar" select="'#'"/> <xsl:param name="padCount" select="0"/> <xsl:param name="actualvalue" select="0"/> <xsl:param name="justification" select="left" /> <xsl:choose> <xsl:when test="$justification='left'"> <xsl:value-of select="$actualvalue"/> <xsl:variable name="valuelength"> <xsl:value-of select="string-length($actualvalue)"/> </xsl:variable> <xsl:if test="$valuelength &lt; $padCount"> <xsl:call-template name="pad"> <xsl:with-param name="padChar" select="$padChar" /> <xsl:with-param name="padCount" select="$padCount - $valuelength" /> </xsl:call-template> </xsl:if> </xsl:when> <xsl:otherwise> <xsl:variable name="valuelength"> <xsl:value-of select="string-length($actualvalue)"/> </xsl:variable> <xsl:if test="$valuelength &lt; $padCount"> <xsl:call-template name="pad"> <xsl:with-param name="padChar" select="$padChar" /> <xsl:with-param name="padCount" select="$padCount - $valuelength" /> </xsl:call-template> </xsl:if> <xsl:value-of select="$actualvalue"/> </xsl:otherwise> </xsl:choose> </xsl:template> <xsl:template name="pad"> <xsl:param name="padChar" select="'#'"/> <xsl:param name="padCount" select="0"/> <xsl:value-of select="$padChar"/> <xsl:if test="$padCount&gt;1"> <xsl:call-template name="pad"> <xsl:with-param name="padCount" select="number($padCount) - 1"/> <xsl:with-param name="padChar" select="$padChar"/> </xsl:call-template> </xsl:if> </xsl:template> </xsl:stylesheet>

Which returns this:

<?xml version="1.0" encoding="utf-8"?> <XML> <Example Instance="1"> <Title>Padding zeros at the end of A435 into a 6 byte field</Title> <Value>A43500</Value> </Example> <Example Instance="2"> <Title>Dealing with spaces with a 40 character field (left justified)</Title> <Value>Look ma, spaces: </Value> </Example> <Example Instance="3"> <Title>Dealing with spaces with a 40 character field (right justified)</Title> <Value> ...and then there is text</Value> </Example> </XML>

Sep 142012
 

So this is the first blog post from my phone!
I decided that I might as well find out what diagnosis pointers are (I am rewriting an inbound claim process right now also).
To keep things simple and less EDI’ish we will use the following example.
CLAIM diagnosis codes: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14
SERVICE: 2, 5, 7
SERVICE: 1, 2, 3, 12
SERVICE: 7
SERVICE: 10, 13

Depending on the claim type, you can have quite a few diagnosis’
The first one listed however is the primary diagnosis code.
Each service line can have 1 up to 4 diagnosis code pointers.
So it really is a many-to-many relationship.

Jul 032012
 

So I finally sat down and decided to figure out what all of the numbers are for the DelimiterSetSerializedData attribute in the X12InterchangeXml rood node.

<ins0:X12InterchangeXml DelimiterSetSerializedData="126:13:10:42:58:-1:-1:-1:-1" xmlns:ins0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006/InterchangeXML">

Ascii Table

So the values are for the

  • segment delimiter:126+13+10=~<CR><LF>
  • element delimiter:42=*
  • sub-element delimiter:58=:

If there aren’t any values, you simply need to put –1 in the stream, example

<ins0:X12InterchangeXml DelimiterSetSerializedData="126:-1:-1:42:58:-1:-1:-1:-1" xmlns:ins0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006/InterchangeXML">

 

So I was working with a co-worker over the past week or so creating the following process:

image

Each trading partner (including the same one who got two) receives an interchange of EDI 835 payment advices.

So he had created the map and was getting the interchange xml output out of the orchestration where the map lives. However, every time he would attempt to send it through the EDI send pipeline, it would dehydrate stating that the object reference not set to an instance of an object. I searched high and low (and by that I mean that I went to multiple pages in my Google search).

So my co-worker had taken a day off and I was going ‘supposed’ to work on it. I was working on a few other things and knew that I wouldn’t get to it. The night before he took the day off, I knew exactly what the problem was, and I ended up having to think about the fix for an entire 36 hours (torture).

Resolution: We had created a partner specific 835 schema with a target namespace of http://schema.company.com instead of http://schema.microsoft.com. We had even gone into the agreement settings and set the schema from http://schema.microsoft.com to http://schema.company.com. His map was creating (via pure xsl) the interchange message with the transaction’s namespace set to http://schema.microsoft.com. Well, when the EDI assembler received the xml version of the transactions, it went to the GAC to find the 835 schema with the target namespace of http://schema.microsoft.com (this object is referencing), however, we didn’t have the default 835 schema deployed. This meant that the EDI assembler was referencing an instance of an object. (It didn’t lie after all). Once we changed the map to create transactions with the target namespace with http://schema.company.com did it run through without fail.

Now we have a process that runs in seconds, no batching, maintains the order of claim payment advices by order of the Header record.

Update: not only does this apply to when the EDI assembler can’t find the appropriate schema, if your transformation is creating nodes in the xml document that are not found in the schema representing the EDI document, it will throw the same error. If you know that you have deployed the correct schema, then it means that you should go back and unit test your map.

© 2014 BizTalk Blog Suffusion theme by Sayontan Sinha