Consuming and producing XML with Spotfire Streaming

Problem

Have you ever tried integrating BusinessWorks components and StreamBase?

Consider the following case. StreamBase connects to ESB that talks sophisticated XML defined as CDM (common data model). Such XML messages may have various nesting levels, multiple namespaces, some mixed content, just to name some pitfalls. 

Can StreamBase speak XML at all? Yes, all what you need are XML2Tuple and Tuple2XML operators. The only problem is that the XML used in such cases is pretty simple. It primilarily reflects the tuple schema with some additional features.

How to use complex XML with StreamBase then? XSLT is the answer. XSLT transformation can convert arbitrary XML to XML, which means we can adapt the externally given contract to the nice compact data structure in StreamBase schema. The XSLT operator can be used as preprocessor to XML2Tuple and postprocessor to Tuple2XML operators.

Let's have a look on concrete example. Assume there is transaction reporting component producing XML, like the following:

<t:transaction xmlns:t="http://demos.tibco.com/retail/transactions">
    <t:transactionId>30356110</t:transactionId>
    <t:customerId>1931233</t:customerId>
    <t:transactionTime>2012-01-13T00:00:00</t:transactionTime>
    <t:transactionLines>
        <t:transactionLine>
            <t:productSKU>420397</t:productSKU>
            <t:quantity>1</t:quantity>
            <t:nominalPrice>20</t:nominalPrice>
            <t:purchasePrice>20</t:purchasePrice>
        </t:transactionLine>
<!-- and so on -->

This message shows three challenges:

• the elements are in special namespace
• the transactionLine and transactionLines form a hierarchical structure in tuple schema with unnecessary level
• the transactionTime is in ISO-8601 format used by XML

Solution

It is possible to process it directly, but it complicates the event flow design. The proposed solution is to consume the message as StreamBase friendly XML. In order to do that use XSLT that transforms message to:

<transaction>
	<transactionId>30356110</transactionId>
	<customerId>1931233</customerId>
	<transactionDate>2012-01-13 00:00:00.000+0100</transactionDate>
	<items>
		<productSku>420397</productSku>
		<quantity>1</quantity>
		<price>20</price>
		<revenue>20</revenue>
	</items>
<!-- and so on -->

An example XSLT would look like the following:

<xsl:stylesheet 
  version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:trn="http://demos.tibco.com/retail/transactions"
  exclude-result-prefixes="trn date"
  xmlns:date="http://exslt.org/dates-and-times"
  >

  <xsl:template match="trn:transactionId">
    <transactionId>
      <xsl:value-of select="text()"/>
    </transactionId>
  </xsl:template>
  <xsl:template match="trn:customerId">
    <customerId>
      <xsl:value-of select="text()"/>
    </customerId>
  </xsl:template>
  <xsl:template match="trn:transactionTime">
    <transactionDate>
      <!-- requires exslt, but it is available ootb in JDK -->
      <xsl:value-of select="date:format-date(., 'yyyy-MM-dd HH:mm:ss.SSSZ')"/>
    </transactionDate>
  </xsl:template>
  <xsl:template match="trn:transactionLine">
    <items>
      <productSku><xsl:value-of select="trn:productSKU"/></productSku>
      <quantity><xsl:value-of select="trn:quantity"/></quantity>
      <price><xsl:value-of select="trn:nominalPrice"/></price>
      <revenue><xsl:value-of select="trn:purchasePrice"/></revenue>
      <xsl:if test="count(trn:discount)&gt;0">
      <discount><xsl:value-of select="trn:discount"/></discount>
      </xsl:if>
    </items>
  </xsl:template>

  <xsl:template match="trn:transactionLines">
      <xsl:apply-templates select="trn:transactionLine"/>
  </xsl:template>

  <xsl:template match="trn:transaction">
    <transaction>
      <xsl:apply-templates select="*"/>
    </transaction>
  </xsl:template>

</xsl:stylesheet>

In the end the XML parsing looks like the following:

Use similar way to produce messages. Check for examples in the XSLT operator and Accelerator for Apache Spark available on TIBCO Community.

Attachments

You can download attachments from resources.

transaction-1.xml

transaction-2.xml

transaction.xml