Tuesday, June 27, 2017

XML Standards Schema Library Updates

The following new XML Standards have been added to the XML Schema Library:

Membrane Service Proxy 1.4.1

Membrane Service Proxy is an open source, reverse HTTP proxy and framework written in Java for REST and SOAP APIs, that can be used for: Service Virtualization Backend Security HTTP based Integration API Monitoring OAuth2 SSO.
http://schemas.liquid-technologies.com/MembraneSOA/1.4.1/

FpML 5.9

FpML® (Financial products Markup Language) is the industry-standard protocol for complex financial products. It is based on XML (eXtensible Markup Language), the standard meta-language for describing data shared between applications.
Confirmation : http://schemas.liquid-technologies.com/FpML/5.9/confirmation/
Legal : http://schemas.liquid-technologies.com/FpML/5.9/legal/
Pretrade : http://schemas.liquid-technologies.com/FpML/5.9/pretrade/
Record Keeping : http://schemas.liquid-technologies.com/FpML/5.9/recordkeeping/
Reporting : http://schemas.liquid-technologies.com/FpML/5.9/reporting/
Transparency : http://schemas.liquid-technologies.com/FpML/5.9/transparency/


Please contact us if you would like to suggest an XML Standard which should be added to the XML Schema Library.

Tuesday, June 20, 2017

Liquid Data Mapper - Contexts

The Liquid Data Mapper makes building complex data transformations quick and easy, but before you start there are a few basic concepts you need to learn.

We looked at Contexts in the article Aggregate Functions, but it's worth examining this transform again in a little more detail in order to better understand the concept of contexts. It is a good idea to step through the transform using F11 as you follow the description and also experiment with changing the Sum.Context input and look at the different results produced.


Liquid Data Mapper - Contexts
Liquid Data Mapper - Contexts

The output of the transform is a CSV with two columns ‘Store’ and ‘Total Assets’. When the transform runs it reads the XmlReader.Bookstore items, and creates a CsvWriter.Row from each XmlReader.Bookstore entry returned. So the CsvWriter.Row is in the context of the current XmlReader.Bookstore.

Next the mapper reads XmlReader.Bookstore.Name and creates a CsvWriter.Store item. This action is made within the context of the parent CsvWriter.Row, so the CsvWriter.Store value comes from XmlReader.Bookstore.Name using the current XmlReader.Bookstore.

Next the Sum.Result value is requested. The Sum will produce a Sum.Result for each value read from Sum.Context. The Sum.Context points to XmlReader.Bookstore, which is in context, so the Sum evaluates all the Sum.Value items within the Sum.Context. (i.e. the current XmlReader.Bookstore).

The values come from the Multipy.Result, which is calculated by evaluating Multipy.Value1 and Multipy.Value2. An implied context is created for the parent nodes XmlReader.StockItem and XmlReader.Stock so when XmlReader.price and XmlReader.quantity are read they are both taken from the same XmlReader.StockItem.

We can examine the output window for a trace describing what actions will be performed.

0001 [0000] : WriterTextFileDocument(file:02, data:03)
0002 [0000] :     Constant(..\..\Output\Agregate Functions 01.csv:String)
0003 [0000] :     WriteTextFileRow(file:01, value:04)
0004 [0000] :         ReadChildItems(source:05, nodeType:Bookstore (Node))
0005 [0000] :             XmlRootNode(xmlDocument:06, nodeType:Company (Node))
0006 [0000] :                 XmlFileReader(filename:07)
0007 [0000] :                     Constant(..\..\Data Files\CompanyData.xml:String)
0008 [0000] :             WriterTextFileColumn(file:01, type:Store (String?), value:09)
0009 [0000] :                 ReadChildItems(source:10, nodeType:Name (String))
0010 [0000] :                     ReferenceTo(04)
0011 [0000] :             WriterTextFileColumn(file:01, type:Total Assets (String?), value:12)
0012 [0000] :                 SumFunction(Ctx:13, value:14)
0013 [0013] :                     ReferenceTo(04)
0014 [0013] :                         MultiplyFunction(value 1:15, value 2:19)
0015 [0013] :                             ReadChildItems(source:16, nodeType:price (Double))
0016 [0013] :                                 ReadChildItems(source:17, nodeType:StockItem (Node))
0017 [0013] :                                     ReadChildItems(source:18, nodeType:Stock (Node))
0018 [0013] :                                         ReferenceTo(04)
0019 [0013] :                                 ReadChildItems(source:20, nodeType:quantity (Int32))
0020 [0013] :                                     ReferenceTo(16)

A breakdown of this trace is included in the help.


Tip: Execute the transform by pressing Shift-F5. Debug the transform by pressing F5 to start the debugger, then:
    F11 - Step Into
    F9  - Set break point
    F5 - Run (stops on break points)



A free trial of the Liquid Data Mapper is available from:
https://www.liquid-technologies.com/trial-download

Friday, June 16, 2017

Liquid Data Mapper - Sub Functions

The Liquid Data Mapper makes building complex data transformations quick and easy, but before you start there are a few basic concepts you need to learn.

Custom Sub Functions are a good way to keep your transforms maintainable and easy to read by creating a library of custom components with reusable functionality.



Liquid Data Mapper - Sub Functions
Liquid Data Mapper - Sub Functions

In this example we want to tidy up our customers first name and last name, by capitalizing the first letter. The transform to do this is fairly easy to follow, it takes the first letter and makes it upper case, it takes everything after the first letter and makes it lower case and concatenates the 2 bits back together.

Although its simple there are a few parts to it and we need to use it more than once, so it ends up making our transform look quite complex.

The solution is to create a Sub Function.

Sub Function Sample - CapitalizeFirstLetter

Liquid Data Mapper - Sub Functions
Liquid Data Mapper - Sub Functions
This is a simple sub function that will capitalize the firth letter of the input string and return it as an output string.

The name and type of the sub functions inputs and outputs can be configured. You can also add more inputs and outputs as required.

It is important to name the sub function. This can be done by clicking on the canvas and setting the ‘Name’ Property in the properties window. This is the name that will then be used to find the sub function.

Sub functions can also make use of other sub functions.


Liquid Data Mapper - Sub Functions
Liquid Data Mapper - Sub Functions
We can now use the Sub Function 'CapitalizeFirstLetter' in our original transform to massively simplify the diagram but with identical functionality.


Tip: Execute the transform by pressing Shift-F5. Debug the transform by pressing F5 to start the debugger, then:
    F11 - Step Into
    F9  - Set break point
    F5 - Run (stops on break points)



A free trial of the Liquid Data Mapper is available from:
https://www.liquid-technologies.com/trial-download

Wednesday, June 14, 2017

Liquid Data Mapper - Dynamic Data Sources

The Liquid Data Mapper makes building complex data transformations quick and easy, but before you start there are a few basic concepts you need to learn.

It is possible to load data source documents from a URL that is calculated at runtime. These include XML documents, JSON documents, CSV files, and EDI documents.

This makes it possible to open associated files based on data in other files or to process all the files in a folder.


Liquid Data Mapper - Dynamic Data Sources
Liquid Data Mapper - Dynamic Data Sources

This simple example shows how to open two XML files 'ContactList.part1.xml'  and 'ContactList.part2.xml', and combine their contents. Although the filenames come from 'Constant' component values in this example, they could just as easily be values read from another data source.

ListFiles Component


Liquid Data Mapper - Dynamic Data Sources
Liquid Data Mapper - Dynamic Data Sources

This second example shows how to combine all the files in the folder “..\..\Data Files” that match the pattern “ContactList.part*.xml” (relative paths are resolved relative to the transform).

These settings can be edited by selecting the 'ListFiles 1' component and looking at the properties window.
Liquid Data Mapper - ListFiles Component Properties
Liquid Data Mapper - 'ListFiles' Component Properties

In both of the above examples, the first XML document 'ContactList.part1.xml' contains eight 'ContactDetails' elements and the second XML document 'ContactList.part2.xml' contains twelve 'ContactDetails' elements. When you run the transform, the resulting output contains all twenty 'ContactDetails' elements.

<!--Created by Liquid Data Mapper Libraries (www.liquid-technologies.com)-->
<Contacts>
    <ContactDetails>
        <Name>Aleshia Tomkiewicz</Name>
        <Emails>atomkiewicz@hotmail.com</Emails>
        <PhoneNumbers>01835-703597</PhoneNumbers>
    </ContactDetails>
    <ContactDetails>
        <Name>Evan Zigomalas</Name>
        <Emails>evan.zigomalas@gmail.com</Emails>
        <PhoneNumbers>01937-864715, 01714-737668</PhoneNumbers>
    </ContactDetails>
    <ContactDetails>
        <Name>France Andrade</Name>
        <Emails>france.andrade@hotmail.com</Emails>
        <PhoneNumbers>01347-368222</PhoneNumbers>
    </ContactDetails>
    <ContactDetails>
        <Name>Ulysses Mcwalters</Name>
        <Emails>ulysses@hotmail.com</Emails>
        <PhoneNumbers>01912-771311, 01302-601380</PhoneNumbers>
    </ContactDetails>
    <ContactDetails>
        <Name>Tyisha Veness</Name>
        <Emails>tyisha.veness@hotmail.com</Emails>
        <PhoneNumbers>01547-429341, 01290-367248</PhoneNumbers>
    </ContactDetails>
    <ContactDetails>
        <Name>Eric Rampy</Name>
        <Emails>erampy@rampy.co.uk</Emails>
        <PhoneNumbers>01969-886290, 01545-817375</PhoneNumbers>
    </ContactDetails>
    ...etc...
</Contacts>


Tip: Execute the transform by pressing Shift-F5.Debug the transform by pressing F5 to start the debugger
    F11 - Step Into
    F9  - Set break point
    F5 - Run (stops on break points)



A free trial of the Liquid Data Mapper is available from:
https://www.liquid-technologies.com/trial-download

Monday, June 12, 2017

Liquid Data Mapper - Duplicating Nodes

The Liquid Data Mapper makes building complex data transformations quick and easy, but before you start there are a few basic concepts you need to learn.

By default the data mapper diagram will only allow us to connect one data source node connection point to one data target node connection point. In order to connect a second connection point you can add additional duplicate nodes by right clicking on connection point and selecting 'Duplicate'.


Liquid Data Mapper - Duplicating Nodes
Liquid Data Mapper - Duplicating Nodes
In this example, we are copying all the data from our source XML file from 'XML reader 3' to the output 'XML Writer 3', but we also need to take data from a second data source (in this case the second data set are constant values e.g. ‘Joe Bloggs’, but it could just as easily come from another data source such as XML file, Database etc.).

As you can see in the diagram, 'Xml Writer 3' has a Duplicate 'ContactDetails' node allowing us to connect a second data source.

The resulting XML would merge the data from the two data sources and look as follows:

<!--Created by Liquid Data Mapper Libraries (www.liquid-technologies.com)-->
<Contacts>
    <ContactDetails>
        <Name>Aleshia Tomkiewicz</Name>
        <PhoneNumbers>01835-703597</PhoneNumbers>
    </ContactDetails>
    <ContactDetails>
        <Name>Evan Zigomalas</Name>
        <PhoneNumbers>01937-864715 , 01714-737668</PhoneNumbers>
    </ContactDetails>
    <ContactDetails>
        <Name>France Andrade</Name>
        <PhoneNumbers>01347-368222</PhoneNumbers>
    </ContactDetails>
...etc...
    <ContactDetails>
        <Name>Joe Bloggs</Name>
        <Emails>joe@bloggs.com</Emails>
        <PhoneNumbers>0123456789</PhoneNumbers>
    </ContactDetails>
</Contacts>

Notice the last ContactDetails is constructed from our constant data source.


Tip: Execute the transform by pressing Shift-F5.Debug the transform by pressing F5 to start the debugger
    F11 - Step Into
    F9  - Set break point
    F5 - Run (stops on break points)



A free trial of the Liquid Data Mapper is available from:
https://www.liquid-technologies.com/trial-download