Tag: INTEGRATION

In the area of internet of things (IoT) system integration and integration platforms will reign high and dominate most IT discourse.

It is commonplace that in most enterprise IT infrastructures there is a central enterprise resource planning (ERP) system and multiple supplementary or complimentary applications that augment the capabilities of ERP at the center to serve the needs of business. Today there is an emerging trend that large scale organizations adopt two tier ERP setup where there is high power ERP at center work in tandem with an ERP of lesser caliber placed at identified organization location together with a series of peripheral devices. The complexity of the integration landscape gets compounded by the need to deal with customer or supplier ERPs, different ERPs in subsidiaries or sister companies within the same organization and with multiplicity of devices. All these dimensions of integration entail a robust yet simple integration mechanism that ensure all the applications and devices work seamlessly. But most organizations do not enjoy the luxury of robust yet simple integration mechanism.

One of our customers headquartered in Scandinavia with a global presence used at least three different integration platforms to establish a connection between peripheral applications such as payrolls, expense registration systems, invoice scanning systems, tools handling applications and the central ERP, the IFS Applications at the center.  One such integration platform was a ‘gateway’ tool with extremely limited market penetration, another was the inbuilt ‘connect’ capabilities of their ERP and third was the old fashion ODBC method. None of these were capable of handling complexities of customer’s IT landscape and reach the level of sophistication they desired.

Then they were introduced to the WSO2, a world-renowned integration and identity access management platform (www.wso2.com ). With WSO2 our customer has been able to replace multiple modes of integration with a single mode of integration and thereby achieve the uniformity in integration they desired. Apart from technological superiority and security offered by WSO2, moving to WSO2 brought forward operational advantages to the organization.

As we noticed such operational advantages are,

Reduce Rework and Increase Reuse

Our customer is a global service provider to energy sector with special focus on the offshore market. On daily basis they deal with enormous number of employees and use high number of IT applications such as planning tools, time registration tools, expense management tools, payroll to cater to country specific needs. All these applications need one common set of data, the employee information. Previously there were multiple and ad hoc ways to deal with each peripheral application but with WSO2 there is one employee information output from their ERP that can cater to unique needs of peripheral applications. When the next new application comes no need begin from nothing, but instead existing interfaces can cater to it. So, rework is reduced, and use of existing features is increased.

Access to Resources

One of the integration platforms previously used, the ‘gateway’ tool was a creation of one of their IT suppliers. The knowledge about the tool was with handful of employees at the supplier. Since its low market penetration, no third parties to which the customer could resort to for support or consultancy.

Therefore, the access to quality resources when the customer needs them was extremely limited. But WSO2 comes with state-of-the-art customer support of their own and network of partners who are equally capable of supporting users of WSO2.

Reduce Dependency

With lesser-known products comes the hazel of lesser access to capable resources, lesser competitive pricing and higher dependency on vendor. But with a mature product with significant market penetration, you get the benefit of competent resources at competitive prices and thereby reduce or even diminish dependency on vendors with limited global presence.

Uncompromised Uptime

No product is fool proof, and any product can get glitches here and there. WSO2 ensures uncompromised uptime with superiority of its product backed by professional user support provided by WSO2 itself and a series of capable partners such as CONIFS. This availability of support and thereby constant push to reduce downtime is not a luxury that most integration platforms can boast of.

Integration itself is not a destination but instead it is more of a journey with constant product updates WSO2 together with its partner like CONIFS is committed to enable the next level of IT enablement for enterprises.

Scatter and Gather is one of the common Enterprise Integration Patterns (EIP). As one of the leading open-source integration platforms, the WSO2 Enterprise Integrator (EI) provides an out-of-the-box approach to achieve it using synapse mediators.

In this article, we will be checking two approaches to achieve the Scatter and Gather pattern using Iterate and Clone mediators respectively.

1.Using Iterate-Aggregate mediators

Iterate mediator splits the parent message into smaller messages and sends each child message to the backend service. Then the aggregate mediator can be used to combine responses from each backend call to a single message before responding to the client.

Here is a sample API service that uses the Iterate and Aggregate mediator combination.

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse" name="IterateAPI" context="/employees">
   <resource methods="POST">
      <inSequence>
         <iterate id="employee_iterator" preservePayload="true" attachPath="json-eval($.employees)" expression="json-eval($.employees)">
            <target>
               <sequence>
                  <send>
                     <endpoint>
                        <http method="POST" uri-template="http://run.mocky.io/v3/a9d63220-8fa8-4021-add0-28a635e965b2" />
                     </endpoint>
                  </send>
               </sequence>
            </target>
         </iterate>
      </inSequence>
      <outSequence>
         <property name="info" scope="default">
            <Aggregated_Response />
         </property>
         <aggregate id="employee_iterator">
            <completeCondition>
               <messageCount min="-1" max="-1" />
            </completeCondition>
            <onComplete expression="json-eval($)" enclosingElementProperty="info">
               <respond />
            </onComplete>
         </aggregate>
      </outSequence>
      <faultSequence />
   </resource>
</api>

The API service is designed to accept a JSON message and split the message into employee JSON objects, before sending it to the desired backend service.

Use the below payload to invoke the above API service.

{
    "employees":[
        {"emp_id":15693, "name":"Adrian"},
        {"emp_id":16180, "name":"Andrea"},
        {"emp_id":15025, "name":"Barry"}
   ]
}

You will get a response as below.

{
    "Aggregated_Response": [
        {"Response": "Success" },
        { "Response": "Success" },
        {"Response": "Success" }
    ]
}

2. Using Clone-Aggregate mediators

Similar to the Iterate mediator, the clone mediator also can be used to send messages to multiple backends. The difference is that when the Iterate mediator splits the original message into child messages the clone mediator clones the original message and sends identical copies to the backend.

Try the below API service to understand the clone mediator functionality.

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse" name="CloneAPI" context="/employees">
   <resource methods="POST">
      <inSequence>
         <clone>
            <target>
               <endpoint name="endpoint_1">
                  <http method="POST" uri-template="http://run.mocky.io/v3/6e367e76-e197-4ab3-a4c9-ea659b8ffbe9" />
               </endpoint>
            </target>
            <target>
               <endpoint name="endpoint_2">
                  <http method="POST" uri-template="http://run.mocky.io/v3/332f7d23-e4d3-427c-91bf-b8fd2d35c6df" />
               </endpoint>
            </target>
            <target>
               <endpoint name="endpoint_3">
                  <http method="POST" uri-template="http://run.mocky.io/v3/537616ea-355c-47f1-ad3c-64fa3378661f" />
               </endpoint>
            </target>
         </clone>
      </inSequence>
      <outSequence>
         <property name="info" scope="default">
            <Aggregated_Response />
         </property>
         <aggregate>
            <completeCondition>
               <messageCount min="-1" max="-1" />
            </completeCondition>
            <onComplete expression="json-eval($)" enclosingElementProperty="info">
               <respond />
            </onComplete>
         </aggregate>
      </outSequence>
      <faultSequence />
   </resource>
</api>

Use the below payload to invoke the above API service.

{
    "employees":[
        {"emp_id":15693, "name":"Adrian"},
        {"emp_id":16180, "name":"Andrea"},
        {"emp_id":15025, "name":"Barry"}
   ]
}

You will get the below response.

{
    "Aggregated_Response": [
        {"Response": "From Backend 1"},
        {"Response": "From Backend 2"},
        {"Response": "From Backend 3"}
    ]
}

The Iterate mediator sends every request to the same backend service. But when using the Clone mediator, we can invoke different backend services parallelly and aggregate the response from each.

That’s it then.

Each approach has its pros and cons. Give both a try and see what suits you the most. 

We suppose that you learned something out of this. Keep in touch with us as we will be sharing more content in our future blog posts.

Thanks for reading.

– Dilhara Hewamaddumage

It is common to use query and path parameters when invoking integration services. Here we will be learning how we can read query and path parameters of the incoming request and access them inside the integration logic.

1. Reading query params

There are two ways of doing this:

• Using synapse xpath variable $url

Using the synapse xpath variable is the easiest and the best way to access query parameters. It is faster than using the get-property function.

Let’s assume we are invoking an API service with two query params called ‘first_name’ and ‘last_name’. The following API service will read those query params and return them as the response payload.

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse" name="ParamAPI" context="/params">
   <resource methods="GET">
      <inSequence>
         <!-- set parameter values to response payload -->
         <payloadFactory media-type="json">
            <format>{"Hello":"$1 $2"}</format>
            <args>
               <arg evaluator="xml" expression="$url:first_name" />
               <arg evaluator="xml" expression="$url:last_name" />
            </args>
         </payloadFactory>
         <respond />
      </inSequence>
   </resource>
</api>

You can invoke the above APi using the URL:

https://localhost:8243/params?first_name=John&last_name=Doe

The above API service shows how to read query param values using $url:first_name, $url:last_name and set them in the response payload.

Similarly, you can read param values to properties as below.

<property name="first_name" expression="$url:first_name" scope="default" type="STRING" />
<property name="last_name" expression="$url:last_name" scope="default" type="STRING" />

• Using the get-property(‘query.param.<param_name>’)

Here is a sample API service that uses the get-property() function.

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse" name="ParamAPI" context="/params">
   <resource methods="GET">
      <inSequence>
         <payloadFactory media-type="json">
            <format>{"HI":"$1 $2"}</format>
            <args>
               <arg evaluator="xml" expression="get-property('query.param.first_name')" />
               <arg evaluator="xml" expression="get-property('query.param.last_name')" />
            </args>
         </payloadFactory>
         <respond />
      </inSequence>
   </resource>
</api>

You will get a similar result as in earlier.

2. Reading path params

In order to read path params, we have to set the resource path dynamically. Ex: /{name}. Then we can read the param value using the get-property function as below.

get-property(‘uri.var.name’)

Try this API service to test the theory.

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse" name="ParamAPI" context="/params">
   <resource methods="GET" uri-template="/{name}">
      <inSequence>
         <payloadFactory media-type="json">
            <format>{"HI":"$1"}</format>
            <args>
               <arg evaluator="xml" expression="get-property('uri.var.name')" />
            </args>
         </payloadFactory>
         <respond />
      </inSequence>
   </resource>
</api>

Invoking the API: https://localhost:8243/params/John

The value ‘John’ set for the /{name} path parameter has been read and set to the response payload.

That’s it then. Now you know how to read query and path params in integration services and use them in the integration flow.

Thanks for reading.

– Dilhara Hewamaddumage

WSO2 Enterprise Integrator (EI) facilitates more than one option of routing messages. One of such approaches is to route the request based on its transport headers. This is a variant of the integration pattern content-based routing but rather than checking the message payload, we will be using transport headers to determine the message destination.

For this article, let’s assume a scenario where there are two backend services such that one accepts and responds to JSON messages whereas the other one accepts and responds to XML messages. The integration service will check a  ‘Content Type’ of the message and forward the request to the corresponding backend service.

We can use the below switch mediator configuration to achieve the above use case.

<switch source="$trp:Content-Type">
   <case regex="application/json">
      <send>
         <endpoint>
            <http uri-template="http://run.mocky.io/v3/dce12279-030d-4190-805c-8a48d5997fec" />
         </endpoint>
      </send>
   </case>
   <case regex="application/xml">
      <send>
         <endpoint>
            <http uri-template="http://run.mocky.io/v3/029baaef-a803-486e-a4ad-1ecb1ca0e04a" />
         </endpoint>
      </send>
   </case>
   <default>
      <payloadFactory media-type="json">
         <format>{"Error":"Unsupported Content Type"}</format>
         <args />
      </payloadFactory>
   </default>
</switch>

Alternatively, we can configure the switch mediator below as well.

<switch source="get-property(‘transport’,’Content-Type’">
…
</switch>

But using the synapse xpath variable $trp works better when accessing transport headers [1].

Here is the complete API service with the above switch mediator logic.

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse" name="TestAPI" context="/routing/header">
   <resource methods="POST">
      <inSequence>
         <switch source="$trp:Content-Type">
            <case regex="application/json">
               <send>
                  <endpoint>
                     <http uri-template="http://run.mocky.io/v3/dce12279-030d-4190-805c-8a48d5997fec" />
                  </endpoint>
               </send>
            </case>
            <case regex="application/xml">
               <send>
                  <endpoint>
                     <http uri-template="http://run.mocky.io/v3/029baaef-a803-486e-a4ad-1ecb1ca0e04a" />
                  </endpoint>
               </send>
            </case>
            <default>
               <payloadFactory media-type="json">
                  <format>{"Error":"Unsupported Content Type"}</format>
                  <args />
               </payloadFactory>
            </default>
         </switch>
         <respond />
      </inSequence>
   </resource>
</api>

After deploying the API service, we can test it using messages with different content types.

JSON:

{“Request”:”JSON Message”}

XML:
<Request>XML Message</Request>

The switch mediator has read the Content-Type header and directed the request to the corresponding backend. If the Content-Type header does not exist or match with any of the cases it will execute the <default> block.

Similarly, you can configure the above API to read a custom header as well.

<switch source="$trp:CustomHeader">
…
</switch>

It allows you to achieve more advanced header-based routing use cases.

We hope the article is clear enough to follow and you will learn something from it. You can find more of such content from our blog section.

Thanks.

– Dilhara Hewamaddumage

Content-based routing is one of the common patterns in many integration platforms. Here we will discuss how we are going to achieve it using WSO2 Enterprise Integrator (EI). In simple terms, Content-Based Routing means checking the message content and routing it to the intended flow of execution based on the content. This simplifies the integration development when it is needed to execute different flows depending on variations of the message content while enhancing the code reusability. It is not that difficult to achieve using the WSO2 EI.

There are a few ways to do this. In this article, we will be using an API service with the Switch Mediator which is a built-in synapse mediator packed inside the WSO-EI. The switch mediator facilitates checking message payload and allows us to direct the message in multiple flows.

So where do we begin?

Let’s assume that the client sends the following JSON payload and the value of the ‘symbol’ field varies as ‘IBM’, ‘MSFT’, and ‘Oracle’.

{
   "getQuote":
   {
      "request":{"company":"IBM" }
   }
}

We can use the below switch mediator configuration to check the value of ‘symbol’ and route the request to the corresponding path.

<switch source="json-eval($.getQuote.request.company)">
   <case regex="IBM">
      <send>
         <endpoint>
            <http uri-template="http://run.mocky.io/v3/016870a2-622f-4bb3-9d96-39c02a89d9c0" />
         </endpoint>
      </send>
   </case>
   <case regex="MSFT">
      <send>
         <endpoint>
            <http uri-template="http://run.mocky.io/v3/8477318d-5035-4e2c-b436-fdde88c901aa" />
         </endpoint>
      </send>
   </case>
   <case regex="Oracle">
      <send>
         <endpoint>
            <http uri-template="http://run.mocky.io/v3/e6b3ea9b-d52b-421c-abd0-d980c65caa09" />
         </endpoint>
      </send>
   </case>
   <default>
      <payloadFactory media-type="json">
         <format>{"Error":"Invalid company name. Please set a valid value"}</format>
         <args />
      </payloadFactory>
   </default>
</switch>

If the switch mediator found the value of ‘symbol’ matches with any of the cases, it will select that particular flow of execution and invoke the endpoint. If none of the cases match, it will execute the default flow and return the error message.

Here is the complete API service with the above switch mediator logic.

<?xml version="1.0" encoding="UTF-8"?>
<api xmlns="http://ws.apache.org/ns/synapse" name="TestAPI" context="/routing">
   <resource methods="POST">
      <inSequence>
         <switch source="json-eval($.getQuote.request.company)">
            <case regex="IBM">
               <send>
                  <endpoint>
                     <http uri-template="http://run.mocky.io/v3/016870a2-622f-4bb3-9d96-39c02a89d9c0" />
                  </endpoint>
               </send>
            </case>
            <case regex="MSFT">
               <send>
                  <endpoint>
                     <http uri-template="http://run.mocky.io/v3/8477318d-5035-4e2c-b436-fdde88c901aa" />
                  </endpoint>
               </send>
            </case>
            <case regex="Oracle">
               <send>
                  <endpoint>
                     <http uri-template="http://run.mocky.io/v3/e6b3ea9b-d52b-421c-abd0-d980c65caa09" />
                  </endpoint>
               </send>
            </case>
            <default>
               <payloadFactory media-type="json">
                  <format>{"Error":"Invalid company name. Please set a valid value"}</format>
                  <args />
               </payloadFactory>
            </default>
         </switch>
         <respond />
      </inSequence>
   </resource>
</api>

After adding the above API to the EI, you can test it by setting different values to the given request payload.

This is a simple API service to test the concept of content-based routing in EI. You can achieve more advanced use cases by configuring the above flow according to your requirements. We hope this was a good read.