Integration Broker Basics

I don't consider myself an Integration Broker expert.  Far from it.  I can get messages to fly from one PeopleSoft instance to another, I can write a transformation if pressed, and I can get messages that are stuck in a "New" status working again.  So I thought I'd go out on a limb here and pass along my mental model of how I visualize Integration Broker working.  If you're an expert, you might find this article simplistic and inaccurate at times and I'm hoping you'll speak up and let me know of the glaring problems.  But if you're a new to IB or trying to support somebody else's integrations you might find something in here useful.

If you think about it, any messaging system needs some basic building blocks to work.  Think about an e-mail system.  You have a Sender and a Recipient.  E-mail addresses are made up of a user (the part before the @ sign) and a domain (the part after the @ sign).  You have an e-mail server that routes the e-mail to the Recipient’s domain.  A server on the Recipient’s domain then gets it to the recipient’s inbox.  When the Recipient opens it, a Return Receipt e-mail might be generated back to the Sender as an acknowledgement.

Integration Broker follows a lot of these same rules.  But since its messages are generated and consumed by machines, things are more structured.
 

Building Blocks of Integration Broker 

Nodes

If you think about the e-mail analogy, the Node would be like the Domain part of the e-mail addresses.  For PeopleSoft-to-PeopleSoft communication, Nodes are (usually) PSFT_EP for Financials, PSFT_HR for HRMS, PSFT_LM for LMS, and PSFT_CRM for CRM.  They basically tell which application a message belongs to.

The node definition is where you define what messages are valid for that node.  Prior to PeopleTools 8.48, you’d define them on the “Transaction” tab.  In 8.48 and above, you’d define them on the “Routings” tab.

Since you might not want just anybody being able to publish a message to a node, you’ll need to set a node password on the first page of the node definition.  This password will have to be the same in all of the environments.  For example, if you want to publish a message to PSFT_EP from HRMS, the PSFT_EP node password will have to be the same in both Financials and HRMS.

Messages

The message definition is where the developer specifies what data a message will contain.  It includes records and fields, and child records are nested under their parent records.

If you think of the e-mail analogy, the message name would be both the User part of the e-mail address, and the message itself would be the Body of the e-mail.  

Transformations

A transformation is a program that gets executed against a message either when it’s sent, or when it’s received.  If you think about it, this can be important because data structures are different with different PeopleSoft versions.  So if you’re sending a message with employee data from HRMS 8.8 back to Financials version 8.4, there’s a good chance that the message that HRMS sends is different from what Financials is able to receive.  So you need a transformation.

The transformation itself is a special type of Application Engine program that integration broker can execute by itself against the XML of a message.  It uses either PeopleCode, or XSLT (a special language for transforming XML) to put the message into the new format.

In the e-mail analogy, this would be like me sending an e-mail to someone who doesn’t speak English.  I'd either have to translate it before I sent it, or the recipient would have to translate it.

Prior to PeopleTools 8.48, you use “Relationships” to associate a Transformation program to a message.  In 8.48 and above, you can associate a Transformation to a Message with either a Service Operation or a Routing.

Gateways

The gateway is kind of like the e-mail server.  It knows the nodes – that is to say for a given node it knows the server name, app server port number, username and password so that it can connect to the node’s app server and push the message to the integration broker running in that environment.

The gateway runs as part of the PIA web server.  Integration Broker sends messages to it with a plain ‘ol Post HTTP request.  This makes talking to Integration Broker pretty easy for 3rd party applications since they don’t have to write any special protocols.  

Asynchronous versus Synchronous

Integration Broker can do either Synchronous or Asynchronous messages.  Synchronous messages are sent and the program waits for a successful response from the remote system before it will continue.

Asynchronous messages are more like the E-mail analogy – the message is sent and the program gets on with its life, assuming the message will be OK.

Most PeopleSoft EIP's are Asynchronous, and so I’ll only talk about Asynchronous messaging in this document.

Message Channels / Queues

PeopleSoft lets you group message definitions into queues.  Queues can be paused or running.  So if you want to keep messages with employee data from trying to go from HRMS to Financials when Financials is going to be down, you can pause the message queue.  When the maintenance is over, you can set the queue to running.

They have another normally unused feature:  You can change how many messages get posted to the integration broker at one time by chunking on specific combinations of fields.  Message Chunking is more of a developer topic, so that’s all I’ll say about it for now.

Prior to PeopleTools 8.48, Queues were called Message Channels.  I don’t believe there’s any real difference in what they are or what they do.

Service Operations

Service Operations were invented in PeopleTools 8.48.  I believe the intention was to create a single place where you can define which nodes a message is valid for and what transformations need to be applied to it.  

Steps in Integration Broker prior to Tools 8.48

1.    PeopleCode event creates and publishes a message

2.    Integration Broker looks to see if the message channel for that message is active.

3.    Integration Broker creates a message instance for the message

4.    Integration Broker looks to see what nodes the message is active for

5.    Integration Broker creates a publication contract for each message node.

6.    Integration Broker looks to see if any “relationships” exist for the source node/target node/message/version combination, and executes the transformation associated with the relationship if it exists.

7.    For each publication contract, Integration Broker publishes the message (in XML format) to the integration gateway.  This includes the Source and Target nodes.

8.    The Integration Gateway looks for the target node in its configuration file (integrationgateway.properties), connects to the application server, and passes the message off to the target integration broker.

9.    Integration broker creates a message instance for the message.

10.    Integration Broker looks to see if the message is set up as an inbound message on the source node.

11.    Integration Broker creates a subscription contract for the source node (if active).

12.    Integration Broker looks to see if any “relationships” exist for the source node/target node/message/version combination and executes the transformation program if applicable.

13.    Integration Broker inserts the message into the database based on the message definition.

Steps in Integration Broker 8.49

The process is basically the same, but the terminology has changed.

1.    PeopleCode event creates and publishes a message

2.    Integration Broker looks to see if the Queue for that message is active.

3.    Integration Broker creates a transaction for the message

4.    Integration Broker looks to see what nodes the message is active for

5.    Integration Broker creates a publication contract for each message node.

6.    Integration Broker looks to see if any transformations programs exist for the service operation routing or the node routing, and executes them if found.

7.    For each publication contract, Integration Broker publishes the message (in XML format) to the integration gateway.  This includes the Source and Target nodes.

8.    The Integration Gateway looks for the target node in its configuration file (integrationgateway.properties), connects to the application server, verifies the node passwords from the source and the target environments match, and passes the message off to the target integration broker.

9.    Integration broker creates a message instance for the message.

10.    Integration Broker looks to see if the message is set up as an inbound message on the source node.

11.    Integration Broker creates a subscription contract for the source node (if active).

12.    Integration Broker looks to see if any transformation programs exist for the service operation routing or the node routing, and executes them if found.

13.    Integration Broker inserts the message into the database based on the message definition.

Integration Gateway Considerations

Integration Broker got an overhaul in PeopleTools 8.48, and the older PeopleTools versions are no longer compatible with the newer PeopleTools versions.

So how can you actually make the old PeopleTools versions send and receive messages with the new PeopleTools versions?  You have to make the older versions use the new Integration Gateway.

So what the does that mean?  Well, you have to do is go to PeopleTools > Integration Broker >  Gateways, and select the LOCAL gateway.  Change the URL to be the same as the environment with latest copy of PeopleTools’ LOCAL gateway URL.

Now if you want to change any Gateway configuration, be sure to do it from the latest PeopleTools environment.  Its bad luck to edit Integration Gateway configuration using a version lower than the gateway is running.  Also older tools versions won’t encrypt passwords like the new ones will.

Also, if you shut down a shared Integration Gateway web server, it’s going to impact Integration Broker on all of the environments that share it.  Messages should catch up whenever you bring the Integration Gateway web server back up, as long as you go to Message Monitor and resubmit the ones in error.

Summary

I'm hoping this clears up some of the compexities of Integration Broker.  Please let me know if I didn't say something right or got something wrong or missed something that should have been covered.

 

Written by Brent Martin