23 Support for Resequencing in Oracle Mediator

This chapter describes support for message resequencing in Oracle Mediator.

This chapter contains the following sections:

Section 23.1, "Introduction to the Resequencer"
Section 23.2, "Resequencing Order"
Section 23.3, "Configuring the Resequencer"
Section 23.4, "Limitations in the Resequencer"

23.1 Introduction to the Resequencer

The resequencer rearranges a stream of related but out-of-sequence messages back into order. It sequences the incoming messages that arrive in a random order and then sends them to the target services in an orderly manner. The sequencing is done based on the sequencing strategy selected.

23.1.1 Groups and Sequence IDs

The resequencer works with two central concepts: groups and sequence IDs. The sequence ID is an identifier part of the message, based on which messages are rearranged. The messages arriving for resequencing are split into groups and the messages within a group are sequenced according to the sequence ID. Sequencing within a group is independent of messages in any other group. Groups in themselves are not dependent on each other and can be processed independently of each other.

Messages attached to certain groups arrive to an Oracle Mediator in the following order:

msg9(a), msg8(b), msg7(a), msg6(c), msg5(a), msg4(b), msg3(c), msg2(b), msg1(a)

Table 23-1 shows how the Oracle Mediator sorts the messages into groups. The order of the messages in each group depends on the type of resequencer used.

Table 23-1 Messages Sorted into Groups

Group c	Group b	Group a
msg6(c), msg3(c)	msg8(b), msg4(b), msg2(b)	msg9(a), msg7(a), msg5(a), msg1(a)

All the groups are processed independently of each other and any error occurring in any group does not affect the processing of other groups.

23.1.2 Identification of Groups and Sequence IDs

Groups and sequence IDs are identified through XPath expressions in the payload. You specify XPath expressions that point to the elements in the message payload, on which:

Grouping is done
Sequencing is done

In the message payload shown in Figure 23-1, CustomerId is the field on which to base instance sequencing and CustomerName is the field on which to base grouping.

Figure 23-1 Message Payload

Description of "Figure 23-1 Message Payload"

Note:

Resequencing is supported only for Oracle Mediators with a request operation type and a request-callback operation type in the WSDL file. In other words, resequencing is not allowed by the user interface if the WSDL operation has a synchronous reply element. For more information about these operation types, see Chapter 24, "Understanding Message Exchange Patterns of an Oracle Mediator."

23.2 Resequencing Order

Oracle Mediator can resequence the incoming messages in a user-specified order. This implementation enables you to specify three types of resequencing orders:

Standard Resequencer
FIFO Resequencer
BestEffort Resequencer

23.2.1 Standard Resequencer

This section describes the standard resequencer.

23.2.1.1 Overview of Standard Resequencer

The standard resequencer supports a standard resequencer pattern, which is relevant for applications that use identifiers from a simple numeric identifier sequence in their messages. The standard resequencer receives a stream of messages that may not arrive in order; it then stores the out-of-sequence messages until a complete sequence based on the sequenceID is obtained. The in-sequence messages are then processed asynchronously, based on their sequenceID.

It is important to note that the messages to outbound services of the standard resequencer Oracle Mediator service component are guaranteed to arrive in sequence. The standard resequencer does not modify the message contents; it just orders them.

23.2.1.2 Information Required for Standard Resequencing

When using the standard Oracle Mediator resequencer, you must always specify a group XPath expression and a sequenceID XPath expression. These are used to specify where the Oracle Mediator resequencer should find the group and the sequence ID in the messages. You must also supply the sequence numbering in terms of start sequenceID and the sequenceID incremental delta. This numbering is used to form each group.

23.2.1.3 Example of the Standard Resequencer

Table 23-2 shows how groups are formed differently for two different values of the incremental delta.

Table 23-2 Groups Formed Differently for Two Different Values

Start SequenceID	Incremental Delta	Group1	Group2	...	Groupn
1	1	1,2,3,4,5,...	1,2,3,4,5,...	...	1,2,3,4,5,...n
1	5	1,5,10,15,...	1,5,10,15,...	...	1,5,10,15,...

Notes:

If the sequence numbering is different for various groups (for example, if the groups do not have the same incremental delta or start sequenceID) and the messages do not arrive in order, then you can use the BestEffort resequencer.
The Oracle Mediator standard resequencer holds back messages in the Oracle Mediator resequencer database until it can produce the right sequence for different groups. It means that if for a given group, the message with a particular sequenceID does not arrive within the timeout period^Foot 1, then the consecutive messages for that group are held back forever. In such a case, you must manually unlock the group through Oracle Enterprise Manager Fusion Middleware Control Console and go to the next available message, skipping the pending message.

23.2.2 FIFO Resequencer

This section describes the FIFO resequencer.

23.2.2.1 Overview of the FIFO Resequencer

The FIFO resequencer supports the FIFO pattern. This pattern is relevant to applications that need sequencing based on the time of arrival of the message to the Oracle Mediator. The FIFO resequencer receives a stream of messages that are in order and processes them in sequence for every group based on the time of arrival to the Oracle Mediator.

It is important to note that the messages to outbound services of the Oracle Mediator acting as a FIFO resequencer are guaranteed to arrive in order based on the time of arrival. Therefore, the messages are delivered in the order they were stored in the resequencer data store.

23.2.2.2 Information Required for FIFO Resequencing

When using the FIFO resequencer, you must always specify a group XPath expression. However, you do not have to specify a sequence ID because the messages are processed according to the time of arrival to the Oracle Mediator configured as a FIFO resequencer. The group XPath expression is used to specify where the FIFO resequencer should find the group information in the message to perform the grouping of the messages. No further configuration is needed for a FIFO pattern.

23.2.2.3 Example of the FIFO Resequencer

Table 23-3 illustrates the behavior of the FIFO resequencer, where msgX(Y,Z) means that the message arrives as message number X to the Oracle Mediator service component and the message contains sequenceID Y and group Z:

Table 23-3 FIFO Resequencer Behavior

Incoming Messages	Sequenced Messages
msg03(2,c) msg06(1,c) msg07(5,a) msg10(3,a) msg10(3,c) msg02(7,a) msg05(9,a) msg12(4,c)	msg12(4,c),msg10(3,c),msg06(1,c),msg03(2,c) msg05(9,a), msg02(7,a), msg10(3,a), msg07(5,a)

Incoming Messages

Sequenced Messages

msg03(2,c)

msg06(1,c)

msg07(5,a)

msg10(3,a)

msg10(3,c)

msg02(7,a)

msg05(9,a)

msg12(4,c)

msg12(4,c),msg10(3,c),msg06(1,c),msg03(2,c)

msg05(9,a), msg02(7,a), msg10(3,a), msg07(5,a)

As shown in Table 23-3, the messages are sequenced based on their time of arrival and the sequenceID is not used for sequencing.

Note:

When using the FIFO resequencer, use a single-threaded inbound adapter to avoid unpredictable results. For example, when you use the file/FTP adapter, the database adapter, or the AQ adapter in front of an Oracle Mediator service component that is configured as a resequencer, make sure to configure the adapters as single-threaded in processing. Otherwise, unpredictable results occur because the arrival time of the messages is calculated when the messages arrive to the Oracle Mediator service component, and not when they arrive to the adapter service.

23.2.3 BestEffort Resequencer

This section describes the BestEffort resequencer.

23.2.3.1 Overview of the BestEffort Resequencer

The resequencer supports a best effort pattern. This pattern is relevant to applications that produce a large number of messages in a short-span of time and cannot inform the resequencer about the identifier to be used for sequencing. Typically, the identifier used for sequencing in such scenarios is of a dateTime type or numeric type. Using the dateTime field as the sequence ID XPath enables you to control the sequencing. It is expected that the messages are sent in sequence by the applications. The Oracle Mediator makes the best effort to ensure that the messages are delivered in sequence.

The BestEffort resequencer can receive a stream of messages that are in order or slightly out of order. This resequencer can also reorder messages based on no knowledge about the increment of the sequence ID. This means that unlike the standard resequencer, you do not need to define the increment of the sequence ID for the BestEffort resequencer in advance. When the messages are processed, they are processed in sequence based on the specified sequence ID and the messages that arrived at that point in time, no matter if a true sequence cannot be obtained. The sequence IDs are either numeric or dateTime. Therefore, sequencing occurs on the numeric order or the dateTime order of the sequence IDs.

The BestEffort resequencer processes messages asynchronously. Whenever new messages are available in the resequencer database, the BestEffort resequencer orders them according to the specified sequence ID, locks and picks messages equal to the value of the maxRowsRetrieved parameter from the ordering, and processes the messages one after one in its own transaction in sequence.

It is important to note that the messages to outbound services of the Oracle Mediator service component acting as a BestEffort resequencer are not guaranteed to arrive in the order based on the sequenceID. At any point of time, a snapshot of the available messages is taken and sequencing is done only on those messages. Therefore, unlike a standard resequencer, it is not guaranteed that any message with a smaller sequence ID value is sent before the messages that arrive earlier, but which have a greater sequence ID value. These messages with a smaller sequence ID value that arrive later are picked and processed in the next cycle when again the snapshot of available messages is taken and the messages are ordered.

23.2.3.2 Information Required for BestEffort Resequencing

When using the BestEffort resequencer, you must specify a group XPath expression and a sequence ID XPath expression. These are used to specify where the resequencer should find the group and the sequence ID in the messages. Unlike the standard resequencer, the BestEffort resequencer has no knowledge about how the sequence is built. No further information is used by the BestEffort resequencer to perform its responsibilities.

23.2.3.3 Example of BestEffort Resequencing

Table 23-4 illustrates the behavior of the BestEffort resequencer, where msgX(Y,Z) means that the message arrives as message number X to the Oracle Mediator service component and the message contains sequenceID Y and group Z:

Table 23-4 BestEffort Resequencer Behavior

Group C	Sequenced Messages
msg03(1,c) msg06(2,c) msg10(3,c) msg12(4,c)	msg12(4,c),msg10(3,c),msg06(2,c),msg03(1,c)

Group C

Sequenced Messages

msg03(1,c)

msg06(2,c)

msg10(3,c)

msg12(4,c)

msg12(4,c),msg10(3,c),msg06(2,c),msg03(1,c)

Note:

For the BestEffort resequencer to work correctly, the messages must be completely synchronous or almost synchronous. Otherwise, they are not sequenced correctly. If the messages do not arrive close to each other, then set the value of the maxRowsRetrieved parameter to 1. In this way, the next message in sequence has the time to deliver the actual message to arrive so it is fetched by the worker thread in the next processing loop and therefore delivered in sequence.

23.3 Configuring the Resequencer

You can configure the resequencer using Oracle JDeveloper. This section describes how to configure the resequencer in Oracle JDeveloper.

23.3.1 How to Determine the Resequencing Level

You should first determine the level at which resequencing must be defined. For Oracle Mediator service components, which have only one operation, configuring resequencing at the operation or service component level results in the same behavior. For Oracle Mediators having multiple operations, specifying the resequencing at the service component level means that resequencing is applied to all the operations. Therefore, messages arriving at any operation are resequenced. By default, the resequencing level is operations.

To set the resequencing level:

Select the resequencing level from the Resequence Level dropdown list in the Mediator Editor, as shown in Figure 23-2.

Figure 23-2 Mediator Editor with Resequence Level Field

Description of "Figure 23-2 Mediator Editor with Resequence Level Field"

If you choose the Resequence Level of component, then the resequencing options under each operation are no longer displayed and the Resequence Mode field appears to set the resequencing mode for the service component. By default, the resequencing mode is set to off.

When you select a resequencing mode, the Resequence Options section is displayed for the service component, as shown in Figure 23-3. If the Resequence Mode field for an operation is set to off, then the Resequence Options section disappears.

Figure 23-3 Mediator Editor with Resequence Options Section

Description of "Figure 23-3 Mediator Editor with Resequence Options Section"

The options in the Resequence Options section change with changes in the resequencing mode.

23.3.2 How to Configure the Resequencing Strategy

This section describes how to configure the resequencing strategy.

23.3.2.1 Standard Resequencing

Table 23-5 describes the fields required to configure the standard resequencer:

Table 23-5 Standard Resequencing

Field Name	Description	Default Value	Mandatory
`groupIDExpression`	XPath that points to the field in the incoming message on which grouping is done.	N/A	N
`sequenceIDExpression`	XPath that points to the field in the incoming message on which resequencing is done.	N/A	Y
`timeoutDuration`	Time period in seconds to wait for an expected message. The resequencer locks the group as timed-out, if a time out happens.	0^Foot 1	N
`sequenceStart`	Starting number of the number sequence.	1	N
`sequenceIncrement`	Increment of the number sequence.	1	N

^Footnote 1This default value means that the timeout never happens for a group by default.

Figure 23-4 shows an Oracle Mediator with the Resequence field set to Standard.

Figure 23-4 Oracle Mediator with Resequence Mode set to Standard

Description of "Figure 23-4 Oracle Mediator with Resequence Mode set to Standard"

23.3.2.2 FIFO Resequencing

Table 23-6 describes the fields required configure the FIFO resequencer:

Table 23-6 FIFO Resequencing

Field Name	Description	Default Value	Mandatory
groupIDExpression	XPath that points to the field in the incoming message on which grouping is done.	N/A	N

Figure 23-5 shows an Oracle Mediator with the Resequence field set to FIFO.

Figure 23-5 Oracle Mediator with Resequence Mode set to FIFO

Description of "Figure 23-5 Oracle Mediator with Resequence Mode set to FIFO"

23.3.2.3 BestEffort Resequencing

Table 23-7 describes the fields required to configure the BestEffort resequencer:

Table 23-7 BestEffort Resequencing

Field Name	Description	Default Value	Mandatory
`groupIDExpression`	XPath that points to the field in the incoming message, on which grouping is done.	N/A	N
`sequenceIDExpression`	XPath that points to the field in the incoming message, on which resequencing is done.	N/A	Y
`sequenceIDDataType`	The data type of the sequence ID. Ordering is done based on the data type. Supported values are date/time and numeric.	Numeric	N
`maxRowsRetrieved`	Number of in-sequence messages that the resequencer should pick from the data store at a time.	5	N

Figure 23-6 shows an Oracle Mediator with the Resequence field set to Best Effort.

Figure 23-6 Oracle Mediator with Resequence Mode set to BestEffort

Description of "Figure 23-6 Oracle Mediator with Resequence Mode set to BestEffort"

23.4 Limitations in the Resequencer

The following limitation of resequencer has been noted in this release:

Resequencer Fails If XSD File Contains Multibyte Characters That the Server Locale Encoding Does Not Support

If the XSD file contains multibyte characters that the server locale encoding does not support, then the resequencer execution fails after triggering the project flow.