8 Understanding Batch Processing

In situations that warrant the high-performance movement and transformation of very large volumes of data between heterogeneous systems in batch, real time, and synchronous and asynchronous modes, Oracle Application Integration Architecture (AIA) leverages Oracle's extract, transform, and load (ETL) tool called Oracle Data Integrator (Oracle ODI). By implementing an ETL architecture, based on the relevant RDBMS engines and SQL, Oracle AIA can perform data transformations on the target server at a set-based level, giving a much higher performance.

For more information about batch processing, see "Using Oracle Data Integrator for Bulk Processing" in the Oracle Fusion Middleware Developer's Guide for Oracle Application Integration Architecture Foundation Pack.

8.1 Batch Processing Use Cases

Oracle AIA leverages batch processing technology for the following types of use cases:

To perform an initial synchronization of reference data across disparate applications.
To load an Operational Data Store to provide fresh, integrated information.
To load production databases from data entered over the Internet (by sales forces, agencies, suppliers, customers, and third parties) that strictly respects security constraints.
To leverage the use of Cross Reference and Domain Value Map, for those cases in which the data transferred is used for the running of services from an Integration Scenario.

8.2 Bulk Data Integration Patterns

Oracle AIA supports the following bulk data integration patterns:

Initial data loads
High volume transactions with XREF table
High volume transactions without XREF
Intermittent high volume transactions

8.2.1 Initial Data Loads and High Volume Transactions with XREF Table

When implementing a new integration, there may be some records that need to be synchronized. Using Oracle ODI allows you to handle a potentially large data-set and maintain the cross-reference if necessary.

8.2.2 High Volume Transactions Without XREF

If no further DML operations need to occur with synchronized records, maintaining a cross-reference (XREF) record would not be necessary. For example in a retail environment where store locations transmit their daily sales transactions to HQ, if the HQ location does not need to perform DML operations (as in the case of a data-warehouse), then maintaining the cross-reference is not necessary.

8.2.3 Intermittent High Volume Transactions

The Intermittent High Volume Transactions pattern covers the scenario where data integration is managed both by Oracle ODI and by AIA EBS. Normal (instance-based) synchronization is done through AIA EBS, but if there is the need to run transactions in batch, then Oracle ODI can be used to perform the synchronization. Oracle ODI can also be used for intermittent transactions that may contain a very large object, such as an order with an exceptional number of lines.