Skip Headers
Oracle® Fusion Middleware Developer's Guide for Oracle Data Integrator
11g Release 1 (11.1.1)

Part Number E12643-03
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Master Index
Master Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
View PDF

10 Working with Packages

This chapter gives an introduction to Packages and Steps. It also passes through the creating process of a Package and provides additional information about handling steps within a Package.

This chapter includes the following sections:

10.1 Introduction to Packages

The Package is the largest unit of execution in Oracle Data Integrator. A Package is made up of a sequence of steps organized into an execution diagram.

Each step can either succeed or fail its execution. Depending on the execution result (success or failure), a step can branch to another step.

10.1.1 Introduction to Steps

Table 10-1 lists the different types of steps. References are made to sections that provide additional details

Table 10-1 Step Types

Type Description See Section

Flow (Interface)

Executes an Interface.

Section 10.3.1.1, "Executing an Interface"

Procedure

Executes a Procedure.

Section 10.3.1.2, "Executing a Procedure"

Variable

Declares, sets, refreshes or evaluates the value of a variable.

Section 10.3.1.3, "Variable Steps"

Oracle Data Integrator Tools

These tools, available in the Toolbox, enable to access all Oracle Data Integrator API commands, or perform operating system calls

Section 10.3.1.4, "Adding Oracle Data Integrator Tool Steps"

Models, Sub-models, and Datastores

Performs journalizing, static check or reverse-engineering operations on these objects

Section 10.3.1.5, "Model, Sub-Models and Datastore Related Steps"


Figure 10-1 Sample Package

Description of Figure 10-1 follows
Description of "Figure 10-1 Sample Package"

For example, the "Load Customers and Invoice" Package example shown in Figure 10-1 performs the following actions:

  1. Execute procedure "System Backup" that runs some backup operations.

  2. Execute interface "Customer Group" that loads the customer group datastore.

  3. Execute interface "Customer" that loads the customer datastore.

  4. Execute interface "Product" that loads the product datastore.

  5. Refresh variable "Last Invoice ID" step to set the value of this variable for use later in the Package.

  6. Execute interface "Invoice Header" that load the invoice header datastore.

  7. Execute interface "Invoice Lines" that load the invoices datastore.

  8. If any of the steps above fails, then the Package runs the "Send Alert" step that sends an email to the administrator using an Oracle Data Integrator tool.

10.1.2 Introduction to Creating Packages

Packages are created in the Package Diagram Editor. See Section 10.1.3, "Introduction to the Package editor" for more information.

Creating a Package consists of the following main steps:

  1. Creating a New Package. See Section 10.2, "Creating a new Package" for more information.

  2. Working with Steps in the Package (add, duplicate, delete,...). See Section 10.3, "Working with Steps" for more information.

  3. Defining Step Sequences. See Section 10.4, "Defining the Sequence of Steps" for more information.

  4. Running the Package. See Section 10.5, "Running the Package" for more information.

10.1.3 Introduction to the Package editor

The Package editor provides a single environment for designing Packages. Figure 10-2 gives an overview of the Package editor.

Figure 10-2 Package editor

Description of Figure 10-2 follows
Description of "Figure 10-2 Package editor"

Table 10-2 Package editor Sections

Section Location in Figure Description

Designer Navigator

Left side

The Designer Navigator displays the tree views for projects, models, solutions, and other (global) components.

Package Diagram

Middle

You drag components such as interfaces, procedures, datastores, models, sub-models or variables from the Designer Navigator into the Package Diagram for creating steps for these components.

You can also define sequence of steps and organize steps in this diagram.

Package Toolbox

Left side of the Package diagram

The Toolbox shows the list of Oracle Data Integrator tools available and that can be added to a Package. These tools are grouped by type.

Package Toolbar

Top of the Package diagram

The Package Toolbar provides tools for organizing and sequencing the steps in the Package.

Properties Panel

Under the Package diagram

This panel displays the properties for the object that is selected in the Package Diagram.


10.2 Creating a new Package

To create a new Package:

  1. In the Project tree in Designer Navigator, click the Packages node in the folder where you want to create the Package.

  2. Right-click and select New Package.

  3. Type in the Name of the Package.

  4. Go to the Diagram tab.

  5. Add steps as described in Section 10.3, "Working with Steps"

  6. From the File menu, click Save.

10.3 Working with Steps

Packages are an organized sequence of steps. Designing a Package consists mainly in working with the steps of this Package.

10.3.1 Adding a Step

Adding a step depends on the nature of the steps being inserted. See Table 10-1 for more information on the different types of steps. The procedures for adding the different type of steps are given below.

10.3.1.1 Executing an Interface

To insert a Flow (Interface) step:

  1. Open the Package editor and go to the Diagram tab.

  2. Select the interface to add to the Package from the Designer Navigator Projects tree.

  3. Drag and drop the interface into the diagram. A flow Step appears.

  4. Click the step icon in the diagram. The properties panel opens.

  5. In the General tab, edit the Step Name field.

  6. From the File menu, click Save.

10.3.1.2 Executing a Procedure

To insert a Procedure step:

  1. Open the Package editor and go to the Diagram tab.

  2. Select the procedure to add to the Package from the Designer Navigator Projects tree.

  3. Drag and drop the procedure into the diagram. A procedure step appears.

  4. Click the step icon in the diagram. The properties panel opens.

  5. In the General tab, edit the Step Name field.

  6. In the Options tab, set the procedure options if needed.

  7. From the File menu, click Save.

10.3.1.3 Variable Steps

There are different variable steps within Oracle Data Integrator:

  • Declare Variable: When a variable is used in a Package (or in elements of the topology which are used in the Package), it is strongly recommended that you insert a Declare Variable step in the Package. This step explicitly declares the variable in the Package.

  • Refresh Variable: This step refreshes the variable by running the query specified in the variable definition.

  • Set Variable: There are two functions for this step:

    • Assign sets the current value of a variable.

    • Increment increases or decreases a numeric value by the specified amount.

  • Evaluate Variable: This step compares the value of the variable with a given value according to an operator. If the condition is met, then the evaluation step is true, otherwise it is false. This step allows for branching in Packages.

Declaring a Variable

To insert a Declare Variable step:

  1. Open the Package editor and go to the Diagram tab.

  2. In Designer Navigator, select the variable to add to the Package from the Projects tree for a project variable or from the Others tree for a global variable.

  3. Drag and drop the variable into the diagram. A variable step appears.

  4. Click on the step icon in the diagram. The properties panel opens.

  5. In the General tab, fill in the Step Name field. Select Declare Variable in the Step Type.

  6. From the File menu, click Save.

Refreshing a Variable

To insert a Refresh Variable step:

  1. Open the Package editor and go to the Diagram tab.

  2. In Designer Navigator, select the variable to add to the Package from the Projects tree for a project variable or from the Others tree for a global variable.

  3. Drag and drop the variable into the diagram. A variable step appears.

  4. Click on the step icon in the diagram. The properties panel opens.

  5. In the General tab, fill in the Step Name field. Select Refresh Variable in the Step Type.

  6. From the File menu, click Save.

Setting a Variable

To insert a Set Variable step:

  1. Open the Package editor and go to the Diagram tab.

  2. In Designer Navigator, select the variable to add to the Package from the Projects tree for a project variable or from the Others tree for a global variable.

  3. Drag and drop the variable into the diagram. A variable step appears.

  4. Click on the step icon in the diagram. The properties panel opens.

  5. In the General tab, fill in the Step Name field. Select Set Variable in the Step Type.

  6. Select Assign or Increment depending on the operation you want to perform on the variable value.

  7. Type in the Value field the value to set or the increment. This value may be another variable.

  8. From the File menu, click Save.

Evaluating a Variable

To insert an Evaluate Variable step:

  1. Open the Package editor and go to the Diagram tab.

  2. In Designer Navigator, select the variable to add to the Package from the Projects tree for a project variable or from the Others tree for a global variable.

  3. Drag and drop the variable into the diagram. A variable step appears.

  4. Click on the step icon in the diagram. The properties panel opens.

  5. In the General tab, fill in the Step Name field. Select Evaluate Variable in the Step Type.

  6. Select the Operator used to compare the variable value.

  7. Type in the Value field the value to compare with your variable. This value may be another variable.

    Note:

    You can specify a list of values in the Value field. When using the IN operator, use the semicolon character (;) to separate the values of a list.
  8. From the File menu, click Save.

An evaluate variable step can be branched based on the evaluation result. See Section 10.4, "Defining the Sequence of Steps" for more information on branching steps.

10.3.1.4 Adding Oracle Data Integrator Tool Steps

Oracle Data Integrator provides tools that can be used within Packages for performing simple operations. The tools are either built-in tools or Open Tools that enable the user to enrich the data integrator toolbox.

To insert an Oracle Data Integrator Tool step:

  1. Open the Package editor and go to the Diagram tab.

  2. From the Package Toolbox, select the tool that you want to use. Note that Open tools appear in the Plugins group.

  3. Click in the Package diagram. A step corresponding to your tool appears.

  4. In the General tab of the properties panel, fill in the Step Name field.

  5. Set the values for the parameters of the tool. The parameters descriptions appear when you select one, and are detailed in Appendix A, "Oracle Data Integrator Tools Reference".

  6. You can edit the code of this tool call in the Command tab.

  7. From the File menu, click Save.

The following tools are frequently used in Oracle Data Integrator Package:

  • OdiStartScen: starts an Oracle Data Integrator scenario synchronously or asynchronously. To create an OdiStartScen step, you can directly drag and drop the scenario from the Designer Navigator into the diagram.

  • OdiInvokeWebService: invokes a web service and saves the response in an XML file.

  • OS Command: calls an Operating System command. Using an operating system command may make your Package platform-dependant.

The Oracle Data Integrator tools are listed in Appendix A, "Oracle Data Integrator Tools Reference".

Note:

When setting the parameters of a tool via the steps properties panel, graphical helpers allow value selection in a user-friendly manner. For example, if a parameter requires a project identifier, the graphical interface will redesign it and display a list of project names for selection. By switching to the Command tab, you can review the raw command and see the identifier.

10.3.1.5 Model, Sub-Models and Datastore Related Steps

You can perform journalizing, static check or reverse-engineering operations on models, sub-models, and datastores. The process for creating these steps are described in the following sections.

10.3.1.6 Checking a Model, Sub-Model or Datastore

To insert a check step in a Package:

Note:

It is necessary to define the CKM in the model to perform this static check.
  1. Open the Package editor and go to the Diagram tab.

  2. In Designer Navigator, select the model, sub-model or datastore to check from the Models tree.

  3. Drag and drop this model, sub-model or datastore into the diagram.

  4. In the General tab or the properties panel, fill in the Step Name field. Select Model, Datastore or Sub-Model Check in the step type.

  5. Select Delete Error from the Checked Tables if you want this static check to remove erroneous rows from the tables checked in this process.

  6. From the File menu, click Save.

10.3.1.7 Journalizing a Model or a Datastore

To insert a journalizing step:

Note:

It is necessary to define the JKM in the model to perform the journalizing operations.
  1. Open the Package editor and go to the Diagram tab.

  2. In Designer Navigator, select the model or datastore to check from the Models tree.

  3. Drag and drop this model or datastore into the diagram.

  4. In the General tab or the properties panel, fill in the Step Name field. Select Journalizing Model or Journalizing Datastore in the step type.

  5. Set the journalizing options. See Chapter 7, "Working with Changed Data Capture" for more information on these options.

  6. From the File menu, click Save.

10.3.1.8 Reverse-Engineering a Model

To insert a reverse-engineering step:

  1. Open the Package editor and go to the Diagram tab.

  2. In Designer Navigator, select the model to reverse-engineer from the Models tree.

  3. Drag and drop this model into the diagram.

  4. In the General tab or the properties panel, fill in the Step Name field. Select Model Reverse in the step type.

  5. From the File menu, click Save.

Note:

The reverse-engineering options set in the model definition are used for performing this reverse-engineering process.

10.3.2 Deleting a Step

To delete a step:

  1. In the Package toolbar tab, select the Free Choice tool.

  2. Select the step to delete in the diagram.

  3. Right-click and then select Delete Step.

  4. Click OK.

The step disappears from the diagram.

Note:

It is not possible to undo a delete operation in the Package diagram.

10.3.3 Duplicating a Step

To duplicate a step:

  1. In the Package toolbar tab, select the Free Choice tool.

  2. Select the step to duplicate in the diagram.

  3. Right-click and then select Duplicate Step.

A copy of the step appears in the diagram.

10.3.4 Running a Step

To run a step:

  1. In the Package toolbar tab, select the Free Choice tool.

  2. Select the step to run in the diagram.

  3. Right-click and then select Execute Step.

  4. In the Execution window, select the execution parameters:

    • Select the Context into which the step must be executed.

    • Select the Logical Agent that will run the step.

  5. Click OK.

  6. The Session Started Window appears.

  7. Click OK.

You can review the step execution in the Operator Navigator.

10.3.5 Editing a Step's Linked Object

The step's linked object is the interface, procedure, variable, and so forth from which the step is created. You can edit this object from the Package diagram.

To edit a step's linked object:

  1. In the Package toolbar tab, select the Free Choice tool.

  2. Select the step to edit in the diagram.

  3. Right-click and then select Edit Linked Object.

The Editor for the linked object opens.

10.3.6 Arranging the Steps Layout

The steps can be rearranged in the diagram in order to make it more readable.

To arrange the steps in the diagram:

  1. From the Package toolbar menu, select the Free Choice tool.

  2. Select the more steps you wish to arrange:

    • Keep the CTRL key pressed and select each step

    • Drag the cursor on the diagram with the left mouse button pressed.

  3. To arrange the selected steps, you may either:

    • Drag them to arrange their position into the diagram

    • Right-click, then select a Vertical Alignment or Horizontal Alignment option from the context menu.

It is also possible to use the Reorganize button from the toolbar to automatically reorganize the steps.

10.4 Defining the Sequence of Steps

Once the steps are created, you must reorder them into a data processing chain. This chain has the following rules:

A Package has one entry point, the First Step, but several possible termination steps.

Failure Conditions

The table below details the conditions that lead a step to a Failure state. In other situations, the steps ends in a Success state.

Step Type Failure conditions
Flow
  • Error in an interface command.
  • Maximum number or percentage of errors allowed reached.

Procedure Error in a procedure command.
Refresh Variable Error while running the refresh query.
Set Variable Error when setting the variable (invalid value).
Evaluate Variable The condition defined in the step is not matched.
Declare Variable This step has no failure condition and always succeeds.
Oracle Data Integrator Tool Oracle Data Integrator Tool return code is different from zero. If this tool is an OS Command, a failure case is a command return code different from zero.
Journalize Datastore, Model or Sub-Model Error in a journalizing command.
Check Datastore, Model or Sub-Model Error in the check process.
Reverse Model Error in the reverse-engineering process.

Defining the Sequence

To define the first step of the Package:

  1. In the Package toolbar tab, select the Free Choice tool.

  2. Select the step to set as the first one in the diagram.

  3. Right-click and then select First Step.

The first step symbol appears on the step's icon.

To define the next step upon success:

  1. In the Package toolbar tab, select the Next Step on Success tool.

  2. Select one step in the diagram.

  3. Keep the mouse button pressed and move the cursor to the icon of the step that must follow in case of a success, then release the mouse button.

  4. Repeat this operation to link all your steps in a success path sequence. This sequence should start from the step defined as the First Step.

Green arrows representing the success path between the steps, with a ok labels on these arrows. In the case of an evaluate variable step, the label is true.

To define the next step upon failure:

  1. In the Package toolbar tab, select the Next Step on Failure tool.

  2. Select one step in the diagram.

  3. Keep the mouse button pressed and move the cursor to the icon of the step that must follow in case of a failure, then release the mouse button.

  4. Repeat this operation to link steps according to your workflow logic.

Red arrows representing the success path between the steps, with a ko labels on these arrows. In the case of an evaluate variable step, the arrow is green and the label is false.

To define the end of the Package upon failure:

By default, a step that is linked to no other step after a success or failure condition will terminate the Package when this success or failure condition is met. You can set this behavior by editing the step's behavior.

  1. In the Package toolbar tab, select the Free Choice tool.

  2. Select the step to edit.

  3. In the properties panel, select the Advanced tab.

  4. Select End in Processing after failure or Processing after update. The links after the step disappear from the diagram.

  5. You can optionally set a Number of attempts and an Time between attempts for the step to retry a number of times with an interval between the retries.

10.5 Running the Package

To run a Package:

  1. In the Project tree in Designer Navigator, select the Package you want to execute.

  2. Right-click and select Execute.

  3. In the Execution window, select the execution parameters:

    • Select the Context into which the Package will be executed.

    • Select the Logical Agent that will run the step.

  4. Click OK.

  5. The Session Started Window appears.

  6. Click OK.

You can review the Package execution in the Operator Navigator.