Good info on Temporary Tables

Temporary tables are used exclusively with Application Engine programs and are intended to provide parallel processing. The Application Engine programs may be executed online via CallAppEngine or invoked through the Process Scheduler.
 

Parallel or concurrent processing allows multiple instances of an Application Engine program to execute against the same tables while drastically reducing the risk for table contention.

 

Batch process performance can be improved by splitting the data to be processed into groups and simultaneously running multiple instances of the program to deal with different groups of data. For example, students could be processed by last name by splitting the group into multiple groups alphabetically.

 

If you have a program that uses a temporary table and it is invoked multiple times, that single temporary table could be used concurrently in multiple executions of the code.

 

This could create unpredictable results since the different instances of the code would be issuing Deletes, Inserts, and/or Updates unsynchronized with each other.

 

You could solve this problem by creating multiple temporary tables as a pool of tables. Each invocation of your program would have to allocate an unused temporary table, mark it as “in use” and release it back to the pool when you are through with it.

 

By using the Temporary Table record type definition, you are able to define a record and the PS Build process will build multiple uniquely named copies of your Temporary Table as a pool.

 

Additionally, PS does Temporary Table management for your Application Engine programs. You can code your program with supplied meta-SQL (%Table) so each execution of your Application Engine program will be given access to its own copy of the Temporary Table for its exclusive use. When the program ends, the table will be returned to the pool of Temporary Tables.

 

Temporary Table Pools

 

Online Pool

•  Defined in PeopleTools Options by PS Admins. Our current setting is five (5).
•  Used by Application Engine programs invoked by CallAppEngine.

Batch Pool

•  Defined in Application Engine program properties. Properties that must be set include:
•  Assign Temporary Tables to the Application Engine program (Temp Table tab)
•  Set the Instance Count (Temp Table tab). The number entered here will dedicate the number of instances requested for the AE program.
•  Set the Runtime option (Temp Table tab). This is the action the AE program will take if batch/dedicated tables cannot be allocated at runtime.
•  Continue – the base table will be used instead (using Process Instance)
•  Abort – program execution terminates.
•  Set the Batch Only checkbox (Advanced tab). If the program will only be run in batch mode and not executed from the CallAppEngine() PeopleCode function, you should use this checkbox. Any dedicated temporary tables used for Batch Only programs do not have online instances created.

Important Notes

•   Instance Count and the Batch Only checkbox should be set prior to building the record definition.
•   If the temporary table was originally built with online instances (Batch Only checkbox is not checked) and then changed to “batch only”, online tables must be dropped manually.

 

Program Meta-SQL

 

A critical step in implementing parallel processing is to make sure that you’ve included all of the appropriate meta-SQL within the PeopleCode that your Application Engine program executes.

 

To reference a temp table (Online or Batch), you need to use:

 

%Table(record)

 

You can reference any table with %Table, but only those records defined as Temporary Tables get replaced by Application Engine with a numbered instance of a Temporary Table from the Temporary Table pool.

 

For batch/dedicated Temporary Tables, when Application Engine resolves any %Table, it checks an internal array to see if a Temporary Table instance has already been chosen for the current record. If so, then Application Engine substitutes the chosen table name. If there are no more batch/dedicated instances available, then Application Engine uses the base table instance by default (if the Runtime option Continue has been chosen). Regardless of whether %Table is in PeopleCode SQL or in an Application Engine SQL Action, the program uses the same physical SQL table.

 

For synchronous calls to Application Engine, an available instance number will be selected at random according to internal rules. Synchronous refers to using the CallAppEngine PeopleCode function; all other methods that you use to invoke Application Engine programs are asynchronous which means the page is not “frozen” while the program runs to completion.

 

Populate your Temporary Table Process Instance with the Process Instance

 

All temporary tables should be keyed by Process Instance as a general rule. Also, if you have opted to use the “Continue” runtime option when batch/dedicated tables can’t be assigned, Process Instance is required as a key field. The current Process Instance is automatically put into the State record, but when you Insert rows into your Temporary Tables you must supply that Process Instance.

 

%ProcessInstance or %Bind(PROCESS_INSTANCE)

 

This meta-SQL returns the numeric (unquoted) Process Instance. The %PROCESSINSTANCE meta-SQL is more efficient and faster than using the %Bind(PROCESS_INSTANCE).

 

Note: The Process Instance value is always zero for programs initiated with CallAppEngine. This is because the program called with CallAppEngine runs “in process”, that is, it runs within the same unit of work as the component with which it is associated.

 

Clear Temporary Tables (%TruncateTable)

 

You do not need to delete data from a Temporary Table manually. The Temporary Tables are truncated automatically at the end of processing. If the shared base table has been allocated because no batch/dedicated instances were available, then Application Engine performs a delete of rows by Process Instance instead of performing a truncate. In such a case, the PROCESS_INSTANCE is required as a high-level key.

 

You can perform additional deletes of Temporary Table results during the run, but you will need to include your own SQL Action that does a %TruncateTable.

 

Implementing Parallel Processing

 

There is no simple switch or checkbox that enables you to turn parallel processing on and off. To implement parallel processing, you need to complete a set of tasks in the order that they appear in the following list.

 

1. Define your Temporary Tables by defining and saving your Temporary Table records as type “Temporary Table”.

 

2. Set the Temporary Table Online pool. This will set the basic Temporary Table Online pool based on the PeopleTools Options specifications. (Note: This is done one time by the PS Admin group).

 

3. Assign Temporary Tables to your Application Engine program in its Program Properties, setting the appropriate number of Instance Counts and Runtime option.

 

4. Build/Rebuild your Temporary Table record. This will build the necessary Batch temporary tables into that record’s Temporary Table pool for use at execution time.

 

5. Code %Table meta-SQL as references to Temporary Tables in your Application Engine program, so that Application Engine can resolve table references to the assigned Temporary Table instance dynamically at runtime.