Configure Parallel Concurrent Processing (PCP) in EBS R12

Parallel Concurrent Processing (PCP):

PCP is the method by which we configure the Concurrent Manager in a multi tier environment with 2 or more concurrent nodes. This allows concurrent processing load to be distributed across the nodes and provides high availability in case of node failure. Managers migrate to surviving node (failover)when one of the concurrent nodes goes down and migrate back (failback) when the failed node comes back.

Each node with concurrent managers may or may not be running an ORACLE instance. The concurrent manager(s) connect via sqlnet to database using tns alias specified by TWO_TASK in adcmctl.sh and gsmstart.sh on each concurrent node.

When the primary node fails, the ICM will restart the manager on the secondary node. If the ICM’s node fails, an Internal Monitor on surviving node can spawn a new ICM on that node.

Services/Managers move back to their primary nodes when those nodes come back up.

What is the role of ICM in PCP

Internal Manager (ICM) monitors, activates and deactivates all managers.

ICM migrates managers during node and/or instance failures and needs to be active for failover/failback to work.

ICM uses the Service Manager (FNDSM) to spawn and terminate all concurrent manager processes, and to manage GSM services like Workflow mailer, Output Post Processor, etc.

ICM will contact the APPS TNS Listener on each local and remote concurrent processing node to start the Service Manager on that node.

ICM will not attempt to start a Service Manager if it is unable to TNS ping the APPS TNS Listener

One Service Manager is defined for each application node registered in FND_NODES.

Each service/manager may have a primary and a secondary node. Initially, a concurrent manager is started on its primary node. In case of node failure, all concurrent managers on that node migrate to their respective secondary nodes.

Service Manager and PCP

Service manager (FNDSM process) is used to manage services/managers on each concurrent node. It is a requirement in all concurrent processing environments and is therefore an integral part of PCP. PCP cannot be implemented without Service manager.

The Service Manager is spawned from the APPS TNS Listener

The APPS TNS Listener must be started on every application node in the system, and started by the user that starts ICM (e.g. applmgr)

TNS Listener spawns Service Manager to run as agent of ICM for the local node

The Service Manager is started by ICM on demand when needed. If no management actions are needed on a node, Service Manager will not be started by ICM until necessary. When ICM exits its Service Managers exit as well.

The Service Manager environment is set by gsmstart.sh and APPSORA.env as defined in listener.ora

Internal Monitors and PCP

The only function of Internal Monitor (FNDIMON process) is to check if ICM is running and restart failed ICM on local node.

Internal Monitors are seeded on every registered node by default by autoconfig.

Activate Internal Monitor on each concurrent node where the ICM can start in case of a failure. By default, Internal Monitor is deactivated.

If the ICM goes down, the Internal Monitor will attempt to start a new ICM on the local node.

If multiple ICMs are started, only the first will stay active. The others will gracefully exit.

Basic steps to configure parallel concurrent Processing (PCP) in EBS R12 :

A) Backup all .ora files

Copy the existing tnsnames.ora, listener.ora and sqlnet.ora files, where they exist, under the 10.1.2 and 10.1.3 ORACLE_HOME locations on the each node to preserve the files

B) Edit Context file

Stop the application on all nodes. Edit the applications context file via Oracle Applications Manager

Set the value of the variable

> APPLDCP to ON
> s_applcsf to <shared location between apps servers>
> s_appltmp to <shared location between apps servers>

C) Edit parameters in spfile (for the transaction manager)

By user having sysdba privilege

alter system set “_lm_global_posts”=true scope=spfile;
alter system set “_immediate_commit_propagation”=true scope=spfile;

D) Edit utl_file_dir variable

From database node as sysdba

ALTER SYSTEM SET utl_file_dir='<shared temp location’ SCOPE=BOTH sid=’*’;

E) Bounce database to reflect the changes made

Using grid user from any database node

srvctl stop database -d <db_name>
srvctl start database -d <db_name>

F) Execute AutoConfig

Execute AutoConfig by running the following command on all concurrent processing nodes:

$INST_TOP/admin/scripts/adautocfg.sh

G) Check the tnsnames.ora and listener.ora configuration files

Check the tnsnames.ora and listener.ora configuration files, located in $INST_TOP/ora/10.1.2/network/admin. Ensure that the required FNDSM and FNDFS entries are present for all other concurrent nodes.

H) Start the Applications

Log on to Oracle E-Business Suite Release 12 using the SYSADMIN account, and choose the System Administrator Responsibility.

• Navigate to Install > Nodes screen, and ensure that each node in the cluster is registered.
• Set up the primary and secondary node names

Navigate to Concurrent > Manager > Define, and set up the primary and secondary node names for all the concurrent managers according to the desired configuration for each node workload.

• Verify that the Internal Monitor and Service Manager for each node is defined properly, with correct primary node specification, and work shift details. For example, Internal Monitor: Host1 must have primary node as host1. Assign a standard work shift with one process to both managers. Also ensure that the Internal Monitor manager is activated: this can be done from Concurrent > Manager > Administrator.
• set Concurrent: TM Transport Type profile

Navigate to Profile > System, change the profile option ‘Concurrent: TM Transport Type’ to ‘QUEUE’, and verify that the transaction manager works across the Oracle RAC instance.

• If any of the transaction managers are in deactivated status, activate them from Concurrent > Manager > Administrator

What happens if we don’t assign any primary/secondary node

Managers with no primary node assignment will be assigned a default target node. In general this will be the node where the ICM is currently running.

I) Checks

>> In the tnsnames.ora file of each Concurrent Processing node ensure that there is an alias that matches the instance name from GV$INSTANCE of each Oracle instance on each RAC node in the cluster.

>> Ensure that the Applications Listener is active on each node in the cluster where Concurrent or Service processes will execute.

>> Stop and restart the Concurrent Manager processes on their primary node(s) and verify that the managers are starting on their appropriate nodes. On the target (secondary) node in addition to any defined managers you will see an FNDSM process (the Service Manager), along with the FNDIMON process (Internal Monitor).