10 Using Enterprise Manager For Grid Automation With Deployment Procedures

10.1.1 Deployment Procedures Shipped In Oracle Enterprise Manager

In version 10.2.0.3 of Oracle Enterprise Manager, the out-of-box deployment procedures include the following:

• Application Server Deployment

• Oracle Clusterware/Oracle Real Applications Clusters (RAC) Provisioning

• Delete/Scale Down Oracle Real Applications Clusters

• One Click Extend Cluster Database

• Patch Oracle RAC Database -- All Nodes

• Patch Oracle RAC Database -- Rolling

• Patch Oracle Clusterware (Rolling Upgrade)

• Patch Oracle Database

• Patch Application Server

• Patch Solaris Hosts

• Patch Linux Hosts

• Patch Windows Hosts

• Patch Standalone Oracle Automatic Storage Management

10.3.1 Using Deployment Procedures to Apply Security-Related Critical Patch Updates to Oracle Databases

The out-of-box deployment procedure Patch Oracle Database can be used to apply critical patch updates (CPU) to several single instance databases simultaneously. In the following example, patch 5049080, a critical patch, is applied to 10.2.0.1 databases.

This involves the following steps:

1. Patch Download and Upload step (optional)

   Upload to Patch Cache is required only if there is no connection between the Management Server and Oracle Metalink. In 10.2.0.3 the Software Library is integrated with the Patch Cache. While uploading a patch to the Patch cache, you can choose the ÒCreate Oracle Software Update ComponentÓoption. This will ensure that the patch is visible in the Software Library.

2. Runtime (Deployment time) steps:

   There are 3 inputs that must be provided at runtime - the patch number(s), the targets and the credentials. Finally the procedure must be scheduled for immediate or deferred execution.

   1. Select the procedure ÒXYZ Corp's CPU patching procedureÓ and run it.

      You can choose to select the patch from Metalink or from the Software Library.

   2. Deployment Procedure by default will run the default SQL (catcpu.sql for CPUs) if present or you can provide custom SQL script to run.

   3. The targets are then chosen from a list of targets that are automatically populated on the screen based on the version of the product the patch applies to.

   4. Credentials follow. You can choose to use the ORACLE_HOME credentials from the repository.

   5. Finally the procedure is scheduled and submitted. The procedure can then be monitored by using ÒProcedure Completion StatusÓ link. It can be retried from a failed step, if required.

10.3.2 Using Deployment Procedures for Single-Click Extend of Real Application Clusters

The out-of-box "Extend Real Application Clusters" deployment procedure can be used for extending an existing Real Application Cluster to one or many nodes. In the following example, an existing cluster is extended to one additional node.

Use the following steps:

1. Runtime (Deployment time) steps:

   1. Run the out-of-box Extend Real Application Clusters procedure.

   2. Choose the existing cluster to be extended and mention the details of the new node where the cluster will be extended.

   3. Mention the credential information for the new nodes and a schedule for extension operation. After you finish, click the Submit button to execute the deployment procedure.

   Once you submit the procedure, you can monitor it using the Procedure Completion Status link.

10.3.3 Using Deployment Procedures for Delete/Scale Down of Real Application Clusters

You can use the out-of-box Delete/Scale Down Real Application Clusters deployment procedure to delete or scale down an existing Real Application Cluster.

Use the following steps:

1. Runtime (Deployment) steps:

   1. Run the out-of-box Delete/Scale Down Real Application Clusters procedure.

   2. From the list of available nodes, select one, multiple, or all nodes for deletion from the cluster.

   3. Mention the credential information for the new nodes and a schedule for extension operation. When you finish, click Submit to execute the deployment procedure.

Once you submit the procedure, you can monitor it using the Procedure Completion Status feature.

10.4.1 Customization Examples

The following sections describe three examples that illustrate how deployment procedures can be customized.

10.4.2 Importing or Exporting Deployment Procedures

In version 10.2.0.3 there is an out-of-box PARDeploy utility to export deployment procedures created by a user and distribute them across various Oracle Enterprise Manager deployments.

Provisioning Archive files can contain deployment procedures and/or components and directives from Software Library. Oracle provides PAR files that contain Oracle best-practices deployment procedures and the directives required to run them.PARDeploy tool is located at $ORACLE_HOME/bin directory. Please be sure the $ORACLE_HOME environment variable is set to the oracle home of the OMS and the Software Library path is configured before you run PARDeploy. When you run $ORACLE_HOME/bin/PARDeploy, it will print out the following usage information.

Usage:

parFile <file> -force(optional)

PARDeploy -action <deploy|view> -parDir <dir> -force(optional)

PARDeploy -action export -guid <procedure guid> -file

<file> -displayName <name> -description <desc> -metadataOnly(optional) PARDeploy -check

• -force <force> -- Force the Software Library entities to be created

• -check <check> -- Check to determine whether Software Library is configured

• -file <file> -- PAR file

• -action <deploy|view|export> -- Deploy, view, or export par file

• displayName <displayName> -- PAR file name

• parDir <dir> -- Directory where par files are located

• metadataOnly <metadataOnly> -- Flag for metadata-only exports

• guid <guid> -- Procedure GUID to export

• parFile <file> -- Path of PAR file

• description <description> -- PAR file description

PAR files provided by Oracle are located at $ORACLE_HOME/sysman/prov/paf. To deploy asprov.par in that directory, you can run the following command.

$ORACLE_HOME/bin/PARDeploy -action deploy -parFile $ORACLE_HOME/sysman/prov/paf/asprov.par -force

You can also deploy all of the PAR files in that directory by running the following command:

$ORACLE_HOME/bin/PARDeploy -action deploy -parDir $ORACLE_HOME/sysman/prov/paf/ -force

To create a PAR file that contains a specific procedure, use the export action. You must have the GUID of the deployment procedure, which you can acquire by logging into Enterprise Manager and moving to the Deployments tab. Then you can click on the Deployments procedure link in the Deployment Procedure Manager section. In the table, find the deployment procedure that you want to export and click on the name. In the procedure view page, note the URL address on the Navigation toolbar. The format should be similar to this:

http://<OMS host>:<port>/em/console/paf/procedureView?guid=<value of GUID>

You will then need to use the GUID value to run PARDeploy tool. For example, if the GUID of the deployment procedure that you want to export is FAC05DD31E3791C3E030579D23106C67, then run the following command:

$ORACLE_HOME/bin/PARDeploy -action export -guid FAC05DD31E3791C3E030579D23106C67 -file exportedDP.par -displayName "User exported DP" -description "Deployment Procedure to be copied to other OMS"

After you run this command, a new PAR file named exportedDP.par will be created in the directory where you run the command. Then you can import this PAR file to another OMS. You can use this PARDeploy tool to import or deploy it to an OMS, or you can also log in to the second Enterprise Manager, go to the Deployment Procedures main page and click on the Upload button to upload the PAR file.

Please note that when a procedure is exported using PARDeploy, any directives or components referred by the procedure will also be exported. However, only the latest revision of these directives or components will be exported. If you do not want to export components or directives, you can specify the metadataOnly flag when running PARDeploy.

10.5.1 Design Time Experience Example (One Time)

This process must be completed only once to customize the deployment procedure.

1. User selects the Oracle shipped Patch Oracle Database Deployment Procedure and does a Create Like for that deployment procedure and provides it a name.

2. User selects run-as sudo for all steps in the edited deployment procedure

3. User selects the sudo command to be run. In this example it may be Òsudo –u oracleÓ

4. User gives the staging location for the directives in ÒStaging Area PathÓ. The default is %oracle_spool%/EMStagedPatches/Others. Example: It could be %emd_root%/EMStagedPatches/Others This directory needs to have 770 permissions. This could also be any other directory which can be written onto by user ÒoracleÓ.

5. User saves the modified deployment procedure (Patch Oracle Database with sudo).

10.5.2 Run Time Experience (Every Time)

This process must be completed for every run of the deployment procedure. User selects the modified patching deployment procedure (Patch Oracle Database with sudo) and runs it.

1. User selects the Database Updates to be patched.

2. User provides the location for staging the patch. The default for this is: User gives the PA Staged location as: %oracle_spool%/EMStagedPatches/Others Example: It could be %emd_root%/EMStagedPatches/Others This directory needs to have 770 permissions and it should exist before the DP is run. This could also be any other directory which can be written onto by user ÒoracleÓ.

3. Now user gives foo's credentials (Patching User credentials).

4. Schedule and submit the job.

10.7.1 Creating the RuntimeData Templates

If you have tested and are satisfied with the provisioning procedures provided by Oracle, you can skip this section. If you have customized the procedures to your environment, you can use these steps to create templates for your system.

10.7.2 Using the RuntimeData XML Instead Of RuntimeData Templates

Although the previous discussion describes a very generic approach for submitting any deployment procedure using EMCLI where the inputs are a Properties file and Runtime Data template file, there is an alternative approach that uses the same generic script to submit deployment procedures for different use cases using a property file as the only input parameter. No Runtime Data template file is required. Instead, the Runtime data XML file of a previous deployment procedure run is used after downloading it using EMCLI (see Step 1: Locate the Successful Test Run of Your Customized Procedure).

In this approach you can replace the old runtime values in the runtime XML of the previous run with the new runtime values, which are relevant to the new run for the different use case. For example, if you have performed a successful partial cluster scale-down, suppose your deployment runtime XMLfor the run contains the following syntax:

<scalar value="node1" classname="java.lang.String" name="NODE_LIST_FOR_DELETION"/>

On a different occasion, for performing a complete cluster deletion, you might need to have the variable value as:

<scalar value="node1,node2" classname="java.lang.String" name="NODE_LIST_FOR_DELETION"/>

For this you can use a properties file of the form:

<old_value>=<new_value>

In this specific example, the properties file will have:

node1=node1,node2

The script will use the old runtime data XML from the pervious run along with the properties file and it will generate the new runtime data XML file (name will be passed as a parameter) and this can then be used to submit the deployment procedure using EMCLI for the new usecase.

While this second approach would considerably expedite the process, a word of caution is worth mentioning. Since this approach would simply replace an old string with a new string, you might run into issues if the old string is a substring in multiple strings in the deployment procedure runtime XML. In that case the resulting runtime XML might be erroneous. To circumvent this issue, you should format the properties file properly. The rule-of-thumb is to put the specifics before the generics. In particular, while specifying node names you should put the FQDNs before the simple node names. A fragment of a properties file in the form of old-value=new-value pairs is shown below and illustrates this point.

node2.test.com=node3.example.com node1.test.com=node4.example.com node2,node1=node3,node4 node2=node3 node1=node4 ...

10.7.3 Creating a Properties File for Provisioning Procedures

The properties file contains values for the placeholder variables and are essentially in the form of a group of name-value pairs. In the RAC scale-down procedure, the properties file would appear as shown below:

rac_partial_delete_node _PUB_NODELIST_ALL_NODES=node2,node1 rac_partial_delete_node _NODE_LIST_FOR_DELETION=node2 rac_partial_delete_node _HOST_LIST_FOR_DELETION=node2.example.com rac_partial_delete_node _REMAINING_NODE_LIST=node1

The following command may be used to create the properties file. Run this with the runtime data template XML containing the placeholder variables, and the property name as parameters. It will create the property file name in the current working directory:

grep -e "\^.*\^" <runtimedata template xml file> | awk -F"^"'{print $2"="}' | sort | uniq > <prop file>

For example, the command could look like this:

grep -e "\^.*\^" runtimedata_sh.xml | awk -F"^"'{print $2"="}' | sort | uniq > runtime.properties

You can then populate the properties file with appropriate inputs. An alternative approach using the runtime data XML instead of creating templates as described earlier, you could have a set of old-value=new-value pairs in the properties file as seen below:

OraCrs10g_home=OraCrs10g OraASM10g_home1=OraASM10g OraRAC10g_home1=OraRAC10g node2.test.com=stamy02.example.com node1.test.com=stamy01.example.com node2,node1=stamy02,stamy01 node2=stamy02 node1=stamy01

10.7.4 Creating the Properties File for Patching Procedures

The method for creating properties files for patching procedures is identical to the method used for provisioning procedures. The same grep command can be used to generate the properties file by using the template runtime XML for standalone database patching. For the alternative approach using the runtime data XML instead of creating templates, the properties file sample is shown below:

PA_VAR_targetName_1=host1.us.oracle.com PA_VAR_oraHome_1=/scratch/home1 PA_VAR_instances_1_1=target11 PA_VAR_instances_1_2=target12 PA_VAR_targetName_2=host2.us.oracle.com PA_VAR_oraHome_2=/scratch/home2 PA_VAR_instances_2_1=target21

10.7.5 Submitting the Deployment Procedure

After the EMCLI has been set up, follow these steps to submit the deployment procedure:

1. Instantiate the Runtime data

   perl instantiate_runtimedata.pl <source RuntimeData template file name> <destination RuntimeData file name> <Properties file name>

   The destination RuntimeData file name is the file that is generated and contains the substituted property values and is used for the deployment procedure execution. You can pass any desired file name for this parameter. The perl script can be used to generate the runtime data for both provisioning and patching procedures.

2. Retrieve the appropriate procedure GUID

   The GUID associated with a specific procedure can be retrieved using EMCLI or the Enterprise Manager User Interface. To obtain the GUID of a deployment procedure, click on its link. The View Procedure page displays showing the GUID as the last portion of the URL. Alternatively, you use the following EMCLI commands to retrieve the procedure types and the procedure GUID.

   ./emcli get_procedure_types ./emcli get_procedures -type=(procedure type)

3. Submit the deployment procedure for execution.

   To submit the deployment procedure for execution, run the following command:

   emcli submit_procedure -procedure=<DP GUID> -input_file=data:<destination RuntimeData file name> [-schedule="<time>"]

   Usage:

   emcli submit_procedure -procedure="GUID of the procedure" -input_file="data:{file_path}" -instance_name="name of the procedure instance" -schedule= start_time:yyyy/MM/dd HH:mm; tz:{java timezone ID}];

   Description: Submit a deployment procedure

   Example:

   emcli submit_procedure -input_file=data:data.xml - procedure=16B15Cb29C3F9E6CE040578C - -schedule="start_time:2007/6/21 21:23;tz:America/New_York"

10.8.1 Known Issues

If you upgrade an existing 10.2.0.1 or 10.2.0.2 version of Enterprise Manager to version 10.2.0.3, then you must manually re-upload any existing shiphomes for RAC or Application Server procedures.

10.8.2 Troubleshooting

If a specific run of deployment procedures fails, then the following log files can provide insight into the reason of the failure. Correct the reason for failure and rerun the deployment procedure:

• <OMS_ORACLE_HOME>sysman/log/pafLogs/<log file for this session>

• <OMS_ORACLE_HOME>sysman/log/emoms.trc

• <OMS_ORACLE_HOME>sysman/log/emoms.log

For faster resolution of any deployment procedure-related issues, plan to provide these files to support when you create a support request.