OPatch


CRS-6706: Oracle Clusterware Release patch level (‘2565072065’) does not match Software patch level (‘277169099’)

Today on a dirt and quick patching session I end up with the following error when restarting the Oracle Restart Grid Infratructure.

The complete error was:

oracle@anjovm1: [+ASM] crsctl start has
CRS-6706: Oracle Clusterware Release patch level ('2565072065') does not match Software patch level ('277169099'). Oracle Clusterware cannot be started.
CRS-4000: Command Start failed, or completed with errors.

This was because I’ve installed the OJVM RU to the Grid home. The solution was simply:

oracle@anjovm1: ~oracle/ [+ASM] /u00/app/grid/18.0.0/OPatch/opatch rollback -id 28790647

And then I could restart:

oracle@anjovm1:~oracle/ [+ASM] crsctl start has
CRS-4123: Oracle High Availability Services has been started.

Install latest version of Oracle Goldengate 12

With Oracle Goldengate 12 there is now a big change on the way to install: there is an installer!

Up to Oracle Goldengate 11 every new patch was a new complete version. With Oracle Goldengate 12 you need to install a base version and then the latest patch.

Here is a quick sum up of the instructions (done on AIX, but is similar on Linux):

  1. From Oracle website download the Goldengate 12.1.2.1.0 for your operating system
  2. From Oracle support note 1645495.1 download the latest patchset for Goldengate. Check the availability for your OS.
  3. Unzip both downloaded files
  4. Call the <download_dir>/ggs_AIX_ppc_shiphome/Disk1/runInstaller
  5. Follow the instructions by choosing the target DB version and install directory (I recommend $ORACLE_BASE/product/gg)
  6. Oracle GoldenGate 12.1.2.1.0 - Install Wizard - Step 1 of 5

    Oracle GoldenGate 12.1.2.1.0 - Install Wizard - Step 2 of 5

    Oracle GoldenGate 12.1.2.1.0 - Install Wizard - Step 3 of 5

    Oracle GoldenGate 12.1.2.1.0 - Install Wizard - Step 4 of 5

    Oracle GoldenGate 12.1.2.1.0 - Install Wizard - Step 5 of 5

  7. Check it works
  8. # set ORACLE_HOME to an existing Oracle database installation:
    export ORACLE_HOME=$ORACLE_BASE/product/11.2.0.4
    
    # set LD_LIBRARY_PATH to use libraries from $ORACLE_HOME:
    export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_BASE/product/gg
    
    # Start Goldengate:
    cd $ORACLE_BASE/product/gg
    ./ggsci
    
    Oracle GoldenGate Command Interpreter for Oracle
    Version 12.1.2.1.1 19797716 OGGCORE_12.1.2.1.1_PLATFORMS_141026.2215
    AIX 6, ppc, 64bit (optimized), Oracle 11g on Oct 27 2014 00:19:39
    Operating system character set identified as ISO-8859-1.
    
    Copyright (C) 1995, 2014, Oracle and/or its affiliates. All rights reserved.
    
    GGSCI (anjovm1) 1> info all
    
    Program Status Group Lag at Chkpt Time Since Chkpt
    
    MANAGER STOPPED
    GGSCI (anjovm1) 2> exit
    
  9. Install the latest patchset with OPatch
  10. cd <download_dir>/21785294
    $ORACLE_BASE/product/gg/OPatch/opatch apply -oh $ORACLE_BASE/product/gg
    
    Invoking OPatch 11.2.0.1.7
    
    Oracle Interim Patch Installer version 11.2.0.1.7
    Copyright (c) 2011, Oracle Corporation.  All rights reserved.
    
    Oracle Home       : /oracle/app/product/gg121
    Central Inventory : /oracle/oraInventory
       from           : /etc/oraInst.loc
    OPatch version    : 11.2.0.1.7
    OUI version       : 11.2.0.3.0
    Log file location : /oracle/app/product/gg121/cfgtoollogs/opatch/opatch2015-11-11_08-32-55AM.log
    
    Applying interim patch '21785294' to OH '/oracle/app/product/gg121'
    Verifying environment and performing prerequisite checks...
    
    Do you want to proceed? [y|n] y
    User Responded with: Y
    All checks passed.
    
    Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
    (Oracle Home = '/oracle/app/product/gg121')
    
    Is the local system ready for patching? [y|n] y
    User Responded with: Y
    Backing up files...
    
    Patching component oracle.oggcore.ora11g, 12.1.2.1.0...
    Patch 21785294 successfully applied
    Log file location: /oracle/app/product/gg121/cfgtoollogs/opatch/opatch2015-11-11_08-32-55AM.log
    
    OPatch succeeded.
    
  11. Check it is updated
# Start Goldengate:
cd $ORACLE_BASE/product/gg
./ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 12.1.2.1.10 21604177 OGGCORE_12.1.2.1.0OGGBP_PLATFORMS_150902.1337
AIX 6, ppc, 64bit (optimized), Oracle 11g on Sep  2 2015 16:29:21
Operating system character set identified as ISO-8859-1.

Copyright (C) 1995, 2015, Oracle and/or its affiliates. All rights reserved.

GGSCI (anjovm1) 1> 

We have now Oracle Goldengate 12.1.2.1.10 installed.

Interesting enough is that Oracle Goldengate 12.1 still uses the OPatch version 11.2.0.1.7, taken probably from Oracle 11.2.0.3, as we can see from the line “OUI version : 11.2.0.3.0”.


Oracle October 2015 CPU – quick and dirty install on 11.2.0.4 and 12.1.0.2 4

[Update 17.11.2015 – Thanks Lars Johan Ulveseth for the extra testing and comment – see below. In fact I’ve only tested the PSU installation on a non-CDB 12c database. There is a missing step, added now, when you are in a multitenant configuration with pluggable databases. Script updated based on Lars comments.]

Following my January post, here are basic instructions to install the latest Oracle security patch. While one should always read the documentation, below you can find how to install CPU Oct 2015  on single instance. Please count about 5 minutes downtime on your database (a bit more if it is the first time you are doing this).

If you are single instance database (no RAC) and either 11.2.0.4 or 12.1.0.2 here are the quick summary of the installation. Tested on Linux and AIX:

Oracle 12.1.0.2 – October 2015 CPU installation

OPatch: You need to update the OPatch tool, minimum version is 12.1.0.1.7. Direct download from Oracle support.

Bundle 12.1.0.2 – Patch 21520444 – Combo OJVM PSU 12.1.0.2.5 and Database PSU 12.1.0.2.5 for UNIX

  • Patch 21359755 – Database Patch Set Update 12.1.0.2.5 (Oct2015) –> RAC-Rolling Installable
  • Patch 21555660 – Oracle JavaVM Component 12.1.0.2.5 Database PSU (OCT2015) –> Non RAC-Rolling Installable
export PATCH_LOC="/tmp"

1. Download + Upgrade OPatch (Opatch download direct link)

unzip ${PATCH_LOC}/p6880880_121010_Linux-x86-64.zip -d $ORACLE_HOME

2. Stop database+listeners+agent

3. Apply DB PSU (21359755 ) but do NOT run DB PSU post install steps

cd ${PATCH_LOC}
unzip p2*
cd 2*
cd 21359755/
$ORACLE_HOME/OPatch/opatch apply

3. Apply OJVM PSU patch

cd ../21555660/
$ORACLE_HOME/OPatch/opatch apply

4. Restart database [and open pluggable databases] in upgrade mode

SQL> startup upgrade;
SQL> alter pluggable database all open upgrade;

5. Run post install steps

cd $ORACLE_HOME/OPatch
./datapatch -verbose

6. Restart database [and open pluggable databases] in normal mode

SQL> shutdown;
SQL> startup;
SQL> alter pluggable database all open;


Oracle 11.2.0.4 – October 2015 CPU installation

Bundle 11.2.0.4 – Patch 21744335 – Combo OJVM PSU 11.2.0.4.5 and Database SPU 11.2.0.4 (CPUOct2015)

  • Patch 21352646 – Database Security Patch Update 11.2.0.4.0 (CPUOCT2015) –> RAC-Rolling Installable
  • Patch 21555791 – Oracle JavaVM Component 11.2.0.4.5 Database PSU (OCT2015) –> Non RAC-Rolling Installable
export PATCH_LOC="/tmp"
cd ${PATCH_LOC}
unzip p2*.zip
cd 2*

1. Shutdown databases and services
2. Apply DB PSU (21352646) but do NOT run DB PSU post install steps

cd 21352646
$ORACLE_HOME/OPatch/opatch napply -skip_subset -skip_duplicate

3. Apply OJVM PSU patch

cd ../21555791/
$ORACLE_HOME/OPatch/opatch apply

4. Run the OJVM PSU post install steps followed by the DB PSU (or equivalent) post install steps.

cd $ORACLE_HOME/sqlpatch/21555791
sqlplus / as sysdba 
SQL> startup upgrade
SQL> @postinstall.sql
SQL> shutdown immediate
SQL> exit;
cd $ORACLE_HOME/rdbms/admin
$ sqlplus / as sysdba
SQL> startup
SQL> @catbundle.sql cpu apply
SQL> @utlrp.sql
SQL> exit;

 


Check which Oracle PSU are installed

With latest version of OPatch:

$ORACLE_HOME/OPatch/opatch lsinventory -bugs_fixed | grep -i ‘DATABASE PSU’

or

set lines 255

col action_time for a40


col action for a8


col namespace for a7


col version for a11


col comments for a40


select substr(ACTION_TIME,1,40) action_time,substr(ACTION,1,8) action,substr(NAMESPACE,1,7) namespace,


substr(VERSION,1,11) version,ID,substr(COMMENTS,1,40) comments,BUNDLE_SERIES


from registry$history;

To install latest OPatch, after download (make sure you only have OPatch zip for the correct Oracle version: 10.2, 11.1, 11.2):

  • mv $ORACLE_HOME/OPatch $ORACLE_HOME/OPatch.old
  • unzip p6880880* -d $ORACLE_HOME