I encountered an odd situation recently where the extract pump processes kept ABENDing. The report file listed the OGG-01031 error message.
GGSCI (exadb04.new.fastrieve.com) 23> info all
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNING
EXTRACT RUNNING CBOA 00:00:00 00:00:02
EXTRACT RUNNING CGGMONX 00:00:01 00:00:05
EXTRACT ABENDED PBOA 00:00:00 01:46:40
EXTRACT ABENDED PGGMONX 00:00:00 01:43:14
REPLICAT RUNNING RGGMONY 00:00:00 00:03:24
GGSCI (exadb04.new.fastrieve.com) 1> view report pggmonx
***********************************************************************
Oracle GoldenGate Capture for Oracle
Version 11.2.1.0.1 OGGCORE_11.2.1.0.1_PLATFORMS_120423.0230_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Apr 23 2012 08:42:16
Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.
Starting at 2013-08-14 16:15:12
***********************************************************************
.
.
.
2013-08-14 16:15:28 ERROR OGG-01031 There is a problem in network communication, a remote file problem,
encryption keys for target and source do not match (if using ENCRYPT) or an unknown error. (Reply received
is Unable to open file "./dirdat/gX000017" (error 11, Resource temporarily unavailable)).
I wasn’t able to find very much on this issue, but I did run across MOS note 1318980.1, and it turned out to have the fix in my case. By renaming the associated $GG_HOME/dirchk/*.cps and $GG_HOME/dirchk/*.cpe files, I was able to get the processes running again. Note that in this case the $GG_HOME/dirchk directory resides on an NFS mount. Apparently this type of error can occur on shared file systems causing the $GG_HOME/dirchk entries to be non-updatable.
Extract/Replicat Shows Already Running When Started and Already Stopped When Stopped from GGSCI, OGG-01031, Reply received is Unable to open file (Doc ID 1318980.1)
[exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk > ls -lhF total 696K -rw-rw-r-- 1 oracle dba 72K Aug 14 16:28 PGGMONX.cpe -rw-rw-r-- 1 oracle dba 2.2K Aug 14 16:27 PGGMONX.cps [exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk > mv PGGMONX.cpe PGGMONX.cpe_new [exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk > mv PGGMONX.cps PGGMONX.cps_new [exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk > ls -lhF total 696K -rw-rw-r-- 1 oracle dba 72K Aug 14 16:28 PGGMONX.cpe_new -rw-rw-r-- 1 oracle dba 2.2K Aug 14 16:27 PGGMONX.cps_new [exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk > mv PGGMONX.cpe_new PGGMONX.cpe [exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk > mv PGGMONX.cps_new PGGMONX.cps [exadb04:oracle:FTRVXP4] /u01/app/oracle/product/golden_gate/dirchk > ls -lhF total 696K -rw-rw-r-- 1 oracle dba 72K Aug 14 16:28 PGGMONX.cpe -rw-rw-r-- 1 oracle dba 2.2K Aug 14 16:27 PGGMONX.cps
After restart the processes, everything was caught up in a couple of minutes.
GGSCI (exadb04.new.fastrieve.com) 11> info all Program Status Group Lag at Chkpt Time Since Chkpt MANAGER RUNNING EXTRACT RUNNING CBOA 00:00:07 00:00:04 Description "***** Capture for Bank of America *****" EXTRACT RUNNING CGGMONX 00:02:21 00:00:01 Description "***** Capture for GGMON X *****" EXTRACT RUNNING PBOA 00:00:00 00:00:05 EXTRACT RUNNING PGGMONX 00:00:00 00:00:08 Description "***** Pump for GGMON X *****" REPLICAT RUNNING RGGMONY 00:00:00 00:00:02
