OALL8 is in an inconsistent state

"OALL8 is in an inconsistent state" is a generic exception that occurs in JDBC 10.1, 10.2, and 11.1. It indicates that an internal inconsistency has been detected in the JDBC connection but it does not provide information on what caused the inconsistency.

OALL8 is in an inconsistent state.
java.sql.SQLException: OALL8 is in an inconsistent state.
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:138)
at oracle.jdbc.driver.T4C8Oall.init(T4C8Oall.java:325)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:170)
at oracle.jdbc.driver.T4CPreparedStatement.execute_for_describe(T4CPreparedStatement.java:503)
at oracle.jdbc.driver.OracleStatement.execute_maybe_describe(OracleStatement.java:1029)
at oracle.jdbc.driver.T4CPreparedStatement.execute_maybe_describe(T4CPreparedStatement.java:535)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1126)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3001)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3043)
at oracle_jdbc_driver_T4CPreparedStatement_Proxy.executeQuery()
at com.agile.pc.cmserver.util.ClusterLock.acquireClusterLock(ClusterLock.java:61)
at com.agile.pc.cmserver.notification.NotifyTask.run(NotifyTask.java:413)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
<2013-02-27 22:59:37,734> <NotifyTask:ERROR> notification task exception

You may also see a "Protocol Violation error, the thin driver throws this exception when it reads something from the RDBMS that it did not expect. This means that the protocol engine in the Thin driver and the protocol engine in the RDBMS are out of synch. There is no way to recover from this error. The connection is dead and You should try to close it, but that will probably fail too.

If you are able to reproduce the error, please file a TAR with Oracle Global Support by specifying the exact version numbers of the JDBC driver and the RDBMS.

Protocol violation
java.sql.SQLException: Protocol violation
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:138)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:175)
at oracle.jdbc.driver.DatabaseError.check_error(DatabaseError.java:898)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:656)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:185)
at oracle.jdbc.driver.T4CPreparedStatement.execute_for_describe(T4CPreparedStatement.java:503)
at oracle.jdbc.driver.OracleStatement.execute_maybe_describe(OracleStatement.java:1029)
at oracle.jdbc.driver.T4CPreparedStatement.execute_maybe_describeT4CPreparedStatement.
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1126)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3001)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3043)
at oracle_jdbc_driver_T4CPreparedStatement_Proxy.executeQuery()
at com.agile.pc.cmserver.util.ClusterLock.acquireClusterLock(ClusterLock.java:61)
at com.agile.pc.cmserver.notification.NotifyTask.run(NotifyTask.java:199)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)

Possible Solution: It is very hard to troubleshoot and the simplest solution is to upgrade your JDBC Driver.

This exception no longer occurs in JDBC 11.2.

Other scenarios & Reference
1. Oracle Transportation Management (OTM) - Version 5.5.04.02 to 5.5.06.03
If you are facing the issue with OTM then check below metalink note for Cause and Solution
java.sql.SQLException: OALL8 is in an inconsistent state [ID 754144.1]

2. Master Note: Understanding the "OALL8 is in an Inconsistent State" Exception [ID 944692.1]

Regards
Satishbabu Gunukula, Oracle ACE
http://www.oracleracexpert.com

ORA-00283: recovery session canceled due to errors

We have restored a database from backup in different system and while doing the recovery we have received the following the error.

Users will see this error when a new Data file is created and it is not part of the backup. An UNNAMED datafile name is added to the control file and file is created under $ORACLE_HOME/dbs

Errors in file /home/oracle/saptrace/diag/rdbms/orac/ORAC/trace/ORAC_pr00_16269.trc:
ORA-00283: recovery session canceled due to errors
ORA-01111: name for data file 83 is unknown - rename to correct file
ORA-01110: data file 83: '/home/oracle/11.2.0/dbs/UNNAMED00083'
ORA-01157: cannot identify/lock data file 83 - see DBWR trace file
ORA-01111: name for data file 83 is unknown - rename to correct file
ORA-01110: data file 83: '/home/oracle/11.2.0/dbs/UNNAMED00083'
Recovery Slave PR00 previously exited with exception 283
ORA-283 signalled during: ALTER DATABASE RECOVER DATABASE UNTIL TIME '2012-10-02:23:15:00'USING BACKUP CONTROLFILE ...

Solution:
Find the UNNAMED data file using below command.
SQL> SELECT FILE#, NAME FROM V$DATAFILE
FILE# NAME
-------- -----------------------------------------
1 /dbs01/ORAC/system01.dbf
2 /dbs01/ORAC/sysaux01.dbf
4 /dbs01/ORAC/users01.dbf
5 /home/oracle/11.2.0/dbs/UNNAMED00083
....

Rename an UNNAMED Data file using below command.
SQL> alter database create datafile '/home/oracle/11.2.0/dbs/UNNAMED00083' as '/dbs01/ORAC/ /prd_data83.dbf';
Database altered.

Once you create data file proceed with recovery, if needed.

Regards
Satishbabu Gunukula, Oracle ACE
http://www.oracleracexpert.com

RMAN Backup using BRTools failed with RMAN-03009, ORA-19504, ORA-27054 errors

Recently I have encountered below error when doing RMAN backup using BRTools. Make sure that you have mounted your backup mount point with below options.

Once you mount your backup mount point with below options you are able to run the backup successfully.

rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,proto=tcp,suid

This solution is application for with or without BRTOOLS using RMAN.

BRTOOLS Error log file

BR0051I BRBACKUP 7.20 (0)
BR0055I Start of database backup: bekozflm.fnd 2013-02-19 15.46.46
BR0484I BRBACKUP log file: /oracle/ORAC/sapbackup/bekozflm.fnd
BR0477I Oracle pfile /oracle/ORAC/102_64/dbs/initORAC.ora created from spfile /oracle/ORAC/102_64/dbs/spfileORAC.ora

BR0280I BRBACKUP time stamp: 2013-02-19 15.46.47
BR0057I Backup of database: ORAC
BR0058I BRBACKUP action ID: bekozflm
BR0059I BRBACKUP function ID: fnd
BR0110I Backup mode: FULL
BR0077I Database file for backup: /oracle/ORAC/backup/cntrlORAC.dbf
BR0061I 6 files found for backup, total size 14242.117 MB
BR0143I Backup type: online
BR0112I Files will not be compressed
BR0130I Backup device type: disk
BR0106I Files will be saved on disk in directory: /oracle/backup/ORAC/backup/bekozflm

BR0280I BRBACKUP time stamp: 2013-02-19 15.46.47
BR0256I Enter 'c[ont]' to continue, 's[top]' to cancel BRBACKUP:c
BR0280I BRBACKUP time stamp: 2013-02-19 15.46.48
BR0257I Your reply: 'c'

BR0259I Program execution will be continued...

BR0370I Directory /oracle/backup/ORAC/backup/bekozflm created
BR0280I BRBACKUP time stamp: 2013-02-19 15.46.49
BR1390I RMAN prepared for backup
BR0202I Saving init_ora
BR0203I to /oracle/backup/ORAC/backup/ORAC ...
BR0202I Saving /oracle/ORAC/102_64/dbs/initORAC.sap
BR0203I to /oracle/backup/ORAC/backup/ORAC ...
BR0280I BRBACKUP time stamp: 2013-02-19 15.46.49
BR0198I Profiles saved successfully

BR0202I Saving /oracle/ORAC/sapdata2/sr3db_2/sr3db.data2
BR0203I to /oracle/backup/ORAC/backup/bekozflm/sr3db.data2.S ...
BR0278E Command output of 'SHELL=/bin/sh /oracle/ORAC/102_64/bin/rman nocatalog':

Recovery Manager: Release 10.2.0.4.0 - Production on Tue Feb 19 15:46:49 2013
Copyright (c) 1982, 2007, Oracle. All rights reserved.
RMAN>
RMAN> connect target *
connected to target database: ORAC (DBID=1058933594)
using target database control file instead of recovery catalog
RMAN> **end-of-file**

RMAN>
host command complete
RMAN> 2> 3> 4> 5> 6> 7>
allocated channel: dsk
channel dsk: sid=381 devtype=DISK
executing command: SET NOCFAU
Starting backup at 19-FEB-13
channel dsk: starting incremental level 0 datafile backupset
channel dsk: specifying datafile(s) in backupset
input datafile fno=00005 name=/oracle/ORAC/sapdata2/sr3db_2/sr3db.data2
channel dsk: starting piece 1 at 19-FEB-13
released channel: dsk
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on dsk channel at 02/19/2013 15:46:52
ORA-19504: failed to create file "/oracle/backup/ORAC/backup/bekozflm/sr3db.data2.S"
ORA-27054: NFS file system where the file is created or resides is not mounted with correct options
Additional information: 3
RMAN>

Recovery Manager complete.

BR0200I BR_TRACE: location BrRmanCall-44, commands for RMAN in: /oracle/ORAC/backup/.bekozflm.cmd
*** glibc detected *** /usr/sap/ORAC/SYS/exe/run/brbackup: free(): corrupted unsorted chunks: 0x0000000003dd4980 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3d0ca70f0f]
/lib64/libc.so.6(cfree+0x4b)[0x3d0ca7136b]
/lib64/libc.so.6(fclose+0x14b)[0x3d0ca5f95b]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x679172]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x69cc48]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x426a0f]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x44d8a5]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x3d0ca1d994]
/usr/sap/ORAC/SYS/exe/run/brbackup(xdr_string+0x7a)[0x405fda]

======= Memory map: ========
00400000-008c9000 r-xp 00000000 c7:2710 1675 /sapmnt/ORAC/exe/uc/linuxx86_64/brbackup
009c8000-009e0000 rw-p 004c8000 c7:2710 1675 /sapmnt/ORAC/exe/uc/linuxx86_64/brbackup
009e0000-00aa8000 rw-p 009e0000 00:00 0
03da5000-03ed0000 rw-p 03da5000 00:00 0 [heap]
3d0c600000-3d0c61c000 r-xp 00000000 fd:00 455243 /lib64/ld-2.5.so
3d0c81c000-3d0c81d000 r--p 0001c000 fd:00 455243 /lib64/ld-2.5.so
………….
3d0d616000-3d0d815000 ---p 00016000 fd:00 455376 /lib64/libpthread-2.5.so
3d0d815000-3d0d816000 r--p 00015000 fd:00 455376 /lib64/libpthread-2.5.so
………….
3d1cc00000-3d1cc0d000 r-xp 00000000 fd:00 455395 /lib64/libgcc_s-4.1.2-20080825.so.1
3d1cc0d000-3d1ce0d000 ---p 0000d000 fd:00 455395 /lib64/libgcc_s-4.1.2-20080825.so.1
……………
3d1f400000-3d1f4e6000 r-xp 00000000 fd:01 2192511 /usr/lib64/libstdc++.so.6.0.8
3d1f4e6000-3d1f6e5000 ---p 000e6000 fd:01 2192511 /usr/lib64/libstdc++.so.6.0.8
…………..
3d1f6ee000-3d1f700000 rw-p 3d1f6ee000 00:00 0
2b889d543000-2b889d545000 rw-p 2b889d543000 00:00 0
2b889d55b000-2b889d55c000 rw-p 2b889d55b000 00:00 0
2b889d55c000-2b889e78d000 r-xp 00000000 c7:2712 6022 /oracle/ORAC/102_64/lib/libclntsh.so.10.1
2b889e78d000-2b889e98c000 ---p 01231000 c7:2712 6022 /oracle/ORAC/102_64/lib/libclntsh.so.10.1
2b889e98c000-2b889ea30000 rw-p 01230000 c7:2712 6022 /oracle/ORAC/102_64/lib/libclntsh.so.10.1
2b889ea30000-2b889ea4d000 rw-p 2b889ea30000 00:00 0
2b889ea4d000-2b889ed3c000 r-xp 00000000 c7:2712 8126 /oracle/ORAC/102_64/lib/libnnz10.so
2b889ed3c000-2b889ee3b000 ---p 002ef000 c7:2712 8126 /oracle/ORAC/102_64/lib/libnnz10.so
2b889ee3b000-2b889eeec000 rw-p 002ee000 c7:2712 8126 /oracle/ORAC/102_64/lib/libnnz10.so
2b889eeec000-2b889eef2000 rw-p 2b889eeec000 00:00 0
2b889ef08000-2b889ef12000 r-xp 00000000 fd:00 455194 /lib64/libnss_files-2.5.so
2b889ef12000-2b889f111000 ---p 0000a000 fd:00 455194 /lib64/libnss_files-2.5.so
………….
2b889f136000-2b889f19a000 r--p 00000000 c7:2712 7254 /oracle/ORAC/102_64/oracore/zoneinfo/timezlrg.dat
2b889f19a000-2b889f19d000 rw-p 2b889f19a000 00:00 0
7fff6995b000-7fff6996f000 rwxp 7ffffffe9000 00:00 0 [stack]
7fff69970000-7fff69970000 rw-p 7fffffffe000 00:00 0
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
*** glibc detected *** /usr/sap/ORAC/SYS/exe/run/brbackup: malloc(): memory corruption: 0x00002b889f19b010 ***

======= Backtrace: =========
/lib64/libc.so.6[0x3d0ca71bae]
/lib64/libc.so.6(__libc_malloc+0x6e)[0x3d0ca738de]
/lib64/libc.so.6[0x3d0ca6030a]
/usr/sap/ORAC/SYS/exe/run/brbackup(BrSigHandle+0x108c)[0x6bb65c]
/lib64/libc.so.6[0x3d0ca302f0]
/lib64/libc.so.6(gsignal+0x35)[0x3d0ca30285]
/lib64/libc.so.6(abort+0x110)[0x3d0ca31d30]
/lib64/libc.so.6[0x3d0ca6944b]
/lib64/libc.so.6[0x3d0ca70f0f]
/lib64/libc.so.6(cfree+0x4b)[0x3d0ca7136b]
/lib64/libc.so.6(fclose+0x14b)[0x3d0ca5f95b]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x679172]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x69cc48]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x426a0f]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x44d8a5]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x3d0ca1d994]
/usr/sap/ORAC/SYS/exe/run/brbackup(xdr_string+0x7a)[0x405fda]
======= Memory map: ========
00400000-008c9000 r-xp 00000000 c7:2710 1675 /sapmnt/ORAC/exe/uc/linuxx86_64/brbackup
009c8000-009e0000 rw-p 004c8000 c7:2710 1675 /sapmnt/ORAC/exe/uc/linuxx86_64/brbackup
009e0000-00aa8000 rw-p 009e0000 00:00 0
03da5000-03ed0000 rw-p 03da5000 00:00 0 [heap]
2b889ef08000-2b889ef12000 r-xp 00000000 fd:00 455194 /lib64/libnss_files-2.5.so
2b889ef12000-2b889f111000 ---p 0000a000 fd:00 455194 /lib64/libnss_files-2.5.so
/oracle/ORAC/102_64/oracore/zoneinfo/timezlrg.dat
………….
2b889f19a000-2b889f19d000 rw-p 2b889f19a000 00:00 0
7fff6995b000-7fff6996f000 rwxp 7ffffffe9000 00:00 0 [stack]
7fff69970000-7fff69970000 rw-p 7fffffffe000 00:00 0
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
BR0280I BRBACKUP time stamp: 2013-02-19 15.46.53
BR0261E BRBACKUP cancelled by signal 6
BR0054I BRBACKUP terminated with errors
#######################################################
BR0292I Execution of BRBACKUP finished with return code 0
BR0280I BRTOOLS time stamp: 2013-02-19 15.46.53
BR0668I Warnings or errors occurred - you can continue to ignore them or go back to repeat the last action
BR0670I Enter 'c[ont]' to continue, 'b[ack]' to go back, 's[top]' to abort:
BR0280I BRTOOLS time stamp: 2013-02-19 15.46.56
BR0261E BRTOOLS cancelled by signal 2

BR0280I BRTOOLS time stamp: 2013-02-19 15.46.56
BR0654I BRTOOLS terminated with errors