RMAN Backup using BRTools failed with RMAN-03009, ORA-19504, ORA-27054 errors

Recently I have encountered below error when doing RMAN backup using BRTools. Make sure that you have mounted your backup mount point with below options.

Once you mount your backup mount point with below options you are able to run the backup successfully.

rw,bg,hard,rsize=32768,wsize=32768,vers=3,nointr,timeo=600,proto=tcp,suid

This solution is application for with or without BRTOOLS using RMAN.

BRTOOLS Error log file

BR0051I BRBACKUP 7.20 (0)
BR0055I Start of database backup: bekozflm.fnd 2013-02-19 15.46.46
BR0484I BRBACKUP log file: /oracle/ORAC/sapbackup/bekozflm.fnd
BR0477I Oracle pfile /oracle/ORAC/102_64/dbs/initORAC.ora created from spfile /oracle/ORAC/102_64/dbs/spfileORAC.ora

BR0280I BRBACKUP time stamp: 2013-02-19 15.46.47
BR0057I Backup of database: ORAC
BR0058I BRBACKUP action ID: bekozflm
BR0059I BRBACKUP function ID: fnd
BR0110I Backup mode: FULL
BR0077I Database file for backup: /oracle/ORAC/backup/cntrlORAC.dbf
BR0061I 6 files found for backup, total size 14242.117 MB
BR0143I Backup type: online
BR0112I Files will not be compressed
BR0130I Backup device type: disk
BR0106I Files will be saved on disk in directory: /oracle/backup/ORAC/backup/bekozflm

BR0280I BRBACKUP time stamp: 2013-02-19 15.46.47
BR0256I Enter 'c[ont]' to continue, 's[top]' to cancel BRBACKUP:c
BR0280I BRBACKUP time stamp: 2013-02-19 15.46.48
BR0257I Your reply: 'c'

BR0259I Program execution will be continued...

BR0370I Directory /oracle/backup/ORAC/backup/bekozflm created
BR0280I BRBACKUP time stamp: 2013-02-19 15.46.49
BR1390I RMAN prepared for backup
BR0202I Saving init_ora
BR0203I to /oracle/backup/ORAC/backup/ORAC ...
BR0202I Saving /oracle/ORAC/102_64/dbs/initORAC.sap
BR0203I to /oracle/backup/ORAC/backup/ORAC ...
BR0280I BRBACKUP time stamp: 2013-02-19 15.46.49
BR0198I Profiles saved successfully

BR0202I Saving /oracle/ORAC/sapdata2/sr3db_2/sr3db.data2
BR0203I to /oracle/backup/ORAC/backup/bekozflm/sr3db.data2.S ...
BR0278E Command output of 'SHELL=/bin/sh /oracle/ORAC/102_64/bin/rman nocatalog':

Recovery Manager: Release 10.2.0.4.0 - Production on Tue Feb 19 15:46:49 2013
Copyright (c) 1982, 2007, Oracle. All rights reserved.
RMAN>
RMAN> connect target *
connected to target database: ORAC (DBID=1058933594)
using target database control file instead of recovery catalog
RMAN> **end-of-file**

RMAN>
host command complete
RMAN> 2> 3> 4> 5> 6> 7>
allocated channel: dsk
channel dsk: sid=381 devtype=DISK
executing command: SET NOCFAU
Starting backup at 19-FEB-13
channel dsk: starting incremental level 0 datafile backupset
channel dsk: specifying datafile(s) in backupset
input datafile fno=00005 name=/oracle/ORAC/sapdata2/sr3db_2/sr3db.data2
channel dsk: starting piece 1 at 19-FEB-13
released channel: dsk
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on dsk channel at 02/19/2013 15:46:52
ORA-19504: failed to create file "/oracle/backup/ORAC/backup/bekozflm/sr3db.data2.S"
ORA-27054: NFS file system where the file is created or resides is not mounted with correct options
Additional information: 3
RMAN>

Recovery Manager complete.

BR0200I BR_TRACE: location BrRmanCall-44, commands for RMAN in: /oracle/ORAC/backup/.bekozflm.cmd
*** glibc detected *** /usr/sap/ORAC/SYS/exe/run/brbackup: free(): corrupted unsorted chunks: 0x0000000003dd4980 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3d0ca70f0f]
/lib64/libc.so.6(cfree+0x4b)[0x3d0ca7136b]
/lib64/libc.so.6(fclose+0x14b)[0x3d0ca5f95b]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x679172]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x69cc48]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x426a0f]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x44d8a5]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x3d0ca1d994]
/usr/sap/ORAC/SYS/exe/run/brbackup(xdr_string+0x7a)[0x405fda]

======= Memory map: ========
00400000-008c9000 r-xp 00000000 c7:2710 1675 /sapmnt/ORAC/exe/uc/linuxx86_64/brbackup
009c8000-009e0000 rw-p 004c8000 c7:2710 1675 /sapmnt/ORAC/exe/uc/linuxx86_64/brbackup
009e0000-00aa8000 rw-p 009e0000 00:00 0
03da5000-03ed0000 rw-p 03da5000 00:00 0 [heap]
3d0c600000-3d0c61c000 r-xp 00000000 fd:00 455243 /lib64/ld-2.5.so
3d0c81c000-3d0c81d000 r--p 0001c000 fd:00 455243 /lib64/ld-2.5.so
………….
3d0d616000-3d0d815000 ---p 00016000 fd:00 455376 /lib64/libpthread-2.5.so
3d0d815000-3d0d816000 r--p 00015000 fd:00 455376 /lib64/libpthread-2.5.so
………….
3d1cc00000-3d1cc0d000 r-xp 00000000 fd:00 455395 /lib64/libgcc_s-4.1.2-20080825.so.1
3d1cc0d000-3d1ce0d000 ---p 0000d000 fd:00 455395 /lib64/libgcc_s-4.1.2-20080825.so.1
……………
3d1f400000-3d1f4e6000 r-xp 00000000 fd:01 2192511 /usr/lib64/libstdc++.so.6.0.8
3d1f4e6000-3d1f6e5000 ---p 000e6000 fd:01 2192511 /usr/lib64/libstdc++.so.6.0.8
…………..
3d1f6ee000-3d1f700000 rw-p 3d1f6ee000 00:00 0
2b889d543000-2b889d545000 rw-p 2b889d543000 00:00 0
2b889d55b000-2b889d55c000 rw-p 2b889d55b000 00:00 0
2b889d55c000-2b889e78d000 r-xp 00000000 c7:2712 6022 /oracle/ORAC/102_64/lib/libclntsh.so.10.1
2b889e78d000-2b889e98c000 ---p 01231000 c7:2712 6022 /oracle/ORAC/102_64/lib/libclntsh.so.10.1
2b889e98c000-2b889ea30000 rw-p 01230000 c7:2712 6022 /oracle/ORAC/102_64/lib/libclntsh.so.10.1
2b889ea30000-2b889ea4d000 rw-p 2b889ea30000 00:00 0
2b889ea4d000-2b889ed3c000 r-xp 00000000 c7:2712 8126 /oracle/ORAC/102_64/lib/libnnz10.so
2b889ed3c000-2b889ee3b000 ---p 002ef000 c7:2712 8126 /oracle/ORAC/102_64/lib/libnnz10.so
2b889ee3b000-2b889eeec000 rw-p 002ee000 c7:2712 8126 /oracle/ORAC/102_64/lib/libnnz10.so
2b889eeec000-2b889eef2000 rw-p 2b889eeec000 00:00 0
2b889ef08000-2b889ef12000 r-xp 00000000 fd:00 455194 /lib64/libnss_files-2.5.so
2b889ef12000-2b889f111000 ---p 0000a000 fd:00 455194 /lib64/libnss_files-2.5.so
………….
2b889f136000-2b889f19a000 r--p 00000000 c7:2712 7254 /oracle/ORAC/102_64/oracore/zoneinfo/timezlrg.dat
2b889f19a000-2b889f19d000 rw-p 2b889f19a000 00:00 0
7fff6995b000-7fff6996f000 rwxp 7ffffffe9000 00:00 0 [stack]
7fff69970000-7fff69970000 rw-p 7fffffffe000 00:00 0
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
*** glibc detected *** /usr/sap/ORAC/SYS/exe/run/brbackup: malloc(): memory corruption: 0x00002b889f19b010 ***

======= Backtrace: =========
/lib64/libc.so.6[0x3d0ca71bae]
/lib64/libc.so.6(__libc_malloc+0x6e)[0x3d0ca738de]
/lib64/libc.so.6[0x3d0ca6030a]
/usr/sap/ORAC/SYS/exe/run/brbackup(BrSigHandle+0x108c)[0x6bb65c]
/lib64/libc.so.6[0x3d0ca302f0]
/lib64/libc.so.6(gsignal+0x35)[0x3d0ca30285]
/lib64/libc.so.6(abort+0x110)[0x3d0ca31d30]
/lib64/libc.so.6[0x3d0ca6944b]
/lib64/libc.so.6[0x3d0ca70f0f]
/lib64/libc.so.6(cfree+0x4b)[0x3d0ca7136b]
/lib64/libc.so.6(fclose+0x14b)[0x3d0ca5f95b]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x679172]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x69cc48]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x426a0f]
/usr/sap/ORAC/SYS/exe/run/brbackup[0x44d8a5]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x3d0ca1d994]
/usr/sap/ORAC/SYS/exe/run/brbackup(xdr_string+0x7a)[0x405fda]
======= Memory map: ========
00400000-008c9000 r-xp 00000000 c7:2710 1675 /sapmnt/ORAC/exe/uc/linuxx86_64/brbackup
009c8000-009e0000 rw-p 004c8000 c7:2710 1675 /sapmnt/ORAC/exe/uc/linuxx86_64/brbackup
009e0000-00aa8000 rw-p 009e0000 00:00 0
03da5000-03ed0000 rw-p 03da5000 00:00 0 [heap]
2b889ef08000-2b889ef12000 r-xp 00000000 fd:00 455194 /lib64/libnss_files-2.5.so
2b889ef12000-2b889f111000 ---p 0000a000 fd:00 455194 /lib64/libnss_files-2.5.so
/oracle/ORAC/102_64/oracore/zoneinfo/timezlrg.dat
………….
2b889f19a000-2b889f19d000 rw-p 2b889f19a000 00:00 0
7fff6995b000-7fff6996f000 rwxp 7ffffffe9000 00:00 0 [stack]
7fff69970000-7fff69970000 rw-p 7fffffffe000 00:00 0
ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]
BR0280I BRBACKUP time stamp: 2013-02-19 15.46.53
BR0261E BRBACKUP cancelled by signal 6
BR0054I BRBACKUP terminated with errors
#######################################################
BR0292I Execution of BRBACKUP finished with return code 0
BR0280I BRTOOLS time stamp: 2013-02-19 15.46.53
BR0668I Warnings or errors occurred - you can continue to ignore them or go back to repeat the last action
BR0670I Enter 'c[ont]' to continue, 'b[ack]' to go back, 's[top]' to abort:
BR0280I BRTOOLS time stamp: 2013-02-19 15.46.56
BR0261E BRTOOLS cancelled by signal 2

BR0280I BRTOOLS time stamp: 2013-02-19 15.46.56
BR0654I BRTOOLS terminated with errors