Oracle keeps writing dumps to trace directory; fills up disk

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Article Type: General
Product: Aleph
Product Version: 20

Description:
The Production server started behaving oddly:

1. a 6-gig www_server log was produced within a 1-hour period. The log had 7 million occurrences of this line:

Load: /tmp/utf_files/exlibris/aleph/u20_1/abc01/tab/www_tab_short.eng.abcha

2. several 11-gig dumps were written in the /exlibris/app/oracle/diag/rdbms/aleph20/aleph20/trace directory within a period of 1 minute:

-rw-r----- 1 oracle dba 11779674112 Jun 11 07:40 aleph20_ora_6776.trc
-rw-r----- 1 oracle dba 373072 Jun 11 07:40 aleph20_ora_4812.trm
-rw-r----- 1 oracle dba 11779678208 Jun 11 07:40 aleph20_ora_4812.trc
-rw-r----- 1 oracle dba 340811 Jun 11 07:40 aleph20_ora_2923.trm
-rw-r----- 1 oracle dba 11780169728 Jun 11 07:40 aleph20_ora_2923.trc
-rw-r----- 1 oracle dba 16972 Jun 11 07:40 aleph20_ora_27316.trm
-rw-r----- 1 oracle dba 378638336 Jun 11 07:40 aleph20_ora_27316.trc
-rw-r----- 1 oracle dba 74180 Jun 11 07:40 aleph20_ora_22964.trm
-rw-r----- 1 oracle dba 1484906496 Jun 11 07:40 aleph20_ora_22964.trc
-rw-r----- 1 oracle dba 346401 Jun 11 07:40 aleph20_ora_1378.trm
-rw-r----- 1 oracle dba 11780173824 Jun 11 07:40 aleph20_ora_1378.trc

3. the /exlibris filesystem kept filling up since Oracle kept writing these dumps and the system was extremely slow. “top” showed io-wait at 50%.

4. the alert log indicated problems with certain archive log files. When archiving was turned off there was no longer any indication in the alert log of errors -- but the dumps continued to be written.

5. restarting Oracle did not help.

Resolution:
After the server was brought down (to check the disk -- no problem was found --) and brought back up -- in effect, rebooting the server -- everything was OK.

A util x/7 had been done prior to the time that this problem started occurring. KB 16384-18002 describes dangers in util x/7. The file which Aleph reloaded 7 million times was in utf_files. It's unclear if there may have been a connection.

Article last edited: 10/8/2013