Skip to main content
ExLibris
  • Subscribe by RSS
  • Ex Libris Knowledge Center

    Oracle keeps writing dumps to trace directory; fills up disk

    • Article Type: General
    • Product: Aleph
    • Product Version: 20

    Description:
    The Production server started behaving oddly:

    1. a 6-gig www_server log was produced within a 1-hour period. The log had 7 million occurrences of this line:

    Load: /tmp/utf_files/exlibris/aleph/u20_1/abc01/tab/www_tab_short.eng.abcha

    2. several 11-gig dumps were written in the /exlibris/app/oracle/diag/rdbms/aleph20/aleph20/trace directory within a period of 1 minute:

    -rw-r----- 1 oracle dba 11779674112 Jun 11 07:40 aleph20_ora_6776.trc
    -rw-r----- 1 oracle dba 373072 Jun 11 07:40 aleph20_ora_4812.trm
    -rw-r----- 1 oracle dba 11779678208 Jun 11 07:40 aleph20_ora_4812.trc
    -rw-r----- 1 oracle dba 340811 Jun 11 07:40 aleph20_ora_2923.trm
    -rw-r----- 1 oracle dba 11780169728 Jun 11 07:40 aleph20_ora_2923.trc
    -rw-r----- 1 oracle dba 16972 Jun 11 07:40 aleph20_ora_27316.trm
    -rw-r----- 1 oracle dba 378638336 Jun 11 07:40 aleph20_ora_27316.trc
    -rw-r----- 1 oracle dba 74180 Jun 11 07:40 aleph20_ora_22964.trm
    -rw-r----- 1 oracle dba 1484906496 Jun 11 07:40 aleph20_ora_22964.trc
    -rw-r----- 1 oracle dba 346401 Jun 11 07:40 aleph20_ora_1378.trm
    -rw-r----- 1 oracle dba 11780173824 Jun 11 07:40 aleph20_ora_1378.trc

    3. the /exlibris filesystem kept filling up since Oracle kept writing these dumps and the system was extremely slow. “top” showed io-wait at 50%.

    4. the alert log indicated problems with certain archive log files. When archiving was turned off there was no longer any indication in the alert log of errors -- but the dumps continued to be written.

    5. restarting Oracle did not help.

    Resolution:
    After the server was brought down (to check the disk -- no problem was found --) and brought back up -- in effect, rebooting the server -- everything was OK.

    A util x/7 had been done prior to the time that this problem started occurring. KB 16384-18002 describes dangers in util x/7. The file which Aleph reloaded 7 million times was in utf_files. It's unclear if there may have been a connection.


    • Article last edited: 10/8/2013