Skip to main content
ExLibris
  • Subscribe by RSS
  • Ex Libris Knowledge Center

    Large queue of z07's preventing indexing of GUI updates

    • Article Type: General
    • Product: Aleph
    • Product Version: 20

    Description:
    We are having problem with indexing. This problem has been existing since last week.
    The changes made to call number, title and system number indexes are not showing.
    I just looked at the Z07 table and found there were a huge list of entries and wonder if that is causing the problem.
    Is there anyway to clear up the table?

    Resolution:
    Articles 000037665 ("Diagnosing slow bib record indexing") and 000033440 ("New records are not going to the head of the z07 indexing queue") (links below) are relevant. Article 000045123 (" Why are Thousands of Z07s Being Generated?") (link below) is a general discussion of this issue.

    Examination of the z07's showed that the problem was 31,000 "READEX" e-records loaded via p_manage_18.

    SQL shows this:

    abc01@PALEPH> select substr(z07_sequence,1,4), count(*) from z07 group by substr(z07_sequence,1,4);
    **** Hit return to continue ****

    SUBSTR(Z07_S COUNT(*)
    ------------ ----------
    1998 6457
    2009 38459
    1990 28435

    As described in KB 8192-7463:

    the "2009"s are generated by ue_08 and are not an issue here;

    the "1998"s are GUI- and oclc_server-generated updates;

    the "1990"s are new records, regardless of source (GUI new, OCLC new, and READEX batch-loaded).


    We need to lower the priority of the non-GUI/OCLC-generated (that is, the READEX) requests.

    Another part of the problem is that ue_01 is taking a minute or more to process each READEX record (from the current $data_scratch/run_e_01 log). This issue of slowness in processing e-records is described in Article 000015047 (ue_01 slow after load of "ebrary" records).

    But I think that, if we can lower the z07 priority of these e-records, that should solve the most critical problem.

    Based on your ranges of READEX records, I suggest running the following SQL in ABC01:

    update z07 set z07_sequence = '1999' || substr(z07_sequence,5,11) where z07_sequence like '1990%' and z07_rec_key >= '003000710' and z07_rec_key <= '003031737';


    [Site replied that after doing this, the indexing speed improved but online updates were still not being indexed.]

    There are still all the other GUI and OCLC updates done in the past week which need to be processed. The updates done today are queued behind these other ones.

    abc01@PALEPH> select count(*) from z07 where z07_sequence like '1990%';
    **** Hit return to continue ****

    COUNT(*)
    ----------
    1609

    Since ue_01 is now doing about one record every two seconds, it will take it about one hour to process these -- after which, it will be caught up and any online new records should be indexed immediately.

    It will then go on to process the 6,553 GUI/OCLC updates which are waiting to be processed:

    abc01@PALEPH> select count(*) from z07 where z07_sequence like '1998%';
    **** Hit return to continue ****

    COUNT(*)
    ----------
    6553

    Online updates done today will be queued behind these 6,553. It will take about 4 hours for ue_01 to process these -- after which, it will be caught up and any online updates to existing records should be indexed immediately.

    Additional Information

    Article link:  Diagnosing slow bib record indexing; ue_01 stages **MASTER RECORD** 
    Article link:  New records are not going to the head of the z07 indexing queue
    Article link:  Why are Thousands of Z07s Being Generated? *COMPREHENSIVE RECORD*

    See also Article 000034004,  p_manage_18: override indexing priority .
     

    Category: Cataloging


    • Article last edited: 2/12/2015