Question

What are the symptoms of keyword index problems?

Answer

• Keyword searching inconsistencies
  • Records not retrieved with keyword searches in WebVoyage or Cataloging
  • Same records retrieved with left anchored searches
  • Records retrieved after resaving records in Cataloging
    • Saving to the database generates the keyword data for that record.
• Slowness in adding records
  • Bulkimport
  • Global Data Change (GDC)
  • Cataloging (slow save time)
• Data Files on server
  • Note the 50% rule where the dynamic.dc file (dynamic data file) is over 50% of the size of the xxxdb.1.dc file (static data file) in the /m1/voyager/xxxdb/data directory on the Voyager server.
  • The 50% rule mentioned above is a general guideline that we use for determining when you need a keyword regen.
  • For smaller sized databases
    • The dynamic.dc file is 1-2 times the size of the static file
    • OR more important, you see actual symptoms that indicate a keyword regen, i.e., keyword searches failing, or extremely slow bulkimporting or GDC runs.

Additional Information

See Who can run a keyword regen? for details on scheduling/running one.

Why a large dynamic.dc file can slow keyword indexing (and processes using keyword indexing):

• With BulkImport (or GDC, which uses bulkimport), the dictionary words (keywords) get written to dynamic.que.
• A keysvr process, periodically moves the dictionary words from the dynamic.que to dynamic.dc.
• As the bulkimport or saving/updating in Cataloging continues, the dynamic.dc will continue to grow, but only for non-related words.
• The keysvr process searches through dynamic.dc file to determine if it should move a word to this file or not.
• What slows the import process is that as the dynamic.dc gets larger, the keysvr process has a greater volume to search through.
• As the file grows larger, the search grows longer.

• Article last edited: 10/8/2013