Skip to main content
ExLibris

Knowledge Assistant

BETA
 
  • Subscribe by RSS
  • Back
    Primo

     

    Ex Libris Knowledge Center
    1. Search site
      Go back to previous article
      1. Sign in
        • Sign in
        • Forgot password
    1. Home
    2. Primo
    3. Knowledge Articles
    4. Error: Record too large. record size is: X max allowed record size: 314,572,800

    Error: Record too large. record size is: X max allowed record size: 314,572,800

    1. Last updated
    2. Save as PDF
    3. Share
      1. Share
      2. Tweet
      3. Share
    1. Problem Symptoms
    2. Cause
    3. Resolution
    4. Additional Information
    • Article Type: General
    • Product: Primo
    • Product Version: 4

    Problem Symptoms

    • The Primo pipe stopped with a pipe error. The Harvest Log shows this error:

    2012-03-13 12:21:13,737 ERROR [t-SplitRecords] [c-TarGZReader] - Record too large. record size is: 404,112,200 max allowed record size: 314,572,800

    • The problem file on the server is not the size described in the error (404,112,200). The file size is 23,929,419:

    primo exlibris 23929419 Mar 13 06:14 sfx_primo_export.xml-marc.tar.gz (22.82mb)

    Cause

    • Data extract file is too large to be handled by Primo's file splitter
    • The size of the raw file is the problem, NOT the zipped or gz'd file

    Resolution

    Split the single large file into smaller files using a 3rd-party tool.

    "split" is a common tool and may already be available on your server.

    1. Run the split command (counter, "aa" "ab" etc, will be appended to the newFilePrefix). Example:>> split -b 300000000 fromFile.mrc tonewFilePrefix

    This will create, for example:
    tonewFilePrefixaa
    tonewFilePrefixab

     

    1. Combine the multiple files into 1 package (such as, .tar or .tar.gz). Example:  >> tar -cvzf totargz.tar.gz tonewFilePrefix*

    This command will create a single tar-gziped file which Primo will be able to process:
    totargz.tar.gz

    Additional Information

    An alternative to splitting the files could be to change the Advanced Configuration > General Configuration Wizard > primo.process.MaxUncompressedFileSize (maximum file size). This will allow Primo's file splitter to handle a larger file. We do NOT recommend this solution for 2 reasons:

    1. File loading is done completely in the memory (RAM). This means that the Back Office must have sufficient memory allocated to process such the larger file.
    2. Your input file can be expected to continue growing and a size that works this month may no longer work next month. While setting up a file split solution may initially be more complicated, it is a more stable solution for the long term.

    This should not be a problem coming from Alma, Aleph, or Voyager, because those systems split files to handle this limitation.
    Files provided by other, external systems, such as other ILSs may have this problem, too.

    It is best if properly-sized files are generated by the source system, so that a record at the split point is not broken.
    If splitting is done to an

    • XML file: The split files will have to be corrected in order to be used, because valid XML requires valid structure and the split files will not both have valid opening and closing tags.
    • MarcExchange file or other non-XML files where the file structure is not an issue: A record at the split point will be lost if the split files are not corrected, but only a single record at each split point should be broken, not the entire file.

    • Article last edited: 08-Oct-2013
    View article in the Exlibris Knowledge Center
    1. Back to top
      • Error when scheduling Primo Analytics report in Primo VE
      • Errors in ARS mhs_msg.log: Failure retrieving title/author for item_id
    • Was this article helpful?

    Recommended articles

    1. Article type
      Topic
      Community Content Type
      How To
      Content Type
      Knowledge Article
      Language
      English
      Product
      Primo
    2. Tags
      1. 4
      2. contype:kba
      3. Interoperability
      4. Prod:Primo
      5. sfx
      6. Type:General
    1. © Copyright 2025 Ex Libris Knowledge Center
    2. Powered by CXone Expert ®
    • Term of Use
    • Privacy Policy
    • Contact Us
    2025 Ex Libris. All rights reserved