Skip to main content
  • Subscribe by RSS
  • Ex Libris Knowledge Center

    Splitting a large file of MARC records into smaller files

    • Article Type: General
    • Product: Aleph
    • Product Version: 16.02

    I am working with a set of 136,000 MARC records. When the MARC records are converted to ALEPH sequential format the resulting file is too large to view with vi or other tools.

    Is there a way to break these files into smaller pieces?

    There are two ways to do this:

    (1) Use the UNIX "split" command on the output of file-01 ("bigfile01"):
    >> split -l 20000 bigfile01 splitfile

    This will generate 7 files:
    splitfileaa 00001-20000
    splitfileab 20001-40000
    splitfileac 40001-60000
    splitfilead 60001-80000
    splitfileae 80001-100000
    splitfileaf 100001-120000
    splitfileag 120001-136000

    Then run each of these files through p_file_02.


    (2) Once a file has been run through p_file_02, the lines consist of individual fields rather than records.

    The grep qualifier "^" indicates that the string must be at the beginning of the line.

    You can break the output of p_file_02 (bigfile02) into 1,000-record pieces like this:

    >> grep ^000000 bigfile02 > file000000 - this will give you records 000000001 - 000000999 .
    >> grep ^000001 bigfile02 > file000001 - this will give you records 000001000 - 000001999
    >> grep ^000002 bigfile02 > file000002 - this will give you records 000002000 - 000002999 .

    (Keywords: file-01 file_01 file-02 file_02)

    • Article last edited: 12/1/2016