Skip to main content
ExLibris
  • Subscribe by RSS
  • Ex Libris Knowledge Center

    Working with Normalization Rules

    Translatable
    To work with normalization rules, you must have the following role:
    • Catalog Administrator
    Normalization rules provide the building blocks for making enhancements to MARC 21 records. These enhancements can be made to individual records in the MD Editor (Edit > Enhance the Record) or to a group of records using sets.
    In order to process record enhancements to an individual record, use the Enhance Record option (refer to the section MD Editor Menu and Toolbar Options and Enhance the Record) or Apply Changes to an individual record when you use the Preview normalization rule capability (see the procedure To preview the outcome of a rule file).
    In order to process record enhancements to a set of records, you need to create a process using the MARC Drool Normalization process (see Working with Normalization Processes) and specify the normalization rule that you create with the MD Editor (refer to the procedure To create a new normalization rule file). Once you have created the process, you can, subsequently, run a job using that process (see Running Manual Jobs on Defined Sets).
    Normalization rules are created by following a specific programming syntax and using the editing window provided in the MD Editor under the Rules tab.
    Rules Tab Editing Window
    In addition to creating your own original normalization rules program, you can also copy/paste existing rules into the editing window or use out-of-the-box examples (Rules tab > Edit > Add Rule) to develop your normalization rules. See Normalization Rules Syntax for more information regarding the syntax for normalization rules and examples that you can copy to the editing window.
    Using the preview functionality in the MD Editor, you can:
    • View the normalization rules and metadata records side by side
    • Preview the outcome of a rule file when run on a metadata record
    • Toggle between the rule file and the preview changes
    • Edit rules and test immediately
    This section describes how to create a normalization rule file, work with previously created rule files, and preview the outcome of a rule file when it is run on a single metadata record.
    For additional information on normalization rules, see Normalization Rules and Processes.
    For a video on Normalization rules, see Normalization Rules (41:07 mins). For a detailed Ask the Expert session on configuring normalization rules, see Normalization Rules.
    To create a new normalization rule file:
    1. On the MD Editor page (Resource Management > Cataloging > Open Metadata Editor), select File > New > Normalization Rule. The Normalization Rules Properties dialog box opens.
      Normalization Rules Properties Dialog Box
    2. Enter a name (required) and a description for the normalization rule file.
    3. Select an access option, Private or Shared. If you select the former, only you can work on the rule and the rule cannot be included in a normalization process. If you select the latter, your rule will be shared among catalogers. In this case, more than one user can view the rule at the same time, and if two or more people have the rule open for editing, a warning message appears when one of you tries to save changes. (You have the option of keeping your changes or allowing the other user to make and save changes.)
    4. To enable the rule, click the Enabled check box. (By default, the normalization rule file you create is disabled.)
    5. Click Save. A text box, in which you enter the rule or rules, opens.
      You can include existing rule syntax (Edit > Add Rule > {type of rule}) or define a rule (for details, see Normalization Rules Syntax).
      The following is an example of a normalization rule:
      Normalization Rule Example
    6. Click Save. The rule file is added to the list of rule files in the Normalization Rules tab.
    To work with an existing normalization rule file:
    1. On the MD Editor page (Resource Management > Cataloging > Open Metadata Editor), click the Rules tab and expand the Normalization rules folder to display the saved rule files.
      Rules Tab
    2. Click the rule file with which you want to work and select one of the following options:
      • Edit – Opens the text box with the rule(s) syntax, enabling you to modify this syntax (for details, see Normalization Rules Syntax).
      • Delete – Click Yes to confirm the rule file’s deletion.
      • Duplicate – Duplicates the selected rule file, enabling you to modify and save it as a new rule file without affecting the original file.
      • Properties – Opens the Normalization Rules Properties dialog box, enabling you to modify the properties of the rule file.
    To preview the outcome of a rule file:
    1. Locate the bibliographic record with which you want to work (using the Repository Search or within the MD Editor Records tab) and open it in the MD Editor.
    2. Select Edit > Split Editor (F6) or click Split Editor.
    3. Click the Rules tab, expand the Normalization rules > Shared list to display the saved rule files, select the rule file whose outcome you want to preview, and click Edit.
      Click Edit to Open the Normalization Rule
      The rule file is displayed in the right pane of the MD Editor.
      Normalization Rule – Preview
    4. Click Preview. The rule or rules in the file are applied to the record and the outcome is displayed.
    5. Click Apply Changes to save the modifications to the record or click Back to normalization rules.
    If you return back to your normalization rule, you can make additional changes to your normalization rule and, iteratively, click Preview and view your changes. When you have made your final changes to the normalization rule, click Save (next to the Preview button) to save the final version or your normalization rule.

    Normalization Rule Syntax

    Rule files contain one or more rules, which contain a condition and one or more actions to be applied to records. Actions are applied to a record if the record meets the condition. Each action within a rule can be performed on a single field within a record. Actions are performed in the order in which they appear within the rule.
    Learn more about creating normalization rules in the Normalization Rules video (41:07 mins).
    Learn how to create normalization rules that delete specified fields from records, or change the contents of these fields in the Normalization Routine Syntax for Deletion or Content Change video (9:57 mins).
    When
    (<conditions on MARC record>) then
    Action1
    Action2 if condition
    Action3
    End
    <conditions on MARC record> contains one or more Boolean clauses that apply to the record. If <conditions on MARC record> returns TRUE, the rule is applied to the record; otherwise, the rule is not applied and the record is not processed.
    “When” must be the only word in the first line. The condition must be placed on a separate line.

    Record Elements

    Conditions and actions apply to record elements, such as the MARC record, fields (one or more), indicators, subfields (one or more), and field/subfield contents.
    To test a condition or apply an action to a record element, the element must match the following syntax:
    Syntax
    Expression Meaning
    "<tag>", "<new tag>" Represents a field tag, for example, 001, 245, etc.
    "<oldCode>", "<newCode>" Represents a subfield code, for example, a, b, c.
    "<element>" for a data field The following are the possible values for the data field:
    • FIELD– for example: 245
    • FIELD_VALUE – for example: 245.value*
    • FIELD_INDICATOR – for example: 245.{1,2}
    • FIELD_SUBFIELD_CODE – for example: 245.a
    • FIELD_INDICATOR_SUBFIELD_CODE – for example: 245.{1,2}.a
    • FIELD_SUBFIELD_CODE_VALUE – for example: 245.a.value*
    • FIELD_INDICATOR_SUBFIELD_CODE_VALUE – for example: 245.{1,2}.a.value*
    "<element>" for a control field The following are the possible values for a control field:
    • FIELD_POSITION_LENGTH – for example: LDR.{17,3}
    • FIELD_POSITION_LENGTH_VALUE – for example: LDR.{17,3}.eng
    CONDITION at record level The following are the possible condition options:
    • TRUE
    • not exists "{element}"
    • not existsControl "{element}"
    • exists "{element}"
    • existsControl "{element}"
    • existsMoreThanOnce "{element}" (see Managing Search Queries and Sets for an example of the use of this condition)
    • contains (for merge rules)

    Conditions

    Conditions can be defined at the entire rule level (WHEN), or at a specific action level (IF). The same condition will behave differently depending on the level at which it is defined.
    • WHEN clause – A condition that must be met by the entire record in order to determine whether the rule is applied to the record
    • IF (within an action) – A condition that applies to a single field in order to determine whether the specific action is taken on that field
    Conditions can be:
    • exists <element> – at least one match is found
      • exists <element> – applies to data fields
      • existsControl <element> – applies to control fields
    • not exists <element> – no match is found
      • not exists <element> – applies to data fields
      • not existsControl <element> – applies to control fields
    Each IF clause action can have one of the following formats:
    • Applies if a specific condition is not true, for example: addcontrolField "{element}" if(not exists "{condition}")
    • Applies if a specific condition is true, for example: addcontrolField "{element}" if(exists "{condition}")
    • Applies unconditionally, for example: addcontrolField "{element}"
    The Boolean OR operator can be used in a consequence statement by using the pipe (|) symbol, for example: removefField "866" if (not exists "866.8.0|99")
    If the pipe symbol is part of the value, use four backslashes (\\\\) to escape it, for example: removeField "866" if (exists "866.8.0\\\\|99")

    List of Actions

    The following table provides a list of available actions.
    List of Actions
    Action Condition Comment
    Replace fields and subfields with other fields and subfields changeControlField "<tag>" to "new tag"
    Example: changeControlField "007" to "008"
    Changes the tag identifier of a control field; does not modify contents.
    changeField "<tag>" to "new tag"
    Example: changeField "245" to "246"
    Changes the tag identifier; does not modify indicators or subfields.
    changeSubField "<tag>.<code>" to "new code"
    changeSubFieldOnlyFirst "<tag>.<code>" to "new code"
    changeSubFieldExceptFirst "<tag>.<code>" to "new code"
    Example: changeSubField "035.b" to "a"
    Changes the subfields (or only the first subfield, or all except the first subfield) "<code>" to the subfield "new code" in field "<tag>".
    changeFirstIndicator "<tag>" to "<value">
    changeSecondIndicator "<tag>" to "<value">
    Example: changeFirstIndicator "245" to "3"
    Sets the value of the specified indicator in tag <tag>. If the indicator has a value, it will be changed.
    combineFields "{tag}" excluding "{comma-separated subfield list}"
    Example: combineFields "852" excluding "a,b"
    Combine all fields of the specified number. Copy all subfields from the second and subsequent lines to the first line, excluding the named subfields; only the first occurrences of excluded subfields are copied, and only if they do not already exist in the first line.
    Add fields and subfields addField "<tag>.<code>. <value>"
    addField "<tag>.{<ind1>,<ind2>}.<code>. <value>"
    Example: addField "999.a.RESTRICTED"
    Adds the field to the MARC record. Sets the value of the subfield to the indicated value.
    addControlField "<tag>"."<value>"
    Example: addControlField "008.820305s1991####nyu###########001#0#eng##"
    Adds the control field to the MARC record. Sets the value of the subfield to the indicated value.
    addSubField "<tag>.<code>.<value>"
    addSubField "<tag>.{<ind1>,<ind2>}.<code>.<value>"
    Example: addSubField "245.h.[Journal]"
    Adds the subfield <code> with value <value> to field <tag>. If the field does not exist, nothing is done.
    addSystemNumber "<element>" from "<tag>" prefixed by "<prefix tag>"
    Example: addSystemNumber "035.a" from "001" prefixed by "003"
    Makes the data field <element> equal to the contents of the second control field <prefix tag> in parentheses followed by the contents of the first control field <tag>.
    For example, if 001 has the value 9945110100121 and 003 has the value DAV, the example condition on the left will produce 035 with the value ‡(DAV)9945110100121.
    Copies fields copyField "<tag>" to "<tag>"
    copyField "<tag>.<code>" to "<tag>.<code>"
    copyField "<tag>" to "<tag>.{<ind1>,<ind2>}"
    Example: copyField "971.a" to "100.u"
    Copies the field to another field. If the subfield <code> is not specified, the new subfield is the same as the old subfield.
    copyField creates a separate field rather than adding it to any existing field. You may want to combine the new field with any existing fields (see combineFields).
    Remove fields and subfields removeControlField "<tag>"
    Example: removeControlField "009"
    Removes the control field.
    removeField "<tag>"
    Example: removeField "880"
    Removes all occurrences of the field <tag>.
    removeSubField "<tag>.<code>"
    Example: removeSubField "245.h"
    Removes the subfield <code> from the indicated field.
    Replace text in fields or subfields replaceControlContents "<tag>.{<position>,<length>}.
    <value>" with "new value"
    Example: replaceControlContents "LDR.{7,1}.s" with "m"
    Replaces <value> with "new value" in starting position <position> to <position>+<length> of control field <tag>. Replaces only the text that matches <value>.
    replaceContents "<tag>.<code>.<value>" with "new value"
    replaceContentsOnlyFirst "<tag>.<code>.<value>" with "new value"
    replaceContentsExceptFirst "<tag>.<code>.<value>" with "new value"
    Example: replaceContents "245.h.[Journal]" with "[Book]"
    Replaces the matching strings (or only the first matching string, or all strings except the first matching string) <value> in the subfield <code> of field "<tag>" with "new value". The string or part of the string that does not match <value> is not modified.
    replaceSubFieldContents "<tag>.<code>" with "<tag>.<code>"
    Example: replaceSubFieldContents "245.b" with "100.a"
    Replaces the subfield's contents with the contents of another subfield.
    Add text in subfields prefix "<tag>.<code>" with "<value>"
    Example: prefix "035.b" with "(OCoLC)"
    Adds a prefix to the value of subfield "<code>" in the field "<tag>".
    prefixSubField "<tag>.<code>" with "<value>"
    Example: prefixSubField "910.a" with "906.a"
    Adds a prefix to the subfield "<code>" in the field "<tag>".
    suffix "<tag>.<code>" with "<value>"
    Example: suffix "035.b" with "(OCoLC)"
    Adds a suffix to the value of subfield "<code>" in the field "<tag>".
    suffixSubField "<tag>.<code>" with "<value>"
    Example: suffixSubField "910.a" with "907.c"
    Adds a suffix to the subfield "<code>" in the field "<tag>".
    Maintain agency information in bibliographic and authority records
    For example, this syntax can be used in normalization rules that are selected in the MARC 21 Bibliographic Metadata Configuration Task List to normalize Network Zone bibliographic records upon save.
    This functionality is under construction. To enable this syntax, contact Ex Libris Support.
    addCreatingAgency "<tag>.<code>"
    Example: addCreatingAgency "040.a"
    Adds the creating agency ISIL code to the subfield in "<code>" in the field "<tag>".
    addModifyingAgency "<tag>.<code>"
    Example: addModifyingAgency "040.d"
    Adds the modifying agency ISIL code to the subfield in "<code>" in the field "<tag>". If there already is a modifying agency in the "<tag>.<code>", this adds another agency ISIL code.
    replaceModifyingAgency "<tag>.<code>"
    Example: replaceModifyingAgency "040.d"
    Adds the modifying agency ISIL code to the subfield in "<code>" in the field "<tag>". If any modifying agencies already exist in the "<tag>.<code>", all of them are replaced.

    Wildcards and Special Characters

    The asterisk (*) is used to match any string. For example, "<tag>.<*>.<value>" applies to all the subfields in the tag <tag> that have the value <value>. * is "greedy", so it matches as many characters as possible in the string. For example: if you have a string "a b c b d b e", the pattern "b*b" matches "b c b d b", not just "b c b".
    Empty indicators (but not fields or subfields) are indicated by a dash (-). For example, "<tag> {-,<ind2>}" returns all the fields where the MARC tag is <tag>, the first indicator is not defined, and the second indicator is <ind2>.
    If the text of a subfield contains a period, use four backslashes to match the period. For example:
    rule "Replace 1 v. to Leaves $$a (unconditional)"
    when
    (TRUE)
    then
    replaceContents "3000.a.1 v\\\\." with "Leaves"
    end
    Double quotation marks can be used in conditions (only). To use double quotation marks as part of a condition, use single quotation marks to enclose text in the rule (') instead of double quotation marks ("). This way you can use double quotation marks inside the text following a double slash (//). See the following example.
    Hebrew dates can be used in conditions (only). See the following example.
    rule "populate 008 7-10 2016"
    when
    ((exists '260.{*, }.c.תשע\\"ו') OR (exists '264.{*, }.c.תשע\\"ו'))
    then
    replaceControlContents "008.{7,4}" with "2016"
    end

    Example: Using the Period in a Normalization Rule with replaceContents

    When a period is used in a normalization rule with the replaceContents parameter, it should be preceded with four backslashes. If you have a record with periods as in the following example and you want to replace them with nothing (that is, remove the periods), then your normalization rule should have backslashes as shown in the normalization rule below.
    Example record with periods:
    245 00 $$a Feminist literary theory. : $$b a reader / $$c edited by Mary Eagleton.
    246 0# $$a F.L.T.
    Normalization rule for the example record above:
    rule "remove the periods in 245 and 246 subfield a (and replace periods with nothing); precede period with four backslashes"
    when
    (TRUE)
    then
    replaceContents "245.a.\\\\." with ""
    replaceContents "246.a.\\\\." with ""
    end
    See the figure below for the before and after examples.
    Before and After Examples

    Example: Using the Period in a Normalization Rule with addField

    When a period is used in a normalization rule with the addField parameter, it should be preceded with four backslashes.
    When the period is not immediately followed by another character, it must have the four backslashes preceding it in the normalization rule.
    When the period is immediately followed by another character, it does not require the preceding four backslashes (as in addField "907.a.F.L.T\\\\.").
    Both of these methods work. However, it is considered best practice to always use the four backslashes in the normalization rule to ensure the most consistent desired results.
    Below is an example of a record to which periods need to be added:
    906 $$a Architecture.
    907 $$a F.L.T.
    Below is the normalization rule for the example record above using the best practice of always including slashes:
    rule "Add field 906 with text Architecture and period at end and also add field 907 with F.L.T."
    salience 100
    when
    TRUE
    then
    addField "906.a.Architecture\\\\."
    addField "907.a.F\\\\.L\\\\.T\\\\."
    end
    See the figure below for the before and after record examples.
    Before and After Examples

    Normalization Rule Examples

    For a list of over 50 normalization rule examples and other normalization rule documents, see Normalization Rules.