Configuring Normalization Rules for External Resources (Primo VE)

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Last updated: March 6, 2021

Normalization rules define transformations that are applied to external source records (DC or generic XML) to create the Primo VE expanded Dublin Core records for discovery. You can create your own normalization rules and apply them to externally loaded records.

Primo VE Administration Certification > 06 External Resources: Loading Data (15 min)

For information on how to create normalization rules for MARC21 records from external sources, see Working with Normalization Rules.

Using the Expanded DC Schema

The Primo VE Dublin Core expanded schema includes additional discovery fields that you can populate as needed using normalization rules. The following fields are supported by the expanded DC schema:

discovery.resourceType – These fields hold the resource types that display in Primo, which may be different from the values in the dc:type and dcterms:type fields that are loaded from the external records. For more information, see Mapping Resource Types from External Data Sources.
discovery.local1 thru discovery.local50 – These fields hold additional information that you can use to display additional information from the source and also to use as a search index. For more information, see Mapping to Local Discovery Fields.

Mapping Resource Types from External Data Sources

Because the values for Dublin Core resource types can be anything and may not match the supported codes used in Primo VE, it may be necessary to map the resource type from the source record's dc:type or dcterms:type field to either a supported resource type code in Primo VE (see Supported Resource Types in Primo VE) or a local resource type defined in Primo VE (see Configuring Local Resource Types for Primo VE).

You can use any of the following methods to map the resource types to Primo VE:

Create normalization rules that replace the value of the relevant dc:type or dcterms:type field.
Create normalization rules that map a resource type value to the the discovery:resourceType field and leave the original DC field as it is.
Add resource type mappings to the Dublin Core Type to Discovery Type Mapping mapping table. For more information, see Using the Mapping Table to Map Resource Types.

Primo VE determines a record's resource type based on the existence of the following fields in order:

discovery:resourceType field
dcterms:type field
dc:type field

If processing continues to items 2 and 3, the system compares the value of the item with the mappings in the Dublin Core Type to Discovery Type Mapping mapping table. If a match is found, the matching value in the mapping table is used and stored in the discovery:resourceType field. If no match is found and the mapping table has a default value, the default value will be used. Otherwise, the system will use the value of the DC field as is.

Examples

For more information about creating normalization rules, see Creating Normalization Rules.

The following example maps the Proceedings resource type in dc:type to a supported code in Primo VE.

You can map several resource types in the source (such as Poster and Photograph) to the same resource type (such as image) in Primo VE.

rule "from dc:type A to discovery:type B"

when

"dc"."type" equals "Proceedings"

then

set "conference_proceedings" in "discovery"."resourceType"

end

The following example maps the LibGuides resource type in dcterms:type to a new local resource type that you must define in Primo VE.

If the local resource type had been defined as LibGuides in Primo VE, the following normalization rule would not be necessary, and the system would just use the resource type that was mapped to dcterms:type in Primo VE.

rule "from dcterms:type A to discovery:type B"

when

"dcterms"."type" equals "LibGuides"

then

set "library_guides" in "discovery"."resourceType"

end

Supported Resource Types in Primo VE

The table below lists the resource type codes that are supported by Primo VE. You will need to map your source's values to one of the supported codes or a local resource type (if needed). For more information about creating local resource types, see Configuring Local Resource Types for Primo VE.

Supported Resource Types in Primo VE
Resource Type Code	Display Name
databases	Database
audios	Audio
newspapers	Newspaper
manuscripts	Manuscript
conference_proceedings	Conference Proceeding
dissertations	Dissertation
kits	Kit
other	Other
archival_materials	Archival Material
realia	Realia
books	Book
book_chapters	Book Chapter
collections	Collection
legal_documents	Legal Document
patents	Patent
reference_entrys	Reference Entry
research_datasets	Research Dataset
reviews	Review
statistical_data_sets	Statistical Data Set
technical_reports	Technical Report
journals	Journal
newspaper_articles	Newspaper Article
articles	Article
text_resources	Text Resource
government_documents	Government Document
images	Image
maps	Map
videos	Video
scores	Score
websites	Website

Mapping to Local Discovery Fields

The Dublin Core expanded schema for Primo VE allows you to map information to the following fields, which are populated during the loading of records: discovery.local1 - discovery.local50. After the records have been loaded, Primo VE can use the local fields to display additional information in a record's brief and full displays and also to use them as search fields. Local fields are particularly useful to map non-standard DC fields that are not loaded by default during the import to Primo VE.

When using a local discovery field, you must also configure its similarly named local display field. For example, if you have created a normalization rule for discovery.local2, you must also define the local display field Local_field_02, making sure that you select the Use the parallel Local Field 01/50 from the Dublin Core record check box. For more information, see Configuring Local Display and Search Fields for Primo VE.

Examples

For more information about creating normalization rules, see Creating Normalization Rules.

The following example maps the value in dc:relation to local field 1 in Primo VE.

rule "move relation to local field 1"

when

exist "dc"."relation"

then

move "dc"."relation" to "discovery"."local1"

end

The following example sets the value of local field 2 to Internal Research for every record processed.

rule "set local field 2 for internal research"

when

True

then

set "Internal Research" in "discovery"."local2"

end

Creating Normalization Rules

The MD Editor allows you to create normalization rules, which are used to customize the transformation of external records (for Dublin Core or generic XML) into Primo VE's extended Dublin Core format for discovery. For external sources using the MARC21 format, Primo VE maps these records to the Alma-MARC21 format for discovery. Once the rules are created, you can add them as needed to normalization rule processes, which can then be assigned to import profiles. For more information, see the following pages:

DC and Generic XML: Creating a Normalization Rule Process
MARC21: Working with Normalization Rules

To add a normalization rule file:

Open the MD Editor (Configuration Menu > Discovery > Loading External Data Sources > Normalization Rules for External Data Sources).

Open Rules Tab
Select one of the following folders the folder under the Rules tab to view the relevant normalization rule files: Normalization Rules (DC) or Normalization Rules (XML). You can use them as the basis for your new rules. It is also a good practice to see whether a rule has already been created.

Create New Rule File
Create a new rule by selecting File > New > Normalization Rules (DC), Normalization rules (for MARC21), or Normalization Rules (XML), which depends on the format of the Discovery import files that you are receiving from the external source.

Configure the Normalization Rule File's Properties
Specify the following properties and then click Save:
- Name – Specify an identifiable name for the rule file, which may contain related types of rules.
  
  This field cannot contain single or double quotes: “, ”, ", ‘, ’, and '.
- Description – Enter text to describe the contents of your rule file.
- Subfolder – Select one of the following subfolder locations: Private (for your use only and cannot be included in a normalization process) or Shared (public rule file).
- Enabled – Select this check box to allow the rule file to be applied to normalization rule task lists. (By default, the normalization rule file is disabled.)
Add your rules to the editor pane. For more information, see Normalization Rule Syntax (DC and Generic XML) or Working with Normalization Rules (MARC21).

Normalization Rule Editor Pane
Click Preview. The rule or rules in the file are applied to the specified records and the outcome is displayed.
Click Save under the editor pane.

Normalization Rule Syntax

Rule files contain one or more rules, which contain one or more conditions followed by one or more actions. The following syntax is used for each rule in the file:

rule "<rule name>"

when

...

then

<action 2> [if <condition>]

...

end

Conditions

A rule can contain one or more conditions that apply to specific fields and values in the source record. If all conditions are met, the rule is applied to the record. Otherwise, the rule is not applied and the record is left unchanged. Each condition must include one of the following keywords:

true – Unconditionally applies the rule to all records.
exist – Applies the rule when a specific field exists.
not exist – Applies the rule when a specific field does not exist.
equals – Applies the rule when the entire contents of the specified field matches.
not equals – Applies the rule when the entire contents of the specified field does not match.

The following table provides an example for each type of condition.

DC and XML Conditions
Syntax	Example
General Conditions:
true	true
DC Conditions:
exist "{namespace}"."{field}	exist "dc"."description"
exist "{namespace}"."{field}"$"{xsiType}"$	exist "dc"."identifier"("dcterms:ISBN")
"{namespace}"."{field}" equals "{value}"	"dc"."identifier" equals "sport"
"{namespace}"."{field}"$"{xsiType}"$ equals "{value}"	"dc"."identifier"("dcterms:ISBN") equals "sport"
"{namespace}"."{field}" not equals "{value}"	"dc"."identifier" not equals "sport"
"{namespace}"."{field}"$"{xsiType}"$ not equals "{value}"	"dc"."identifier"("dcterms:ISBN") not equals "sport"
XML Conditions:
"xpathExpression" equals "value"	"ino:object/viaRecord/recordId" equals "test"
"xpathExpression" not equals "value"	"ino:object/viaRecord/recordId" not equals "test"
exist "xpathExpression"	exist "ino:object/viaRecord/recordId"
not exist "xpathExpression"	not exist "ino:object/viaRecord/recordId"

Actions

You can apply actions to single fields in a record based on conditions specified in the rule. If all of the conditions are met, the actions are applied to the record in the order in which they are listed. Each action must include one of the following keywords:

copy – Copies the value from a field to another field.
move – Moves the value from a field to another field.
remove – Removes the value from a field.
set – Sets the value of a field directly.

In some cases, you can append the if equals <value> condition to the above actions to apply rule changes for specific matching values only.

The following table provides an example for each type of action.

DC and XML Actions
Syntax	Example
DC Actions:
copy "{namespace}"."{field}"({"xsiType"}) to "{newTagNameSpace}"."{newTagField}"({"newXsiType"})	copy "dc"."identifier"("dcterms:ISBN") to "dc"."title"("dcterms:ISBN")
copy "{namespace}"."{field}"({"xsiType"}) to "{newTagNameSpace}"."{newTagField}"	copy "dc"."identifier"("dcterms:ISBN") to "dc"."title"
copy "{namespace}"."{field}" to "{newTagNameSpace}"."{newTagField}"({"newXsiType"})	copy "dc"."identifier" to "dc"."title"("dcterms:ISBN")
copy "{namespace}"."{field}" to "{newTagNameSpace}"."{newTagField}"	copy "dc"."identifier" to "dc"."title"
copy "{namespace}"."{field}"({"xsiType"}) if equals "{value}" to "{newTagNameSpace}"."{newTagField}"({"newXsiType"})	copy "dc"."identifier"("dcterms:ISBN") if equals "sport" to "dc"."title"("dcterms:ISBN")
copy "{namespace}"."{field}"({"xsiType"}) if equals "{value}" to "{newTagNameSpace}"."{newTagField}"	copy "dc"."identifier"("dcterms:ISBN") if equals "sport" to "dc"."title"
copy "{namespace}"."{field}" if equals "{value}" to "{newTagNameSpace}"."{newTagField}"({"newXsiType"})	copy "dc"."identifier" if equals "sport" to "dc"."title"("dcterms:ISBN")
copy "{namespace}"."{field}" if equals "{value}" to "{newTagNameSpace}"."{newTagField}"	copy "dc"."identifier" ") if equals "sport" to "dc"."title"
move "{namespace}"."{field}"({"xsiType"}) to "{newTagNameSpace}"."{newTagField}"({"newXsiType"})	move "dc"."identifier"("dcterms:ISBN") to "dc"."title"("dcterms:ISBN")
move "{namespace}"."{field}"({"xsiType"}) to "{newTagNameSpace}"."{newTagField}"	move "dc"."identifier"("dcterms:ISBN") to "dc"."title"
move "{namespace}"."{field}" to "{newTagNameSpace}"."{newTagField}"({"newXsiType"})	move "dc"."identifier" to "dc"."title"("dcterms:ISBN")
move "{namespace}"."{field}" to "{newTagNameSpace}"."{newTagField}"	move "dc"."identifier" to "dc"."title"
move "{namespace}"."{field}"({"xsiType"}) to "{newTagNameSpace}"."{newTagField}"({"newXsiType"})	move "dc"."identifier"("dcterms:ISBN") if equals "sport" to "dc"."title"("dcterms:ISBN")
move "{namespace}"."{field}"({"xsiType"}) if equals "{value}" to "{newTagNameSpace}"."{newTagField}"	move "dc"."identifier"("dcterms:ISBN") if equals "sport" to "dc"."title"
move "{namespace}"."{field}" to "{newTagNameSpace}"."{newTagField}"({"newXsiType"})	move "dc"."identifier" if equals "sport" to "dc"."title"("dcterms:ISBN")
move "{namespace}"."{field}" if equals "{value}" to "{newTagNameSpace}"."{newTagField}"	move "dc"."identifier" if equals "sport" to "dc"."title"
remove "{namespace}"."{field}" if equals "{value}"	remove "dc"."identifier" if equals "sport"
remove "{namespace}"."{field}"({"xsiType"}) if equals "{value}"	remove "dc"."identifier"("dcterms:ISBN") if equals "sport"
remove "{namespace}"."{field}"({"xsiType"})	remove "dc"."identifier"("dcterms:ISBN")
remove "{namespace}"."{field}"	remove "dc"."identifier"
set "{value}" in "{namespace}"."{field}"({"xsiType"})	set "sport" in "dc"."identifier"("dcterms:ISBN")
set "{value}" in "{namespace}"."{field}"	set "sport" in "dc"."identifier"
set "{namespace}"."{field}" to TEMP"{i}"	set "dc"."identifier" to TEMP"1"
set "{namespace}"."{field}"("{xsiType}") to TEMP"{i}"	set "dc"."identifier"("dcterms:ISBN") to TEMP"1"
set TEMP"{i}" to dc value "{namespace}"."{field}"	set TEMP"1" to dc value "dc"."identifier"
set TEMP"{i}" to dc value "{namespace}"."{field}"("{xsiType}")	set TEMP"1" to dc value "dc"."identifier"("dcterms:ISBN")
XML Actions: (For more information, see W3C XPATH Standards.)
copy "concat ({xpathExpression},' ',{xpathExpression})" to "{namespace}"."{field}" The concat keyword is an XPATH function, which is one of many XPATH functions that can be used within the XML actions.	copy "concat(//person[@pers_role='pau']/name/first/text(),' ',//person[@pers_role='pau']/name/last/text())" to "dc"."creator"
copy "xpathExpression" to "newTagNameSpace"."newTagField"({"xsiType"})	copy "ino:object/viaRecord/isbn" to "dcterms"."title"("dcterms:ISBN")
copy "xpathExpression" to "newTagNameSpace"."newTagField"	copy "ino:object/viaRecord/isbn" to "dcterms"."title"
copy all "{xpathExpression}"$"{separator}"$ to "{newTagNameSpace}"."{newTagField}" copy all "{xpathExpression}"$"{separator}"$("{newXsiType}"\) to "{newTagNameSpace}"."{newTagField}"	The copy all action allows you to import more than one instance of a field (such as creator or ISSN) based on a specific separator. To utilize this action, you must create a normalization rule for each instance of the field that you want to support. The following example allows you to import a maximum of three creators for each record: rule "Rule for creator1" when exist "//[local-name()='Item']/[local-name()='Item']/[local-name()='Component']/[local-name()='Resource']/[local-name()='mods']/[local-name()='name' and @type='personal'][1]/child::[local-name()='namePart']" then copy all "//[local-name()='Item']/[local-name()='Item']/[local-name()='Component']/[local-name()='Resource']/[local-name()='mods']/[local-name()='name' and @type='personal'][1]/child::[local-name()='namePart']/text()"(", ") to "dc"."creator" end rule "Rule for creator2" when exist "//[local-name()='Item']/[local-name()='Item']/[local-name()='Component']/[local-name()='Resource']/[local-name()='mods']/[local-name()='name' and @type='personal'][2]/child::[local-name()='namePart']" then copy all "//[local-name()='Item']/[local-name()='Item']/[local-name()='Component']/[local-name()='Resource']/[local-name()='mods']/[local-name()='name' and @type='personal'][2]/child::[local-name()='namePart']/text()"(", ") to "dc"."creator" end rule "Rule for creator3" when exist "//[local-name()='Item']/[local-name()='Item']/[local-name()='Component']/[local-name()='Resource']/[local-name()='mods']/[local-name()='name' and @type='personal'][3]/child::[local-name()='namePart']" then copy all "//[local-name()='Item']/[local-name()='Item']/[local-name()='Component']/[local-name()='Resource']/[local-name()='mods']/[local-name()='name' and @type='personal'][3]/child::[local-name()='namePart']/text()"(", ") to "dc"."creator" end
set "value" in newTagNameSpace"."newTagField"({"xsiType"})	set "whatever" in "dcterms"."title"("dcterms:ISBN")
set "value" in newTagNameSpace"."newTagField"	set "whatever" in "discovery"."local1"
set TEMP"{i}" in "{newTagNameSpace}"."{newTagField}"	set TEMP"2" in "dcterms"."title"
set TEMP"{i}" in "{newTagNameSpace}"."{newTagField}"("{xsiType}")	set TEMP"1" in "dcterms"."title"("dcterms:ISBN")
set TEMP"{i}" to xPath "{xpathExpression}"	set TEMP"3" to xpath "//datafield[@tag='identifier']"
Transformation Actions: For both DC and Generic XML, these operations can be used to transform data in the TEMP"i" variable.
remove substring using regex – This action removes a substring from the contents of a temporary field using a specified regular expression. Valid format: remove substring using regexp (TEMP"{i}","<Java_regex >")	remove substring using regex (TEMP"1","(/\|:\|;\|=\|,)+$")
concatenate with delimiter – This action concatenates the contents of two temporary fields and separates them with a delimiter. The result is placed in the first temporary field. Format: concatenate with delimiter (TEMP"{i}",TEMP"{i}","<delimiter>")	concatenate with delimiter (TEMP"1",TEMP"2","")
add prefix – This action adds the specified value to the beginning of the temporary field. Valid format: add prefix (TEMP"{i}","<value>")	add prefix (TEMP"2","$$Q")
add suffix – This action adds the specified value to the end of the temporary field. Valid format: add suffix (TEMP"{i}","<value>")	add suffix (TEMP"2","$$Q")
replace string by string – This action replaces all occurrences of a regular expression with a specified value. Valid format: replace string by string (TEMP"{i}","<Java_regex>","<string>")	replace string by string (TEMP"1","[^0-9]","\\?")
return list using regex – This action searches for all occurrences of a regular expression in the second temporary field and copies them as a list to the first temporary field. Valid format: return list using regex (TEMP"{i}",TEMP"{i}","<Java_regex>")	return list using regex (TEMP"1",TEMP"2","[a-z]{3}")
remove string – This action removes a specified string from a temporary field. Valid format: remove string (TEMP"{i}","<value>")	remove string (TEMP"1","<<")
remove leading and trailing spaces – This action removes beginning and trailing spaces in a temporary field. Valid format: remove leading and trailing spaces (TEMP"{i}")	remove leading and trailing spaces (TEMP"1")
replace spaces – This action replaces all spaces in a temporary field with a specified character or string. Valid format: replace spaces (TEMP"{i}","<string>")	replace spaces (TEMP"1"," ")
lower case – This action changes all letters in a temporary field to lowercase. Valid format: lower case (TEMP"{i}")	lower case (TEMP"1")

DC Examples

rule "add OA indication"

when

True

then

set "Unrestricted online access" in "dcterms"."accessRights"

end

rule "move description"

when

exist "dc"."description"

then

move "dc"."description" to "dcterms"."description"

end

rule "move description"

when

exist "dc"."identifier"("dcterms:ISBN")

then

move "dc"."identifier"("dcterms:ISBN") to "dcterms"."identifier"("dcterms:ISBN")

end

rule "move description"

when

"dc"."identifier"("dcterms:ISBN") equals "sport1111"

then

move "dc"."identifier"("dcterms:ISBN") to "dcterms"."identifier"("dcterms:ISBN")

end

rule "dc identifier equals sport"

when

"dc"."identifier"("dcterms:ISBN") not equals "sport"

then

move "dc"."title" to "dcterms"."description"

end

rule "from dc:type A to discovery:type C"

when

"dc"."type" equals "Photograph"

then

set "images" in "discovery"."resourceType"

end

rule "from dc:type A to discovery:type D"

when

"dc"."type" equals "Poster"

then

set "images" in "discovery"."resourceType"

end

XML Examples

rule "link to resource"

when

true

then

copy "//*[local-name()='dc']/*[local-name()='identifier'][starts-with(., 'https://www.e.myInst')]/." to "dc"."source"

end

rule "Copy identifier"

when

exist "ino:object/viaRecord/recordId"

then

copy "ino:object/viaRecord/recordId" to "dc"."identifier"

end

rule "copy first title"

when

true

then

copy "(//title/textElement)[1]" to "dc"."title"

end

rule "copy all titles"

when

true

then

copy "(//title/textElement)" to "dcterms"."title"

end

rule "creator"

when

exist "ino:object/viaRecord/work/creator/nameElement"

then

copy "ino:object/viaRecord/work/creator/nameElement" to "dc"."creator"

end

rule "equals english"

when

"ino:object/viaRecord/work/lang/textElement" equals "eng"

then

set "English" in "dc"."language"

end

rule "set Constant"

when

true

then

set "test subject" in "dc"."subject"

end