Testing Normalization Rules for External Data Sources

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Overview

The Testing Normalization of non-Alma Primo VE Records page enables you to test your normalization rules and processes prior to loading either Dublin Core (DC), generic XML, MARC21, or UNIMARC records into Primo VE from an external data source.

To test a normalization rule or process, you run it on a sample record that you provide. Before you begin the testing procedure, you must create any normalization rules or processes that you want to test (For more information, see Configuring Normalization Rules for External Resources (Primo VE), Working with Normalization Rules for MARC21, Creating Normalization Processes for External Data Sources).

You can supply a sample record to use as input for a test in one of the following ways:

Upload a test file containing one or more sample records, and then select one of the records to test.
Manually enter a sample record in a text box.

The sample record that will be used as input is displayed under Source Record. Once you run the test on the sample record, the normalized version appears under Normalized Record.

"Testing Normalization of non-Alma Primo VE records" page when a test file has been uploaded and a test has just run, showing the beginnings of the source record and the normalized record.

Testing Normalization of non-Alma Primo VE Records Page After a Test Has Run

Preparing a Source Record

The following is an example of a qdc source record:

<record>
   <metadata>
       <qdc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:doc="http://www.lyncode.com/xoai" xmlns:qdc="http://www.openarchives.org/OAI/2.0/qdc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/qdc/ http://www.openarchives.org/OAI/2.0/qdc.xsd">
           <dc:title>Efficient contact determination between geometric models</dc:title>
           <dc:creator>Lin, Ming C.</dc:creator>
           <dc:creator>Manocha, Dinesg</dc:creator>
           <dc:contributor>Naval Postgraduate School (U.S.).</dc:contributor>
           <dc:contributor>Computer Science</dc:contributor>
           <dc:subject>COLLISIONS.</dc:subject>
           <dc:description>http://archive.org/details/efficientcontact00linm</dc:description>
           <dc:description>NA</dc:description>
           <dc:date>1994-03</dc:date>
           <dc:date>2013-02-27T23:23:12Z</dc:date>
           <dc:date>2013-02-27T23:23:12Z</dc:date>
           <dc:date>1994-03</dc:date>
           <dc:type>Technical Report</dc:type>
           <dc:identifier>http://hdl.handle.net/10945/28719</dc:identifier>
           <dc:identifier>ocn640484785</dc:identifier>
           <dc:identifier>NPS-CS-94-007</dc:identifier>
           <dc:language>en_US</dc:language>
           <dc:format>31 p. : ill. ; 28 cm.</dc:format>
           <dc:format>application/pdf</dc:format>
           <dc:publisher>Monterey, California. Naval Postgraduate School</dc:publisher>
       </qdc:dc>
   </metadata>
</record>

The following characters must be escaped when not part of the markup text:

Ampersand (&) – use &
Left angle bracket (<) – use <

For example: <dc:subject>Engineering & Technology</dc:subject>

Here are some examples of files containing sample records in various XML formats:

oneGenericRecord.xml
via_export_incr_upload_files.xml
extended_dublin_core_one_record.xml
oai_dc - one record.xml
one_record_testing_external.xml
qdc - one record.xml
MARC_record.xml
MARC XML file with multiple records: MARC21_31_records.xml
MARC binary file with multiple records: MARC_Binary.mrc

Working with Test Files

A test file is an XML or binary file containing one or more records to use as input when testing normalization rules. The following file formats are supported:

XML files:
- Dublin Core (DC) – oai_dc or qdc format
- Generic XML format
- MARC21
- UNIMARC
Binary files: MARC21

A test file should contain no more than 20 sample records. (If it contains more than 20, only the first 20 are loaded.) The file should not be larger than 1MB.

Only one test file can be loaded at a time for each supported format. The last test file uploaded is retained, and the records it contains are available for use as test input by all the Primo VE admins in the institution. Furthermore, the loaded file can be downloaded, making it easy for other admins to access it. In order to upload a different file for a particular format, you must first delete the one that is currently loaded.

The contents of a test file cannot be edited in the Testing Normalization of non-Alma Primo VE Records page. If you want to modify the contents of a test file after you upload it, you must edit the original file and then replace the current version with the edited version.

Loading a Test File

If no test file is currently loaded for the selected format, you can load a test file of that format. The records in the file then become available for testing.

To load a test file:

Navigate to the Testing Normalization of non-Alma Primo VE Records page (Configuration Menu > Discovery > Loading Records from External Data Sources > Test Normalization Rules).

Testing Normalization of Non-Alma Primo VE Records Page
Under Select record type, select the format of the test records: DC, Generic XML, Marc21 XML, Marc21 Binary, or Unimarc.
If you are loading a Generic XML file, fill in the fields as follows:
- Root element tag – The name of the top-level or first tag in the file, which precedes the list of individual records (for example, listRecords)
- Record elements tag – The name of the tag that indicates the beginning of a single record within the XML file (for example, record)
- Path to record title – The XPath to the title tag of the record (for example, record/title)
Under Browse for file, select your test file. The file is shown in the text box.
Select Upload. If the contents of the file are valid for the selected format, the file is loaded for that format. Its name then appears along with a Delete button, and the first record it contains appears in the Source Record area. (If you uploaded a binary file, the records it contains are converted into XML and then displayed in the Source Record area.)

Running a Test

Once you have a loaded test file, you can run tests on it to see how its contents are processed by specific normalization rules or processes.

To run a test on a record:

Under Select test type, select the type of test you want to perform: Normalization rule or Normalization process.
Under Normalization rule or Normalization process, select the rule/process to test.
If the file contained multiple records, in the Source Record area, from the Record to test dropdown list, select the record to use as input for the test.
Select Run test. The test runs, and the normalized version of the record appears in the Normalized Record area.
Review the normalized version of the record to make sure that the source record has been normalized correctly.

Downloading the Currently Loaded Test File

When a test file is loaded for a format, staff with access to the Testing Normalization of non-Alma Primo VE Records page can download the file from the page.

To download the currently loaded test file of a format:

Select the name of the file.

Replacing the Loaded Test File

When a test file is loaded for a format, if you want to load a different test file, beside the file name, select Delete. Then, follow the instructions above to load the new test file.

Manually Entering a Record to Test

As an alternative to uploading a file containing sample records on which to test normalization rules and processes, you can manually enter text into the Testing Normalization of non-Alma Primo VE Records page, either by copying and pasting the code of a record or by typing in the code yourself. Sample records that are entered in this way can be manually edited in the page, making it easier to implement slight modifications to records and then see how those changes affect their normalized versions.

Only one record should be manually entered at a time. The sample record can have a maximum length of 4096 characters.

Entering and Testing a Record

This section explains how to manually enter a generic XML, MARC XML, or UNIMARC XML record and run a rule or process on it. The procedure is slightly different for DC records, as explained in the next section.

To manually enter and test a generic XML, MARC XML, or UNIMARC XML record:

Under Select record type, select the format of the record: Generic XML, Marc21 XML, Marc21 Binary, or Unimarc.
If you are loading a Generic XML record, fill in the fields as follows:
- Root element tag – The name of the top-level or first tag in the file, which precedes the list of individual records (for example, listRecords)
- Record elements tag – The name of the tag that indicates the beginning of a single record within the XML file (for example, record)
- Path to record title – The XPath to the title tag of the record (for example, record/title)
Under Select test type, select the type of test you want to perform: Normalization rule or Normalization process.
Under Normalization rule or Normalization process, select the rule/process to test.
If a file is already loaded for the format you selected, in the Source Record area, from the Record to test dropdown list, select Edit record manually. The text box is cleared.
In the Source Record area, enter the XML code of the record on which you want to run the selected rule or process.
Select Run test. The test runs, and the normalized version of the record appears in the Normalized Record area.
Review the normalized version of the record to make sure that the source record has been normalized correctly.

Entering and Testing a DC Record

You can manually enter a DC record and run a rule or process on it. You can also download the DC record after the rule/process runs.

To manually enter and test a DC record:

Under Select record type, select DC.
Under Select test type, select the type of test you want to perform: Normalization rule or Normalization process.
Under Normalization rule or Normalization process, select the rule/process to test.
If a file is already loaded for the format you selected, in the Source Record area, from the Record to test dropdown list, select Edit record manually. The text box is cleared, and a Edit Record option is added to the Source Record area.
Select Edit Record. The Edit Record window opens, with a blank text box in it.
In the text box, enter the XML code of the record on which you want to run the selected rule or process.
Select Test. The text you entered is parsed and then displayed in a table in the Source Record area, the test runs, and the normalized version of the record appears in a table in the Normalized Record area.
Review the normalized version of the record to make sure that the source record has been normalized correctly.

Downloading a Manually Entered DC Record

You can download the code of a sample DC record that you entered manually. To do so, in the Source Record area, select Download.