Monitoring Primo Status

Pipe Monitoring

The Pipes List page (Primo Home > Monitor Primo Status) lists the pipes that are configured in Primo. You can filter the pipes by institution using the Owner drop-down list and include the pipe templates by selecting the Display Template Pipes check box.

Pipe Monitoring - Pipes List

Institution-level staff users can view pipes that are owned only by their institution.

The Status and Stage columns on the Pipes List page allow you to quickly determine the status of each pipe. The following table describes all of this page's columns and buttons.

Pipes List Components
Component	Description
Pipe Name	This field contains the name of the pipe.
Pipe Type	This field indicates the type of pipe. For more information, see Define Pipe Details.
Status	This column displays the current status of the pipe. The following statuses may display: completed – indicates that the work completed successfully or there were no files/records to harvest. not started – indicates that a pipe has been executed, but the process has not started. pending for reloaDS – indicates that other processes are currently running and this process has been added to the queue. pending resume – indicates that the pipe has been resumed, but the process has not started. pending suspend – indicates that a suspend request has been sent, but the process has not been suspended. pending termination – indicates that the pipe has been terminated, but the process has not been started. resumed – indicates that the pipe has resumed execution. running – indicates that the pipe is executing. stopped error – indicates that the pipe has been stopped due to an error that is generally caused by a technical error in the system. stopped harvest error – indicates that the pipe was stopped due to an error in the harvesting stage. stopped threshold exceeded – indicates that the pipe stopped because the number of records that failed exceeded the error threshold. suspended – indicates that the execution of the pipe has been suspended. terminated – indicates that the execution of the pipe was terminated Click the status to display more detailed information on the job. For more information, see Viewing a Pipe's Details.
Stage	This column displays which stage of the process the system is running for the pipe. The following stages may display: pending for harvest – indicates that the pipe has been placed in the harvesting queue. harvesting-copy-files – indicates that the pipe is copying files from the data source. harvesting – indicates that the pipe is harvesting from the data source. pending for bulkage – indicates that the pipe has started and will retrieve the source data from the Primo database (P_SOURCE_RECORD). bulkaging – indicates that the pipe is splitting the records into bulks from the Primo database (P_SOURCE_RECORD). harvest-nep – indicates that the harvest stage and normalization, enrichment, and persistence (NEP) stage are running in parallel. pending_nep – indicates that the pipe is waiting for the NEP stage to start. nep – indicates that the pipe is running the NEP stage. dedup – indicates that the pipe is at the dedup stage. dedup_merge – indicates that the pipe is created the merged deduped records. frbr – indicates that the pipe is at the FRBRization stage. frbr_merge – indicates that the pipe is creating the merged FRBR records. completed – indicates that the execution of the pipe finished successfully.
Edit button	This button provides access to the Define Pipe page, which allows you to modify the settings of a pipe. For more information, see Editing a Pipe.
Execute button	This button starts the execution of the pipe. During the execution of a pipe, the following stages are performed: Harvesting, NEP (which includes normalization, enrichment, and persistence), Dedup, Dedup merge, FRBR, and FRBR merge. Each pipe is run as a job, which you can monitor via the Job Monitor page. For more information, see Job Monitoring. Log files for each pipe are stored in the harvest and log directories under the following directory on the Primo server, where <n> is the Primo copy: /exlibris/primo/p1_<n>/ng/primo/home/profile/publish/publish/production/pipes/<pipe_name/> In addition, a general log file called publish_server.log is stored under the following directory: /exlibris/primo/p1_<n>/ng/primo/home/profile/publish/publish/production/log If a pipe fails, an e-mail message is sent to the e-mail address specified in the Contact Person E-mail parameter (Primo Home > Advanced Configuration > General Configuration > E-mail and SMS Configuration) page. Pipe email addresses defined at the installation and institution levels in the E-Mail Addresses mapping table override the value specified in the Contact Person E-mail parameter.
History button	This button displays the Job Monitor page, which displays an entry for each execution of the pipe. For more information, see Viewing a Pipe’s History.
Errors button	This button displays the errors that occur during the execution of a pipe. For more information, see Viewing a Pipe’s Errors.
Create New Pipe button	This button provides access to the Define Pipe page, which allows you to add a pipe to the system. For more information, see Defining a Pipe.
Delete button	Deletes an existing pipe from the list. For more information, see Deleting a Pipe. This button does not appear after the pipe has been executed.

Viewing a Pipe's Details

The Job Details page displays detailed information on the execution of a pipe.

Pipe Details Page - Completed Status

This information is split into sections by stage, as follows:

Pipe name and overall pipe status – includes the Job ID, stage, status, and start and end times. To view the Pipe's settings on the Define Pipe page, click Configure.
Harvest – This section displays the results of the harvest stage:
- Files Copied – the number of files that were copied.
- Failed Records – the number of failed records.
- Normal Records – the number of inserted and updated records.
- Deleted Records – the number of records that were marked for deletion.
- Normal/Deleted Bulks – the number of normal and deleted bulks, respectively. Each bulk contains up to 1000 unique records, and a record may span multiple bulks.
- Num Inserts – the number of new records that were harvested.
- Num Updates – the number of updated records, which includes duplicate records that span across bulks.
- Num Duplications – the number of duplicate records, which includes the duplicate records deleted from a bulk (only the latest record is saved in a bulk) and the duplicates that span other bulks.
- Failed Files – the number of files that could not be completely processed due to parsing error. Note that some of the records in the file may have been saved to the M_P_SOURCE_RECORD database table, while the failed record is added to the count in the Failed Records field.
- Empty Files – the number of empty files that were processed.
- Start/End Time – the job’s start and end times, respectively.
NEP – includes the number of successful and failed records and the start and end times. In general, the sum of the NEP succeeded records and the NEP failed records should be equal to the number of harvested normal records. In some cases, the number of duplicate records may have some effect on the sum.
Dedup – includes the number of single records, member records (such as the number of records that belong to a dedup group), and merged records. In addition, this section displays the number of dedup vectors that were created in the NEP stage, and the start and end times.
FRBR – includes the number of single records, member records (such as the number of records that belong to a FRBR group), and merged records. In addition, this section displays the number of FRBR keys that were created in the NEP stage, and the start and end times.

On the Job Details page, you can select the following options:

Pipe Details Page Options
Button/Link	Description
Resume button	Resumes the execution of a pipe that has been suspended by a user or has been stopped by the system due to an error. After a resume request, the work status changes to pending resume and will not resume immediately. Execution will begin when process activity decreases. Resume tries to restart the work from its last point of execution for most cases. Otherwise, it will resume from a previous stage. During a resume, Primo loads and processes bulks that were previously failed, but it does not process individual failed records. If a pipe is stopped during the harvest stage, a status of stopped harvest error is returned. Since a pipe cannot be resumed when it is stopped in the harvest stage, you can terminate the pipe and then restart it. If there were many failed records as a result of problems in normalization, the best solution is to terminate the pipe, fix the normalization rules, and then execute the pipe again. This will override previous changes to the database and update records that were already saved. This option is not recommended and has been deprecated.
Suspend button	Suspends a running pipe. When a pipe is suspended, the work status changes to pending suspend, and when it completes, the status changes to suspended. A suspend request is not permitted during the harvest phase. This option is not recommended and has been deprecated.
Terminate button	Stops the execution of a currently running pipe.
Configure link	Used to modify a pipe’s definition. For more information, see Editing a Pipe.
View Log link	Displays the errors and processes that occurred during each stage of the pipe's execution except for the Harvest stage.
View Harvest Log link	Displays the errors and processes that occurred during the Harvest stage.
View Dedup Log link	Displays the errors and processes that occurred during the Dedup stage.
View FRBR Log link	Displays the errors and processes that occurred during the FRBR stage.
Restore Files link	Restores the harvested files to the original directory so that the pipe can be run again from scratch. This link displays only for a pipe job that is associated with a pipe that is marked as Delete after copy and its current status is stopped error, threshold exceeded error, or terminated. The Restore Files option is not relevant for OAI harvesting.

To display the Job Details page:

On the Back Office’s home page, click Monitor Primo Status.

The Monitor Primo Status page opens.
Click Pipe Monitoring on the Monitor Primo Status page.

The Pipe Monitoring page opens, showing the lists of pipes.
Click the Process Status link next to the pipe you want to monitor to display the Job Details page.

Viewing a Pipe’s History

The Job Monitor page displays the execution history for a pipe.

Job Monitor - History Page

From this page, you can display additional information by performing the following actions:

View error information by clicking an Errors button next to a job in the list.
View job details by clicking a link in the Name column.

To view the history for a job:

On the Back Office’s home page, click Monitor Primo Status.

The Monitor Primo Status page opens.
Click Pipe Monitoring

The Job Monitor page displays the list of jobs.
Click History next to the job that you want to view.

Viewing a Pipe’s Errors

The Error Messages page allows you to view more information on the specific records that contained errors during the harvesting and NEP stages of the pipe's execution.

Error Messages Page

To view the errors:

On the Back Office’s home page, click Monitor Primo Status.

The Monitor Primo Status page opens.
Click Pipe Monitoring.

The Pipe Monitoring page opens, showing the list of pipes.
Click Errors next to the pipe whose errors you want to view.

The Errors page opens, showing the error details for the specified pipe.

Click the link in the Source Record Id column to view the source record.

The errors that occur most frequently are listed in the following table:

Errors While Running the Pipes
Error	How to Solve the Problem
TEMP table space is full in oracle	Talk to your DBA about increasing the TEMP table space.
TS_P_DAT table space is full	The table space used for storing Source Records and PNX is full. Talk to your DBA about increasing the TS_P_DAT table space.
The Oracle user lacks privileges (this is important during the initial dedup stage)	Talk to your DBA to give create any directory privileges to the primo_library Oracle role.
No physical disk space left on the production directory partition	File system for storing bulk zip files is full. Talk to your System Administrator about increasing the available disk space.
Input data is not in a valid XML format	The source data is not valid XML. Check with the owner of the source system.
Input records are missing mandatory fields	The source data is missing mandatory fields: sourcerecordid, sourceid, recordid, type, title, and delcategory. For example: Invalid content was found starting with element 'creationdate'. One of '{"":title}}' is expected. The part "[...] One of '{"":title}}' is expected This indicates that the required display/title field could not be created for the record. To diagnose this problem, check the source record and the normalization rules. In some cases, you may need to check with the owner of the source system.
The source directory is not open for reading by the Primo user	The source directory accessed by Primo by the pipe Harvesting module is not accessible for viewing. Talk to your System Administrator.
Files in source directory have an earlier modification date than pipe	The harvesting date on the file must be more recent than the date configured for the pipe. Talk to your System Administrator and/or owner of the source system.

On the Errors Messages page, click Back to Pipes List to return to the Pipe Monitoring page.

Tools Monitoring

The Tools List page (Primo Home > Monitor Primo Status > Tools Monitoring) lists all of the tools that you have created and displays the status for each tool job. You can filter the tools by institution using the Owner drop-down list. For institution-level staff users, your institution is selected for you.

For information on how to create tools, see Primo Tools.

Although you can monitor all tool jobs for your institution, you are permitted to create and execute the following tools only: Generate Sitemap, Delete Sitemap, Export Primo configuration, and Import Primo configuration.

Tools List Page

From the Tools List page, you can perform the following operations on each tool:

Execute the tool – Click the Execute button to run the tool. After you have executed the tool, you can monitor its progress on the Tools List page.
Edit the tool – Click the Edit button to open the configuration page for the selected tool. For more information on each type of tool, see Primo Tools.
View the log file – Click the History button to view the log file. For more information, see Tools History List.

The Create a Tool section allows you to create additional tools, which are added to the Tools List page. For information on adding tools, see Primo Tools.

Tools History List

The Tools History List page displays status information for each tool that you have executed.

Tools History List Page

For each tool, the following information displays:

Pipe Name – This column displays the name of the tool’s process. Click this link to display the tool’s log. For more information, see Tools Process Log.
Stage – This column displays the execution stage of the tool.
Status – This column displays the overall status of the tool.
Start/End Time – These columns display the start and end times of the tool’s execution.

Tools Process Log

The Tools Process Log page displays the log file for the execution of the tool.

Tools Process Log Page

Process Monitoring

The Process List page allows you to monitor the Primo processes used by your system. From this page, you can view the status of each process, the execution date and time, and the number of succeeded or failed records.

Process List Page

In addition, you can perform the following operations on each process:

View the process history – Click the History button to view the log for the process.
Execute the process – Click the Execute button to run the process. After you have executed the process, you can monitor its progress on the Process List page (on-premises installations only).
Clean up the process – Click the Clean-up button to delete the history of the process and any jobs waiting for execution in the queue (on-premises installations only).

For instructions on creating a new process, see Scheduling Tasks.

To monitor the system's processes:

On the Back Office’s home page, select Monitor Primo Status.

The Monitor Primo Status opens.
Click Process Monitoring.

The Process Monitoring page displays the list of processes. For details on ongoing indexing, see Ongoing Indexing.

Job Monitoring

You can monitor various jobs (such as pipe and indexing jobs) that were executed on your system via the Job Monitor page (Primo Home > Monitor Primo Status > Job Monitoring). This page displays the following information: job status, execution date and time, and last stage executed.

Job Monitor Page

In addition, this page allows you perform the following actions:

View history information by clicking the History button next to the relevant job.
View error information by clicking the Error button next to the relevant job.
View detailed job information by clicking the link in the Name column.

For details on ongoing indexing, see Ongoing Indexing.

Deploy Monitoring

The Deploy Job Summary page (Primo Home > Monitor Primo Status > Deploy Monitoring) allows you to filter and display the deploy jobs that have been run on a specified day. By default, this page lists the deploy jobs for the current day, starting with most recent deploy job.

Filter Deploy List Page

On this page, staff users can perform the following actions:

Click the links in the Deploy ID column to display the log for the corresponding deploy job.
Click Go Back to Deploy All to perform a new deploy on the Deploy All page.

To display a deploy log:

On the Deploy Job Summary page (see Filter Deploy List Page), use the information in the following table to filter the deploy jobs for a specified day.

For installation-level staff users, select the installation name or a specific institution from the Owner drop-down list. The valid values are Installation (if using the default name) or a specific institution.

For institution-level staff users, the Owner field is set to your institution.

Deploy Job Summary Details
Field	Description
Deploy Name	Indicates the type of deploy the system has run. You can click this link to display the log. If the job is still running or has failed, the system automatically displays the log.
Deploy Date	The date on which the deploy job was executed. The system displays the jobs for the current day by default.
User Name	The user name of the staff user that executed the deploy job.
Deploy ID	The deploy ID process. You can click this link to open the Deploy Log Details page for the corresponding deploy job.
Status	The status of the deploy job. The following statuses are possible: Failed – The deploy job has failed. Refer to the Deploy Log Details for more information. For example, a deploy will fail if the target servers are down. Finished – The deploy job has finished without errors. Running – The job is still running. Waiting – The job is waiting in the queue to be executed.

Click Search to filter the jobs as specified.
Click a link in the Deploy ID column to display the log for the corresponding deploy job. For more information, see Deploy Log Details Page.

Deploy Log Details Page

The Deploy Log Details page allows staff users to view details for each stage of the deploy process.

Deploy Log Details Page

Each stage includes the following information:

Target Name – The name of the stage in the deploy process, which includes the following stages:
- MAIN_JOB – The first stage in the deploy process
- DEPLOYER_BE – Deployment to the Back Office server.
- DEPLOYER_FE – Deployment to the Front End server.
You may see multiple entries for DEPLOYER_BE and DEPLOYER_FE depending on the number servers and the types of configurations that were selected for the deploy job.
Deploy name – The type of configuration that the system is deploying. You can click the Deploy name link to display the log. The system automatically displays the log if the stage is still running or has failed.
Target IP – The IP to which the configuration has been deployed.
Description – The description of the stage.
Module – The module indicates whether the stage took place in Back Office (BE), Front End (FE), or Search Engine (SE).
Position – The position of the deploy stage in the process queue
Status – The same statuses as for the deploy as a whole are possible

The system automatically refreshes this page so that you can monitor each stage in real time. You can stop the automatic refresh by clicking the Stop Refresh button.

Wizard-Level Deploys

When deploys are executed on a page other than the Deploy All page (such as the Views wizard), the system will display a message containing a process ID and a link that can be clicked to view the log for the job.

Wizard-Level Deploy

Search Engine Monitoring

The Search Engine Monitoring page allows you to monitor the slices within the Primo search engine to ensure that the search engine is functioning properly.

You can view the status for the SEs, but Ex Libris will monitor and maintain them for cloud installations.

Search Engine Monitoring Page

On this page, you can view the groups of server slices and the statuses of the groups and their slices. Each group of slices is managed by an agent representing a specific computer's search engine.

Search Engine Monitoring Description describes the fields on the Search Servers Monitoring page.

Search Engine Monitoring Description
Field	Description
Search Engine Name	The name of the agent that manages the group of slices. Each agent represents a search engine server. In this example, the search server name is Monitor il-primo03.corp.exlibrisgroup.comServer.
Status	The agent's status.
Slice Number	The slice's Id number. Each slice has a unique slice number which is used throughout the system.
Slice Status (on-premises installations only)	Indicates the status of each slice. The status is Up when the slice is available for use and the status is Down when the slice is unavailable for use. When the status is Down, the Stop button changes to Start. If the slice is down, then something in the system is not functioning properly. Check your server log to figure out the problem. For more information on the server log, refer to the Primo System Administration Guide.
Memory-Disk Synch Status (on-premises installations only)	Each time an indexing of the records is performed, a new version of indexed records is created. After the new version of records is created but before the new records have been loaded into the memory and swapped with the old indexes, the Memory-Disk Status is Unsynchronized. Once the new versions of indexes are swapped with the old indexes in the memory, the Memory-Disk Status changes to Synchronized.

The slices are managed by agents that represent the search engine servers. You can manage Primo's search engine slices by performing one or more of the following actions:

Reset – You can use the reset function to reload the server's agents who then load a new search schema. The reset function is generally only used if one of the servers has been modified, and is normally not used during production.
Deploy – You can use the deploy function to enable the agents to re-initialize themselves with their relevant schemas and then pass along the information received from the search schema. The deploy function is performed automatically and therefore is not necessary to use.
Silent deploy – The silent deploy function is the same as the deploy function in that it sends information to each of the agents to re-initialize themselves with their relevant schemas. The silent deploy function differs from the deploy function in that it does not send a command to execute the information received.

To manage Primo's search engine slices:

From the Back Office’s home page, click Monitor Primo status.

The Monitor Primo Status page opens.
Click Search Engine Monitoring to display the Search Engine Monitoring page.
Next to the slice, click one of the following buttons:
- Start – to make the slice available for use and change the slice's status to Up.
  
  The slice is available for use.
- Stop – to make the slice unavailable for use and change the slice's status to Down.
  
  The slice is no longer available for use.
  
  During the regular process flow, the Stop function should never be used. When the Stop function is used, it indicates that a part of the system is not functioning properly and is unavailable for use.
Optionally, you can perform one of the following actions:
- Click Reset to reload the server's agents who then load a new search schema.
  
  The reset function is generally only used if one of the servers has been modified, and is normally not used during production.
  
  The agents and new search schema are loaded.
- Click Deploy to enable the agents to re-initialize themselves with their relevant schemas and then pass along the information received from the search schema.
  
  The deploy function is performed automatically and therefore is not necessary to use.
  
  The agents are re-initialized and the relevant schemas information is executed.
- Click Silent Deploy to send information to each of the agents to re-initialize themselves with their relevant schemas.

The agents are configured and are waiting for notification to make the slices available.

Watchdog Monitoring

For on-premises installations the Watchdog Configurations page lists the monitors that are available in Primo.

Watchdog Configurations Page

From this page users can enable the Watchdog monitors and edit the configuration settings for each monitor.

To enable a monitor, check the corresponding Enabled check box and then click Save.

To configure the Watchdog monitors:

Click Edit to update the configuration settings of a monitor.

The Edit Watchdog Configuration page opens.

Edit Watchdog Configurations Page

Enter the fields as described in Watchdog Configuration Details.

Watchdog Configuration Details
Field	Description
Adaptor Name	The name of the monitor.
Default Interval (sec)	The time interval (in seconds) for availability checks.
Retries	The time interval (in seconds) for retries.
Error Threshold	Notification will be sent only when the error threshold is met.
Alert Interval (min)	The minimum time interval (in minutes) between alerts if positive notification is not sent out.
Contact Email	The contact e-mail address. Multiple email addresses can be separated with a semicolon.
Contact SMS	The SMS contact number.

Click Save to save your updates.
Repeat this procedure for each monitor you would like to configure.