Intota: Data Retrieval Service (DRS): SUSHI-Protocol Harvesting
- Product: Intota
What can you tell us about the DRS SUSHI-protocol harvesting method?
The SUSHI (Standardized Usage Statistics Harvesting Initiative) protocol is a set of standards for online usage-reporting and analysis that allows librarians to obtain and evaluate their library's usage statistics automatically. See SUSHI Reports Registry COUNTER Code of Practice Release 4 for more information.
Intota's Data Retrieval Service (DRS) uses SUSHI as one of its two methods for retrieving usage statistics from providers (the second method being Administration-Based Harvesting).
The DRS Request Form includes SUSHI-capable providers listed on the SUSHI Server Registry. That website also provides info for contacting the providers to request SUSHI setup and SUSHI credentials.
If you find a provider that has SUSHI usage-statistics retrieval options but is not on the SUSHI Server Registry, please encourage the provider to get on the Registry. Meanwhile, you can ask the provider to give you all their SUSHI information (including SUSHI server URL; which reports are available via SUSHI; and required SUSHI credentials such as Requestor ID, Customer ID, and so forth) and send the information to us using the Submit a Case option. We will add the provider and will monitor it, but we can't guarantee successful upload of all reports every month.
Our SUSHI system will automatically retrieve usage statistics from SUSHI providers (three months ago based on the current date) on a monthly basis.
Click the following links for more information about each subject:
Set Up SUSHI-Protocol Harvesting
To request that we use the SUSHI-protocol harvesting method to get usage-statistics reports from a provider, you need to provide us several pieces of information on the DRS Request Form. The most common information required are the provider's SUSHI-server URL, and your library's SUSHI credentials from that provider (usually Requestor ID and Customer ID). Some providers require more information, and they are noted in the Admin Note on the DRS Request Form. You will need to tell us which usage-statistics report types to harvest with SUSHI (JR1, DB2, and so forth). Finally, you will want to test the SUSHI settings to ensure they are all correct.
If a provider needs to know our SUSHI-system IP addresses before they will allow our SUSHI system to access their SUSHI server, inform them of the following:
207.170.201.0 - 207.170.201.255
207.170.231.0 - 207.170.231.255
207.170.234.0 - 207.170.234.255
207.170.239.0 - 207.170.239.255
207.170.241.176 - 207.170.241.183
216.17.112.0 - 216.17.119.255
216.147.208.0 - 216.147.215.255
Auto-Harvesting by SUSHI
After completely entering SUSHI information for a provider on the Data Retrieval Configuration form, once a month (on the second day of the month) our SUSHI system will automatically harvest your usage-statistics reports from the provider.
You will be able to find the usage-statistics reports on the COUNTER Summary page, just like reports retrieved by the Administration-Based Harvesting method and those Intota: Uploading COUNTER Reports. In the Uploaded By column, they will say "Auto-Harvest":
Usage statistics reports retrieved by the SUSHI-protocol harvesting method will always contain only the most recent, complete month of usage statistics. Therefore, you should not remove older SUSHI-protocol-harvested reports with current-year usage statistics, because if you do you will lose usage statistics from preceding months. This is different than Administration-based Harvesting reports (and those you retrieve and upload yourself), where newer reports usually contain all usage statistics for the calendar year to date.
If, for any reason, our SUSHI system is not able to harvest your data from a particular provider, it will re-queue the request several more times during the month. The only reason it will stop trying is if it receives an error message from the provider's SUSHI server that indicates it shouldn't continue to try. See Unsuccessful Harvesting or Consolidating Harvesting below for details.
If our SUSHI system continues to be unable to harvest data from a particular provider and it is time for your scheduled administration-based harvesting, then our DRS staff will use that method to harvest your data. For this reason, it is important that you add credentials for administration-based harvesting to the DRS Request Form, even if the provider also gives you SUSHI credentials.
Successful Harvesting and Consolidating
The green arrow icon in the Status column indicates that usage-statistics report was successfully harvested by our SUSHI system and the data was pushed to the consolidated reports system.
If you want to see more information about the actual usage-statistics report that was harvested, click the report type (JR reports highlighted in the above screenshot), and you'll see the Counter Report Data page. To download the report, click the filename in the Download button..
Unsuccessful Harvesting or Consolidating
There are two major steps in the SUSHI-harvest process where an error can occur:
-
Our SUSHI system failed to auto-harvest the usage-statistics report from the provider's SUSHI server; or
-
Our SUSHI system did auto-harvest the report but there was an error when Intota tried to validate the data.
Regardless when the error occurred, you can determine what went wrong by selecting the View Logs icon:
The error report will be either an auto-harvest error or a report-validation error (see below). Regardless of the type of error, when a SUSHI retrieval fails, we will ensure that you have your usage statistics according to your predetermined DRS schedule by conducting Administration-Based Harvesting .
Auto-Harvest Error
When you click the View Logs icon and view the Report Log, if the error information page is very short and doesn't contain any usage-report data, then our SUSHI system failed to auto-harvest the usage-statistics report from the provider's SUSHI server.
We can't predict exactly what message each provider will send for every error, but hopefully you will be able to figure out what went wrong based on the provider's message. For example, a message like "Requestor Not Authorized to Access Service" probably means you entered the wrong Requestor ID or Customer ID for the provider on the DRS Request Form. If the message is anything like "Server Not Available," then our SUSHI system was not able to establish a connection at all and will try again on a regular basis until it can harvest the report.
Report-Validation Error
If the log has many entries, then our SUSHI system likely did auto-harvest the report but there was an error when Intota tried to validate the data.
Re-Harvest Usage Statistics
The re-harvest function allows you to retrieve usage statistics from a provider for a specific time (month and year) that was previously retrieved. You can re-harvest a successful or failed data retrieval. Note that the Updated By value will become Auto-reharvest.
-
Go to Reports > Counter Summary
-
Click the re-harvest icon (see below) for the report you want to re-harvest.
-
The message Re-Harvest Scheduled appears briefly on the screen.
The re-harvesting functionality is also available on the COUNTER Report Summary page (accessed by clicking the Summary View button in the top right of the COUNTER Reports page).
- Date Created: 31-May-2014
- Last Edited Date: 20-Jul-2022
- Old Article Number: 10816