Incremental ETL duration too long

Problem Symptoms:

* the incremental ETL takes several hours and takes almost as long as the full ETL

Cause:

Configuration of ./xxx50/tab/arc_conf_table.conf

Resolution:

In order to reduce the duration for the ETL load, reduce the amount of data included into the load.
Check / Correct the settings of table .xxx50tabarc_conf_table.conf to include less data:

[ARC HISTORY]       period for all history files (z30h, z36h, z37h)
[OPAC]                       period for OPAC events (z69)
[EVENTS]                  period for events (z35)

Additional Information

In ARC 2.0 the Incremental Load differs from the Full Load only this way, that the events (z35) and OPAC events (z69) are loaded incremental. All other files are fully loaded during ETL.

Starting with ARC 3.0 / Aleph 21 a Full Complete Incremental Load is available.

For details please see ALEPH Configuration for ARC 3.0.pdf

The following additional options are available starting with ARC 3.0:

[PATRON-DATA]
eliminate_data=N

[CIRC-LOGGER]

from date = 20070101

• Article last edited: 06-Feb-2017