When to Back-Date a Terminated Ongoing Pipe

Question

When an ongoing pipe terminates, does the pipe date stamp remain unchanged so all the files and records from that day will be processed on the following day?

Answer

The trigger for a Harvest Time Stamp (aka Define Pipe > “Start harvesting files/records from” field setting) update is HARVEST completion.
If a Pipe is terminated before it finishes the harvest stage, the pipe’s Harvest Time Stamp will not update.
If you are uncertain, check the pipe Job Details page and, if you see an End Time stamp on the Harvest section, you can be sure that the Define Pipe > Harvest Time Stamp HAS been updated.

Additional Information

IMPORTANT: Ongoing pipes that terminate in a Group B status DO need to be back-dated.
In this case, the pipe must be edited, the Harvest Time Stamp changed to an earlier date, and saved in order to pick up the missed data (this will work as long as the source files have not been deleted).

Pipes that terminate in a Group A status (below) DO NOT need their Harvest Time Stamp back-dated.

Common statuses for terminated pipes:
Group A
* pending for harvest
* harvesting-copy-files
* harvesting
* harvest-nep
Group B
* nep
* dedup
* pending_nep
* dedup_merge

Category: Publishing Platform

Subject: Pipe Execution

• Article last edited: 10/8/2013