publish-06 super-slow when run with multiple processes

Description

As the final step in republishing our (7 million) bib records, we ran the publish-06 Service with 32 processes.  The job "stalled" after processing about 500,000.  (No cycles in the ./p_publish_06_PRIMO-FULL.cycles file ever showed as completed and, though the job never canceled, the only activity was very minor activity seen in the ./PRIMO-FULL/tmp directory.)

Note:  The ./xxx01/prof_library has "p_publish_06_loop_length 200000".

Resolution

Though running the publish-06 with 8 processes showed some improvement, it still eventually stalled.  It was only when the number of processes was reduced to 4, that it was able to successfully complete all of the cycles.

It may be that, if the loop_length is smaller (such as the default 50,000), more processes could run at the same time without a problem(?)  

• Article last edited: 24-Jan-2019