file-20 slow with large xml input file

Description

We recently changed the format for file-20 input from flat to xml. It's working fine, except that the run time for the xml file is about 8-10 times slower that for the flat file. Any ideas as to why this would be? 

The processing speed for xml files seems to be directly related to the size of the xml file. So, 
•processing my full file (about 90,000 - 100,000 records) takes about 8-9 minutes per 1000 records;
•processing a file of about 20,000 records takes about 3-3.5 minutes per 1000 records; and
•processing a file of 5000 or less takes about 1-1.5 minutes per 1000 records, similar to the speed for (any size) flat file.

Resolution

The cause of this has not been determined.  The solution is to break the input file into smaller pieces.

• Article last edited: 8-Mar-2018