Stumped

I know this isn't (well, probably not anyway) directly VM related but I'll try to explain my issue with hopes that you can at least point me in the right direction. I've spent hours so far searching the web for specific info but not able to find.

Scenairo:
I am getting fcgid timeouts and subsequent apache errors about Connection reset by peer. I'm doing the following in a php script:
1) Read a 70MB XML file
2) Parse it
3) for each item parsed...
a) set_time_limit(10)
b) write it to my database
c) using an open ftp connection, get an image (could be big too) and write it to a different site (the ftp connections (2) are persistent within the script)

I ran into several problems:
1) Always got a timeout when trying to read the 70MB XML file via curl() or via file_get_contents(). But was able to load on my PC and then use a multipart/form-data to bring in the XML for parsing. So that one is solved.
2) Had to take the PHP memory limit up to 2048M to handle the XML parsing and associated arrays it creates. But that's working now too.
3) I have changed the following parameters within the for this site:
IPCConnectTimeout 10
ProcessLifeTime 7200
IdleTimeout 7200
BusyTimeout 3600

I get through about 400 (total approx 10000) of the parsed items before I get the following in the error_log:
[Sun Apr 05 04:18:53 2009] [error] [client 206.72.99.10] Premature end of script headers: tool-yahooCatalogConvert.php, referer: http://test.ez-om.com/om/Tests/tool-yahooCatalogConvert.php

[Sun Apr 05 04:18:53 2009] [warn] (104)Connection reset by peer: mod_fcgid: read data from fastcgi server error.

I'm guessing that I'm now running into some kind of system related inactivity timeout. I.e. apache/fcgid is not seeing any I/O to the client (browser) during all this processing and is timing out. Seems to happen after about 5 minutes of wall-clock time. CPU time for the php process is in the 2 minute area and I believe the set_time_limit(10) within the loop should be quite sufficient (plus I don't get the maximum execution time exceeded message).

So after this droning.... My question is do you know how to enable a php script launched from a browser to run for a really long duration without getting timeouts? The stuff actually works, it just takes a while. And yes, I'm using most of that 2GB of memory allocation due to the joys of XML.

I was hoping to modify the fcgid wrapper to specify the timeout within fcgid itself but don't know what options to pass to php-cgi to enable. Everthing I read seems geared toward Apache and are mostly based on stand-alone fcgid versus virtual.

Thoughts? Joe, maybe you need something to wake you up today? :-)

Status: 
Closed (fixed)