[mod_python] mod_python or apache scalability?

I found that this that 

IOError: image file is truncated (14432 bytes not processed)

has to do with using input filter

PythonInputFilter flashfilter FLASHFILTER
SetInputFilter FLASHFILTER

It somehow behaves differently in apache 2.2.6/mod_python 3.3.1 versus
2.0.54/ 3.1.4. For now, I removed this input filter (we can get away without
it), and it seems to work.

However, we have a bigger problem: we now have intermittent import errors
(one error per about 50-80 requests).

This code executed by publisher:

import modules
def _import( module_path ):
    runme = modules[ module_path ]
    #get the last part
    class_name = module_path.split('.')[-1]
    try:
        runme = getattr(runme, class_name ) <--- INTERMITTENT ERROR here
    except AttributeError:
        raise Exception,'Module does not contain class '+class_name+',
runme='+repr(runme)+' module_path='+repr(module_path)

produces this error:

Exception: Module does not contain class getprofile, runme=<module
'getprofile' from '/path/scripts/getprofile.py'> module_path='getprofile'

This used to always work with prefork. Is there anything not thread safe
about importing modules?

> -----Original Message-----
> From: mod_python-bounces at modpython.org [mailto:mod_python-
> bounces at modpython.org] On Behalf Of Graham Dumpleton
> Sent: Monday, November 05, 2007 1:29 PM
> To: Martijn Moeling
> Cc: mod_python at modpython.org; Alec Matusis
> Subject: Re: [mod_python] mod_python or apache scalability?
> 
> The LimitRequestBody directive support in mod_python is broken and
> most likely would have resulted in a 500 error occurring during
> mod_python.publisher processing of the form even before the user code
> was run.
> 
> My guess at the reason is that they have enabled in the new Apache
> server compression on request content. This may not work with
> mod_python because how it reads request content is broken, with it
> only reading up until original Content-Length in certain cases. If
> there is a mutating input filter such as content decompression, you
> will get truncated input data.
> 
> https://issues.apache.org/jira/browse/MODPYTHON-240
> https://issues.apache.org/jira/browse/MODPYTHON-212
> 
> So, OP should ensure that directives to accept compressed request
> content are disabled.
> 
> Graham
> 
> On 05/11/2007, Martijn Moeling <martijn at xs4us.nu> wrote:
> >
> >
> >
> > Hmm it looks to me that you have an upload limit in your apache
> config.
> >
> > It might be the LimitRequestBody directive.
> >
> > Another thing might be that your processing is faster than the data
> comes
> > in, but I doubt that
> > Take a close look at you apache conf, and your apache error log
> (right after
> > an upload or with tail -f /var/log/httpd/error_log (or where ever
> your
> > logdir is)).
> >
> > Martijn
> >
> >
> >  ________________________________
> >  Van: Alec Matusis [mailto:matusis at matusis.com]
> > Verzonden: zo 04.11.2007 21:46
> > Aan: Martijn Moeling
> > CC: mod_python at modpython.org
> > Onderwerp: RE: [mod_python] mod_python or apache scalability?
> >
> >
> >
> >
> > > I had similair problems however, they turned out to be MySQLdb
> > > related and not in a way you suspect
> >
> > Thanks! I will try separating MySQLdb connections. We do pass the
> connection
> > around currently.
> >
> > However, how would you explain this PIL error:
> >
> >     File "/path/publisher/publisher.py", line 78, in handler
> >  flow_instance.dispatch()
> >  File "/path/scripts/updateprofile.py", line 92, in
> > dispatch
> >  im.save(full, self.im_format)
> >     File
> > "/usr/local/encap/Python-2.4.1/lib/python2.4/site-
> packages/PIL/Image.py",
> > line 1272, in save
> >
> >  File
> > "/usr/local/encap/Python-2.4.1/lib/python2.4/site-
> packages/PIL/ImageFile.py"
> > , line 192, in load
> >  IOError: image file is truncated (14432 bytes not processed)
> >
> > This only happened for large images, small ones uploaded fine.
> > The code is
> >
> > def dispatch(self):
> >     pix = self.form['Filedata']
> >     try:
> >         im = Image.open(pix.file)
> >     except IOError:
>