|
Aseem Mohanty
aseem at metaweb.com
Thu Jun 28 17:40:16 EDT 2007
Jim Gallacher wrote: > I've confirmed that mod_python leaks on graceful restart, but I don't > see the dependency on the auth* modules. Apache always seems to leak > when mod_python is loaded, regardless of the other modules. My test > setup is a little different: On Debian with apache 2.2.3, mpm-worker, > mod_python 3.3.1 running within VMWare. Hmm. The reason I felt that the dependency existed was the difference in the size of the leak when the two mod_auth* modules were loaded. In addition, the use of mod_authz_host alone did not leak at all, which led me to this conclusion. I will try out your config and see if the leak size remains the same as just using mod_python only. > > My gut tells me that this may be related to > http://issues.apache.org/jira/browse/MODPYTHON-109. > > This issue also reminds me of > http://issues.apache.org/jira/browse/MODPYTHON-195. > > I think at this point we should log this in JIRA and move the discussion > over to python-dev. > > The obvious (but suboptimal) temporary solution for Aseem is to avoid > using graceful restart. They sure look like good candidates, so should I be looking at mod_wsgi in daemon mode (we are serving WSGI apps via an adapter). The reason I ask is that we need the graceful restarts and cannot really do without it for now. Thanks. AM > > Jim > > Aseem Mohanty wrote: >> Graham Dumpleton wrote: >>> Is the version of mod_python you are using one you compiled yourself, >>> or one from Ubuntu packages? >> >> I have used both and it does not seem to matter. When compiling from >> source, I did the following: >> ./configure --prefix=/opt --with-apxs=/opt/share/apache2/bin/apxs >> make install_dso >> to use the apache build from source. Apache was build with the >> following config: >> ./configure --prefix=/opt --enable-modules=all >> --enable-mods-shared=all --enable-so --with-included-apr >> >>> >>> Is the version of Python you are using one you compiled yourself, or >>> one from Ubuntu packages? >> >> Python-2.5.1 is the one from the ubuntu packages.05:22 PM $ python >> Python 2.5.1 (r251:54863, May 2 2007, 16:56:35) >> [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >> >>> >>> What is the size of mod_python.so? Does it use a shared Python library >>> or static? Use 'ldd' on mod_python.so to see dependencies. >> >> mod_python from ubuntu packages: >> v 3.2.10 >> 05:24 PM $ ll mod_python.so >> -rw-r--r-- 1 root root 99K 2007-01-16 04:44 mod_python.so >> 05:24 PM $ ldd mod_python.so >> linux-gate.so.1 => (0xffffe000) >> libpython2.5.so.1.0 => /usr/lib/libpython2.5.so.1.0 (0xb7e4f000) >> libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 >> (0xb7e38000) >> libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7e33000) >> libutil.so.1 => /lib/tls/i686/cmov/libutil.so.1 (0xb7e2f000) >> libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7e08000) >> libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7cc7000) >> /lib/ld-linux.so.2 (0x80000000) >> >> mod_python built from source: >> v 3.3.1 >> 05:25 PM $ ll mod_python.so >> -rwxr-xr-x 1 aseem aseem 343K 2007-06-27 09:50 mod_python.so* >> aseem @ narada: /opt/share/apache2/modules >> 05:25 PM $ ldd mod_python.so >> linux-gate.so.1 => (0xffffe000) >> libpython2.5.so.1.0 => /usr/lib/libpython2.5.so.1.0 (0xb7e30000) >> libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 >> (0xb7e19000) >> libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb7e14000) >> libutil.so.1 => /lib/tls/i686/cmov/libutil.so.1 (0xb7e10000) >> libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7de9000) >> libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7ca8000) >> /lib/ld-linux.so.2 (0x80000000) >> >> In both cases they do seem to use shared python libs. >> >> Thanks. >> AM >> >>> >>> Graham >>> >>> On 28/06/07, Aseem Mohanty <aseem at metaweb.com> wrote: >>>> Yes, hence the exotic phraseology. Results for the various >>>> combinations are below: >>>> >>>> 1. mod_python only, no mod_auth* modules: leaks >>>> 2. mod_auth_digest only, no other mod_auth*, no mod_python: does not >>>> leak >>>> 3. mod_authz_host only, no other mod_auth*, no mod_python: does not >>>> leak >>>> 4. mod_python + mod_authz_host, no mod_auth_digest: leaks >>>> 5. mod_python + mod_auth_digest, no mod_authz_host: leaks < >>>> (1) or (4) >>>> 6. mod_python + mod_auth_digest + mod_authz_host: does not >>>> leak >>>> >>>> NOTE: 'does not leak' implies ~50-100K leakage per graceful restart >>>> (5) leaks ~500-600K per graceful restart >>>> zero load on server >>>> >>>> As far as I can tell there is definitely some sinister interaction >>>> going on >>>> between these three modules. Unfortunately not being conversant with >>>> apache >>>> module development, I cannot really say what. >>>> >>>> Thanks. >>>> AM >>>> >>>> >>>> Jim Gallacher wrote: >>>> > Aseem Mohanty wrote: >>>> >> This may sound quite exotic >>>> > >>>> > >>>> > Why yes, it does? Interaction between these modules resulting in a >>>> > memory leak sure sounds weird. Did you try it with *only* mod_python >>>> > loaded? (ie, without mod_authz_host) >>>> > >>>> > Jim >>>> > >>>> >> but here it is: >>>> >> >>>> >> In a vanilla apache2.2.x configuration if I only load mod_authz_host >>>> >> and none of >>>> >> the other mod_auth* modules, every graceful restart causes ~1Mb leak >>>> >> in the >>>> >> parent process. This is with mod_python loaded. Without >>>> mod_python the >>>> >> memory profile remains unchanged. >>>> >> >>>> >> Steps to reproduce: >>>> >> Gentoo: >>>> >> 1. In /etc/apache2/httpd.conf comment out all "LoadModule auth*" >>>> >> directives except >>>> >> LoadModule authz_host_module modules/mod_authz_host.so >>>> >> >>>> >> Ubuntu Feisty (repos): >>>> >> 1. In /etc/apache2/mods-enabled rename auth*.load files except >>>> >> authz_host.load >>>> >> to auth*.load.not >>>> >> >>>> >> Ubuntu Feisty (httpd built from source): >>>> >> 1. In conf/httpd.conf comment out all "LoadModule auth*" directives >>>> >> except >>>> >> LoadModule authz_host_module modules/mod_authz_host.so >>>> >> >>>> >> 2. Start apache >>>> >> 3. Track apache parent process: top -d1 -p $(pgrep -o apache/httpd) >>>> >> 4. Hit apache instances with graceful restarts: >>>> >> for i in $(seq 150); do echo $i; kill -USR1 $(pgrep -o >>>> apache/httpd); >>>> >> sleep 2; done >>>> >> >>>> >> In this case the parent will leak ~1Mb per restart. If however in >>>> each >>>> |