[mod_python] Caching issues

> You should be able to add multiple directories and so long as a single
> Python module code file, should be found even when using 'import'.

I tried adding my 'subDir'' to the path, so that it looks like this:
    PythonOption mod_python.importer.path "['~' , '~/include/']"
My 'import' keyword attempts still failed when trying to import across
this 'include' folder into a specific module file residing there.

I simplified the experiment down to the above importer.path, and just
./A.py which uses a keyword 'import' to try to grab C.py from this
'include' folder.  This had surprisingly still lead to failure to find
the module.

> 2009/1/13 Tim Valenta <tonightslastsong at gmail.com>:
>> Thank you for the links-- they prove useful.
>>
>> Capitalization shouldn't be an issue here... When I began learning
>> Unix a good while back, I quickly cast off relying on the OS to
>> 'figure it out'.  I've been keeping everything case sensitive.
>>
>> Backing up to something foundational: I seem to be able to use the
>> import_module() method without problems now, but practically any use
>> of standard python 'import'/'from' keywords fails.  The only success I
>> get is when the desired module is in the same directory as the one
>> trying to use the 'import' keyword.  Does this stem from the fact that
>> import_module isn't look for packages, but rather specific files?  Is
>> it for that reason that I can't use
>>    from subDir import C
>> or, the equally broken,
>>    import subDir.C
>
> Packages not supported with mod_python module importer, so these fall
> through import_module and uses standard Python import mechanism.
>
>> Jiggering around with adding more entries in importer.path to my
>> subdirectories doesn't seem to have any effect.
>
> You should be able to add multiple directories and so long as a single
> Python module code file, should be found even when using 'import'.
>
> Graham
>
>> If the 'import'
>> keyword really is bound to it's working directory only, I will likely
>> in fact have to use import_module() is several places, rather than
>> just at the head of the request (./A.py)
>>
>> Despite my general lack of success, I appreciate the comments you've
>> made along the way.
>>
>> Tim
>>
>> On Mon, Jan 12, 2009 at 10:40 PM, Graham Dumpleton
>> <graham.dumpleton at gmail.com> wrote:
>>> 2009/1/13 Tim Valenta <tonightslastsong at gmail.com>:
>>>>> You just need to ensure you set mod_python's own module search path,
>>>>> ie. mod_python.importer.path
>>>>
>>>> And yet, as I've now removed my www directory from my PythonPath, I
>>>> actually get limited functionality out of the mod_python imports.
>>>> I've got my apache conf file globally applying the option:
>>>>
>>>>        PythonOption mod_python.importer.path "['~']"
>>>> Also have tried using an absolute path, which is producing the same
>>>> functionality:
>>>>        PythonOption mod_python.importer.path "['V:/www/']"
>>>>
>>>> My main module ('A') is requested by Apache in my browser, which uses
>>>> import_module to find and import file B.  File B is in the same
>>>> directory as A, and they only way it was found was through my apache
>>>> conf noted above.  This part works just fine, but two cases develop
>>>> from here:
>>>>
>>>> 1) If I use module B to import a third module 'C', I can't seem to
>>>> make it find module C if it's in a subdirectory.  For instance, doing
>>>> either 'import subDirectory.moduleC' or doing
>>>> 'apache.import_module("subDirectory/moduleC")' results in a failure to
>>>> find the module.  Note that in this case of subdirectories, I'm
>>>> intentionally avoiding absolute paths to reach my modules.  I only
>>>> want to get absolute in my Apache conf.  On the flip side, if module C
>>>> is in the same local directory as A and B, then mod_python finds
>>>> module C just fine.
>>>
>>> Try:
>>>
>>>  apache.import_module("./subDirectory/moduleC.py")
>>>
>>>> 2 ) If I place C in the same directory as A and B, so that mod_python
>>>> finds it, I tried using the natural Python 'import' keyword in B to
>>>> import C.  According to what I understand, this should be fine for
>>>> auto-reloading, since A imports B via import_module, and so then B can
>>>> use either importing strategy to get C in the picture because
>>>> mod_python uses import_module behind the scenes anyway.  In this
>>>> setup, my caching issue remains, and changes to C are definitely not
>>>> reflected in apache until a server restart.  Again note that it
>>>> doesn't matter if I make B use the 'import' keyword or the
>>>> apache.import_module() method.  Both result in failure to reload.
>>>>
>>>> I can't seem to diagnose what's even wrong, since the only reason
>>>> module A, B, or C is being found in the first place is because of my
>>>> importer.path variable is set in httpd.conf.  I've been using
>>>> practically empty files to test this.  Each file contains no more than
>>>> 5 lines to get the modules imported and then to write out a
>>>> confirmation to the screen that it worked.
>>>>
>>>> Wisdom to offer?
>>>
>>> Yes, from memory the module importer isn't properly case insensitive
>>> on Windows or MacOS X where file systems is case insensitive. Thus,
>>> make sure your imports match exactly the case of the directories/files
>>> in the file system.
>>>
>>> Also, use:
>>>
>>>  from mod_python import apache
>>>  apache.log_error('__name__ = %s' % repr(__name__))
>>>  apache.log_error('__file__ = %s' % repr(__file__))
>>>
>>> in modules so that information about imported files is logged as
>>> imported. This will tell you which files are being imported.
>>>
>>> With PythonDebug On you should also see messages in Apache error logs
>>> describing when they are being imported, initially and when changed.
>>>
>>> You can also write some code which extracts out of the mod_python
>>> importer cache the details of everything that has been loaded and the
>>> relationships. The function to call is:
>>>
>>>  from mod_python import apache
>>>  data = apache.request_modules_graph()
>>>
>>> write that to a file and run:
>>>
>>>  http://www.graphviz.org/
>>>  http://www.graphviz.org/Download_windows.php
>>>
>>> on it.
>>>
>>> Graham
>>>
>>>>> Now you do realise you don't need to convert 'import' to
>>>>> import_module() everywhere?
>>>>>
>>>>> The 'import' directive when used inside a file which was already
>>>>> managed by mod_python calls import_module() internally anyway.
>>>>>
>>>>> You just need to ensure you set mod_python's own module search path,
>>>>> ie. mod_python.importer.path
>>>>>
>>>>> Graham
>>>>>
>>>>> 2009/1/12 Tim Valenta <tonightslastsong at gmail.com>:
>>>>> > Okay, I think I'm on the same page as you now-- I've been playing
>>>>> > around with it and it seems that when I leave my PythonPath to find my
>>>>> > import modules, I definitely get no auto reloading goodness, while
>>>>> > when I specify the root path to find them as files, rather than as
>>>>> > modules (as explained in that documentation you linked to), things
>>>>> > seem to work as expected.
>>>>> >
>>>>> > I wasn't getting any warnings in my apache log, and after putting
>>>>> > together the details, I shouldn't be expecting any of that behavior
>>>>> > with two of the same module in memory.  My issue seems to be simply in
>>>>> > avoiding letting my PythonPath find my code.
>>>>> >
>>>>> > Thank you much,
>>>>> > Tim
>>>>> >
>>>>> > On Sun, Jan 11, 2009 at 9:39 PM, Graham Dumpleton
>>>>> > <graham.dumpleton at gmail.com> wrote:
>>>>> >> 2009/1/12 Tim Valenta <tonightslastsong at gmail.com>:
>>>>> >>> Okay, that makes sense.  My PythonPath system var definitely includes
>>>>> >>> my www directory.
>>>>> >>
>>>>> >> That in itself may not be enough to cause problem. You should actually
>>>>> >> see a warning in Apache error log complaining you have done this. That
>>>>> >> is, overlapped sys.path with directories that mod_python module
>>>>> >> importer uses.
>>>>> >>
>>>>> >> The real problem with doing this overlapping is that a module which
>>>>> >> wasn't one managed by mod_python, ie., standard Python module, imports
>>>>> >> something from the www directory, which also happens to be a handler
>>>>> >> module, there will actually be too copies in memory and so can give
>>>>> >> unexpected results.
>>>>> >>
>>>>> >> Graham
>>>>> >>
>>>>> >>> I had been using a forced path in my apache conf
>>>>> >>> file to point to '~', for simple reference for the time being.
>>>>> >>>
>>>>> >>> I think my problem is simply that the modules I'm trying to import
>>>>> >>> aren't 'candidates' for reloading.  Other than that, all other modules
>>>>> >>> should already have been chained together with these
>>>>> >>> apache.load_module calls.  My test case was pretty simple-- just two
>>>>> >>> scripts, the first loading the second.
>>>>> >>>
>>>>> >>> Tim
>>>>> >>>
>>>>> >>> On Sun, Jan 11, 2009 at 8:25 PM, Graham Dumpleton
>>>>> >>> <graham.dumpleton at gmail.com> wrote:
>>>>> >>>> 2009/1/12 Tim Valenta <tonightslastsong at gmail.com>:
>>>>> >>>>> Apologies for more trouble on the caching issue-- I've been adapting
>>>>> >>>>> some of my scripts over to this apache.load_module function, but I
>>>>> >>>>> don't think I'm seeing resolution to the issue.  If I've read the
>>>>> >>>>> documentation properly, it seems that the PythonOption for
>>>>> >>>>> auto-reloading is set to On by default, so to be clear, I haven't put
>>>>> >>>>> a line for that in my httpd.conf file.  Are there any other tricks, or
>>>>> >>>>> have I simply misread the documentation?  I still have to restart the
>>>>> >>>>> Apache service in order to push the changes.
>>>>> >>>>
>>>>> >>>> Reloading does not work for any Python modules/packages installed in
>>>>> >>>> standard Python module search path. Thus, if you have set PythonPath
>>>>> >>>> to some directory where you store your modules, or if they are in
>>>>> >>>> system site-packages directory, they are not candidates for reloading.
>>>>> >>>>
>>>>> >>>> Where are the modules you are modifying and expecting to be reloading
>>>>> >>>> located? Is that location on sys.path, or have you explicitly setup
>>>>> >>>> mod_python module importer path to tell it where they are?
>>>>> >>>>
>>>>> >>>> From within running application, print out __name__ from the module
>>>>> >>>> you have loaded and are expecting to be reloadable, What is it? Does
>>>>> >>>> it look like a normal module name or something magic?
>>>>> >>>>
>>>>> >>>> How was that module imported? There needs to be a chain from
>>>>> >>>> mod_python handler for imports right down through to all modules you
>>>>> >>>> want reloadable. If there aren't, for example, you are using
>>>>> >>>> import_module() from a normal Python module which isn't a candidate
>>>>> >>>> for reloading, you will not see the depth checking for reloading
>>>>> >>>> occurring as you might expect.
>>>>> >>>>
>>>>> >>>> Graham
>>>>> >>>>
>>>>> >>>>> Tim
>>>>> >>>>>
>>>>> >>>>> On Sun, Jan 11, 2009 at 2:32 PM, Tim Valenta <tonightslastsong at gmail.com> wrote:
>>>>> >>>>>> Despite the jab at Windows, the site will be deployed on a Unix
>>>>> >>>>>> machine.  I just prefer Windows for ease of use, especially since my
>>>>> >>>>>> wife has zero experience on a unix terminal or in the dearth of truly
>>>>> >>>>>> professional software found on the Unix/Linux platform.  So yes, in
>>>>> >>>>>> the end it will be on a "real" operating system,  But for now I'm
>>>>> >>>>>> willingly sticking to a more user-friendly environment for the simple
>>>>> >>>>>> folk.
>>>>> >>>>>>
>>>>> >>>>>> Given the characteristics of PHP that allow it to circumvent this
>>>>> >>>>>> caching stuff, I still think a nice little friendly red-flag reminder
>>>>> >>>>>> on the main mod_python tutorial should point out this limitation of an
>>>>> >>>>>> Apache module like this, for those of us like me, who would have gone
>>>>> >>>>>> many months more before having every accidentally found this manual
>>>>> >>>>>> import method.
>>>>> >>>>>>
>>>>> >>>>>> On Sun, Jan 11, 2009 at 2:23 PM, Graham Dumpleton