Lately I’ve been working in virtualenvs which as been great for developing but not so great for installing. I’ve run into numerous issues where I prepend my
PYTHONPATH with the directory I want to get imported first, but to no avail. You’ve run into this: you
export PYTHONPATH exactly as you want, only to see that
import sys; sys.path includes all kinds of other junk before it!
So, after watching a talk about reverse-engineering
virtualenv. I realized I actually don’t understand anything about what python does at startup. So after a lot of searching, I found out that to load packages, Python reads the
site.py file. I read about
site.py. And I found out…..
TURNS OUT PYTHON SECRETLY LOADS PACKAGES BEFORE LOOKING AT PYTHONPATH.
And where it looks is the
*.pth files found in
path/to/python/lib/site-packages/*.pth. The biggest culprit is usually
easy_-nstall.pth. Mine had all kinds of absolute paths to the bigger Python install that I had to remove.
On my computer,
easy-install.pth looks like this:
import sys; sys.__plen = len(sys.path) ./pytz-2013.8-py2.7.egg ./brewer2mpl-1.3.2-py2.7.egg ./pybedtools-0.6.2-py2.7-macosx-10.9-x86_64.egg /Users/olga/workspace-git/pandas /Users/olga/workspace-git/prettyplotlib /Users/olga/workspace-git/statsmodels /Users/olga/workspace-git/seaborn ./husl-2.1.0-py2.7.egg /Users/olga/workspace-git/gffutils ./simplejson-3.3.1-py2.7-macosx-10.9-x86_64.egg ./argcomplete-0.6.5-py2.7.egg ./argh-0.23.3-py2.7.egg ./moss-0.2.0-py2.7.egg /Users/olga/workspace-git/scikit-learn /Users/olga/workspace-git/matplotlib/lib ./Theano-0.6.0-py2.7.egg ./pysam-0.7.7-py2.7-macosx-10.9-x86_64.egg /Users/olga/workspace-git/YeoLab/gscripts import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)
So let’s say I never wanted to use my development versions of something, then I’d remove that line from the file. Although eventually this got so much of a problem in my
virtualenv at work that at I added the
site-packages directory of that python distribution indirectly with
import sys; sys.__plen = len(sys.path) ./ ./distribute-0.6.14-py2.7.egg /home/obotvinnik/workspace-git/gscripts /home/obotvinnik/workspace-git/rnaseqlib import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)
Because I always wanted to import from the
virtualenv FIRST and never look at any other packages if I could help it.
I honestly don’t know how kosher this is but it worked for me. Hope it helps you!