Lion, wxPython and py2app: targeting Carbon & Cocoa

About this time last year I wrote a blog entry on installing wxPython and py2app on my Mac running Snow Leopard. Well I have since upgraded to Lion and the same installation seemed to keep working just fine. This weekend I've actually upgraded to a new Mac and I thought this would be an excellent opportunity to grapple with this installation again and perhaps advance my understanding of what I'm doing.

Apple's Migration Assistant (a.k.a. Setup Assistant) had other ideas. You have to hand it to them, it took about 2-3 hours with the two machines plugged together and everything was copied across and working without any intervention. They really do make it easy to buy a new Mac and get productive straight away.

So is this blog post the Python equivalent of the "pass" statement? Just a no-op?

Well not quite. At the moment, I'm building my QTI migration tool using the Carbon version of wxPython which means forcing Python to work in 32bit mode. That's getting a bit out-dated now and I surely can't take advantage of my new 8GB Mac? I need to embrace 64bit builds, I need to figure out how to build for the Cocoa version of wxPython and I need to figure out how to do this while retaining my ability to create the 32-bit build for older hardware/versions of Mac OS.

So here is how I now recommend doing this...

Step 1: Install Python

The lesson from last time was that you can't rely on the python versions installed by Apple to do these builds for you. If you run python on a clean Lion install you'll get a 64-bit version of python 2.7.1:

$ which python
/usr/bin/python
$ python
Python 2.7.1 (r271:86832, Jul 31 2011, 19:30:53) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;"%X"%sys.maxsize
'7FFFFFFFFFFFFFFF'

Thanks to How to tell if my python shell is executing in 32bit or 64bit mode? for the tip on printing maxsize.

I want to keep python pointing here because the command-line version of the migration tool, and the supporting Pyslet package (which can be used independently) need to work 'out-of-the-box'. The Migration Assistant had helpfully copied over my .bash_profile which included modifications made by my custom Mac binary install of Python 2.7 last year. The modifications are well commented and help to explain the different paths we'll be dealing with:

# Setting PATH for Python 2.7
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH

Firstly, note that the Mac binaries install in /Library/Frameworks/ whereas the pre-loaded Apple installations are all in /System/Library/Frameworks/. This is a fairly subtle difference so a little bit of concentration is required to prevent mistakes. Anyway, as per the instructions above I restored my .bash_profile from .bash_profile.pysave and confirmed (as above) that I was getting the Apple python.

It seems like 2.7.3 is the latest version available as a Mac binary build from the main python download page. This will make it a bit easier to check I'm running the right interpreter! So I downloaded the dmg from the following link and ran the installer: http://www.python.org/ftp/python/2.7.3/python-2.7.3-macosx10.6.dmg. For me this was an upgrade rather than a clean install. The resulting binaries are put on the path in /usr/local/bin. By default, the interpreter runs in 64bit mode but it can be invoked in 32-bit mode too:

$ /usr/local/bin/python
Python 2.7.3 (v2.7.3:70274d53c1dd, Apr  9 2012, 20:52:43) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;"%X"%sys.maxsize
'7FFFFFFFFFFFFFFF'

$ which python-32
/usr/local/bin/python-32
$ python-32
Python 2.7.3 (v2.7.3:70274d53c1dd, Apr  9 2012, 20:52:43) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;"%X"%sys.maxsize
'7FFFFFFF'

This might be enough, but the wxPython home page does recommend that you use different python installations if you want to run both the Carbon and Cocoa versions. So I'll repeat the installation with a python 2.6 build. The current binary build is 2.6.6, this is missing some important security fixes that have been included in 2.6.8 but it looks safe for the migration tool. I downloaded the 2.6 installer from here. When I ran the installer I made sure I only installed the framework, I don't want everything else getting in the way.

I now have a python 2.6 installation in /Library/Frameworks/Python.framework/Versions/2.6

$ /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
Python 2.6.6 (r266:84374, Aug 31 2010, 11:00:51) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys;"%X"%sys.maxsize
'7FFFFFFF'

Notice that this is a 32-bit build. Apple actually ship a 64-bit build of 2.6.7 so care will be needed, typing python2.6 at the terminal will not bring up this new installation.

To make life a little bit easier I always create a bin directory in my home directory and add it to the path in my .bash_profile using lines like these:

PATH=~/bin:${PATH}
export PATH

This is going to come in very handy in the next step.

Step 2: setuptools and easy_install

setuptools is required by lots of Python packages, it is designed to make your life very easy but it takes a bit of fiddling to get it working with these custom installations. It's an egg which means it runs magically from the command line, I'll show you the process of installing it on python 2.6 but the instructions for putting it in 2.7 are almost identical (it's just a different egg).

I downloaded the egg from here: http://pypi.python.org/packages/2.6/s/setuptools/setuptools-0.6c11-py2.6.egg#md5=bfa92100bd772d5a213eedd356d64086 and then took a peek at the top of the script:

$ head -n 8 setuptools-0.6c11-py2.6.egg 
#!/bin/sh
if [ `basename $0` = "setuptools-0.6c11-py2.6.egg" ]
then exec python2.6 -c "import sys, os; sys.path.insert(0, os.path.abspath('$0')); from setuptools.command.easy_install import bootstrap; sys.exit(bootstrap())" "$@"
else
  echo $0 is not the correct name for this egg file.
  echo Please rename it back to setuptools-0.6c11-py2.6.egg and try again.
  exec false
fi

I've only shown the top 8 lines here, the rest is binary encoded gibberish. The thing to notice is that, on line 3 it invokes python2.6 directly so if I want to control which python installation setuptools is installed for I need to ensure that python2.6 invokes the correct interpreter. That's where my local bin directory and path manipulation comes in handy.

$ cd ~/bin
$ ln -s /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 python2.6
$ ln -s /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.7 python2.7

Now for me, and anyone who inherits my $PATH, invoking python2.6 will start my custom MacPython install.

$ sudo -l python2.6
/Users/swl10/bin/python2.6

Fortunately sudo is configured to inherit my environment. It was worth checking as this is configurable. I can now install setuptools from the egg:

$ sudo sh setuptools-0.6c11-py2.6.egg 
Password: [I had to type my root password here]
Processing setuptools-0.6c11-py2.6.egg
Copying setuptools-0.6c11-py2.6.egg to /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages
...

I did the same for python 2.7 (note that you need a different egg) and then added links to easy_install to my bin directory:

$ cd ~/bin
ln -s /Library/Frameworks/Python.framework/Versions/2.7/bin/easy_install easy_install-2.7
ln -s /Library/Frameworks/Python.framework/Versions/2.6/bin/easy_install easy_install-2.6

Step 3: wxPython

wxPython is a binary installer tied to a particular python version. However, I believe it uses a scatter gun approach to search for python installations and will install itself everywhere with a single click. That is the reason why it is better to run completely different versions of python if you want completely different versions of wxPython. In fact, if you find yourself with multiple installs there is a wiki page that explains how to switch between versions. But a little playing reveals that this refers to Major.Minor version numbers. It can't cope with the subtlety of switching between builds or switching between Carbon and Cocoa as far as I can tell so this won't help us.

My plan is to install the Carbon wxPython (which is 32bit only) for python 2.6 and the newer Cocoa wxPython for python 2.7. The wxPython download page has a stable and unstable version but to get Cocoa I'll need to use the unstable version. The stability referred to is that of the API, rather than the quality of the code. Being cautious I downloaded the stable 2.8 (Carbon) installer for python 2.6 and the unstable 2.9 Cocoa installer for python 2.7. Installation is easy but look out for a useful script on the disk image which allows you to review and manage your installations. To invoke the script you can just run it from the command line:

$ /Volumes/wxPython2.9-osx-2.9.3.1-cocoa-py2.7/uninstall_wxPython.py

When I was done with the installations it reported the following configurations as being present:

  1.  wxPython2.8-osx-unicode-universal-py2.6     2.8.12.1
  2.  wxPython2.9-osx-cocoa-py2.7                 2.9.3.1

(If, like me, you are upgrading from previous installations you may have to clean up older builds here.) At this point I tested my wxPython based programs and confirmed that they were working OK. I was impressed that the Cocoa version seems to work unchanged.

Step 4: py2app

With the groundwork done right, the last step is very simple. The symlinks we put in for easy_install make it easy to install py2app.

$ sudo easy_install-2.6 -U py2app
Password: [type your root password here]
Searching for py2app
Reading http://pypi.python.org/simple/py2app/
Reading http://undefined.org/python/#py2app
Reading http://bitbucket.org/ronaldoussoren/py2app
Best match: py2app 0.6.4

...[snip]...

Installed /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/altgraph-0.9-py2.6.egg
Finished processing dependencies for py2app

The plot is very similar for python 2.7.

I've now added the following to my setup script:

import wx
if "cocoa" in wx.version():
  suffix="-Cocoa"
else:
  suffix="-Carbon"

The suffix is then appended to the name passed to the setup call itself. The following command results in a 32bit, Carbon binary, compatible with OS X 10.3 onwards.

$ python2.6 setup.py py2app

While this command creates a Cocoa based 64bit binary for 10.5 and later.

$ python2.7 setup.py py2app

And that is how to target both Carbon and Cocoa in your wxPython projects.