While working on PySoy I was really disappointed by the policy adopted by the setup.py script, anytime I launched it the result was a recompilation of *all* the sources, this was really annoying and slow.

What came after was just that I decided to hack it a bit and make it behave more like a standard build tool, that is to control the modification time of a source file in order to choose whether it is updated or need a fresh compilation.

The new policy for the script is very simple, but useful enough to save plenty of time.
It is aware of the following cases (thank you Arc for tips on how to deal with the last one):

• a .c file is missing, pyrex should compile the .pyx file
• a .pyx file is newer than the corresponding .c file, an update is needed
• a .pxd file is newer than any .pyx file, a global recompile is needed

The last point is not optimal, of course, but it’s a lot simpler than specifying all the .pxd dependecies for any .pyx file, and, anyway, quite close to being optimal, because of the thick web of cross dependencies which actually exists in PySoy.

Here is the piece of code which performs the magic:

# Convert Pyrex sources to C if using Trunk
if version == 'Trunk' :
  import os
  from stat import *
  from Pyrex.Compiler import Main

  options = Main.CompilationOptions(defaults=Main.default_options, include_path=include_path)

  newestpxd = 0
  for dir in include_path:
    for pxdsource in os.listdir(dir):
      pxdsource_path = (os.path.join(dir, pxdsource))

      if os.path.isfile(pxdsource_path) and pxdsource.rsplit('.', 1)[1] == 'pxd':
        if os.stat(pxdsource_path)[ST_MTIME] > newestpxd:
          newestpxd = os.stat(pxdsource_path)[ST_MTIME]

  for pyxsource in pyrex_sources:
    compilation_needed = False

    if os.path.isfile(pyxsource.rsplit('.', 1)[0] + '.c'):
      ctime = os.stat(pyxsource.rsplit('.', 1)[0] + '.c')[ST_MTIME]
    else:
      ctime = 0

    if newestpxd > ctime:
      compilation_needed = True
    elif os.stat(pyxsource)[ST_MTIME] > ctime:
      compilation_needed = True

    if compilation_needed:
      Main.compile(pyxsource, options)

Well, actually Arc commited r65 too, simplifying the conditionals of the script even more, but this is another story. 🙂
Anyway, I hope to have made another little step into making the life of our team a bit simpler. 😉