Tag Archives: Python

PacStats revamped, enhanced and published

Many months ago I was dealing with OpenGL, shaders and C++, the usual tools of a graphics programmer wannabe. πŸ™‚
This time I’m writing about something completely different, the recent developments of PacStats.

The new PacStats logo

PacStats is a 2007 project that was born as a toy experiment borrowing a lot of GL O.B.S. code: it is a program that analyzes the log of pacman, the ArchLinux package manager, and then plots a bunch of statistical charts about its activity.
Have you ever wandered how many packages have a name that begins with the letter “F”, or who is the packager that has contributed the most? With PacStats you can easily answer to those questions. πŸ˜‰

But what have been the main changes since 2007?
Well, first of all I have renewed the code as already done for GL O.B.S., so the GUI is now based on GtkBuilder, Matplotlib imports NumPy and the Python print statements have been converted to functions.
The program have also gained a toon shaded Blender made logo, and the possibility to be installed thanks to distutils through a very simple PKGBUILD that is already on AUR.
While some tables of the database have been refactored, the parsers made more robust and the base/derived chart classes relationship more polished, the most important news for the end users are represented by the possibility to configure the program through the GUI or a text file, the addition of a database information window and a menu item to optimize it and the addition of a couple of new charts and a toolbar to control them.

The Preferences window


The Database Information window

The original project have been hosted for a long time on my personal site, but the efforts spent to reboot it have suggested the possibility to have it published in a more “official” way: as a matter of facts you can find it on Google Code.
I have been interested in this open source project hosting platform for a long time and until now I’m really satisfied by its streamlined yet flexible interface.
I’m also very glad to have employed Mercurial for source revision control, after having been positively struck by the git experience and been advocating distributed systems for quite some time now, my decision to use it instead of Subversion feels very natural. Sure enough it is a really nice tool, easier to grasp than git but capable of most of the things of its famous contender, written in Python but yet not suffering too much when coming to performance.
Just like every other free software I have been responsible for, I have created a project page on Ohloh too, have a look at it and enjoy your charts! πŸ˜‰

GL O.B.S.: two years after

You’re reading about GL O.B.S. after *exactly* 730 days (I swear it was not made on purpose ;-)): today revision 50 has been committed.
No new feature has been added, I’ve dedicated the efforts of the last days to a case study about updating an application after a very long time, the process mainly involved removing deprecated things and adding support for new ones.

GL O.B.S. GUI is now based on GtkBuider

GL O.B.S. GUI is now based on GtkBuider

Let’s read together some lines from the revision log:

  • GUI migrated from libglade to GtkBuilder
    This was, with no doubt, the most time consuming task: gtk-builder-convert tool was not so reliable and I was compelled to recreate the GUI from scratch with Glade. I have also set the minimum GTK version required to 2.16
  • using JSON format for storing benchmark information dictionaries
    This feature is a way to get rid of the deprecated execfile() built-in function, instead of executing a python script to read a dictionary, the Benchmark class now parses a JSON file with the new integrated json module.
    Comparing an old and a new file you can notice how simple it was the conversion, I’ve only lost the ability to comment a line. πŸ˜‰
  • importing numpy instead of numerix in matplotlib
    This is another deprecation related issue, matplotlib is now based upon numpy.
  • matplotlib canvas get refreshed instead of being destroyed and then recreated
    I don’t know why two years ago I was destroying and recreating the object instead of just calling the draw() method every time it had to be refreshed. πŸ˜€
  • some more attributes and objects integrated in the GtkBuilder XML file
    I’ve integrated more attributes in the XML GUI file, like default size for secondary windows or Paned positions.
    Moreover, thanks to GtkBuilder, even some objects have been integrated, like Adjustment or TextBuffer.
    Actually also TreeModel objects should be configurable inside Glade and integrated inside the XML, but I didn’t manage to make them work. πŸ™‚

Other remarkable changes in the field of Python3 future support and deprecated features removal were:

  • all the print statements converted to functions
    Well, we all know the content of PEP 3105… πŸ˜‰
  • using hashlib.md5 instead of md5
    The md5 module has been deprecated in favour of the hashlib one
  • using “key in dict” instead of dict.has_key()
    The has_key() method no longer exists in Python 3
  • using subprocess.Popen instead of os.popen()
    os.popen() have been deprecated in favour of the subprocess module
  • using urllib2.urlopen() instead of urllib.urlopen()
    Another deprecation suggested substitution, from urllib.urlopen() to urllib2.urlopen()
  • removed “copyright” argument and added “classifiers” in setup.py
    The distutils.core.setup() function loses the “copyright” argument but gains “classifiers”. πŸ™‚
  • dropped support for pysqlite2 and old webbrowser module
    Now that the minimum requirement is Python 2.6, there is no need for pysqlite2 or old webbrowser module support
  • I have also removed the deprecated Options class in some SCons scripts.

I’m not sure I will have some time in the future to continue developing, but this was a nice “reboot” and I also have some new ideas in my to-do list. πŸ˜‰

Converting weights to vertex colors

If you have read my LinkedIn profile lately you should already know that, by now, some months have passed since I started my pre-graduation internship activity at Raylight (and since I signed my first NDA πŸ˜‰ ).
The real-time graphic R&D work that I’m doing there for my thesis is enjoying and stimulating, but this is not the topic of the post…

Raylight logo

Some days ago a 3d artist of the team, Alessandro, asked me a script that would have helped him using Blender for one more task along the company asset creation pipeline, weight painting.
He needed a simple script to actually convert vertex weights to per-bone vertex colors layers, in order to bake them to per-bone UV maps and later import them inside 3d Studio Max.

Weight painting

Weight painting

Vertex painting

Vertex painting

At first I didn’t even know where to start, how to extract and match per-vertex weight data with per-face vertex color one, but my second try with it went as smoothly as honey.
The core algorithm is, indeed, very simple:

for f in faces:
  for i,v in enumerate(f):
      infs = me.getVertexInfluences(v.index)
      for vgroup, w in infs:
        me.activeColorLayer = vgroup
        col = f.col[i]
        col.r = col.g = col.b = int(255*w)

Well, he has not yet taken advantage of it nor I know if he will ever use it, nevertheless the script is working and I have shared it on my site, as usual. πŸ˜‰

Composing renders in a strip

First of all happy new year to everyone, then let’s talk about this post topic… πŸ™‚

During these days I was relaxing and practicing subdivision modeling, after a long time away from Blender I was back to the dream of creating a convincing human head model, but my programming side win the day. πŸ˜€
While I was studying in detail some face key parts topology from here, I noticed the PiP-like composed images attached to the first post…

Showing camera keyframes

Last night I was thinking of a way to automates the process and today it becomes reality in the form of a Blender Python script: it is capable of producing an image which is composed of multiple rendered frames, think of a daily comic strip and you understand the name ;).

The user can select which frames to render specifying a string similar to the following one: “1-3, 5, 7, 9-11“.
Moreover it is possible, of course, to choose the size of a single frame and the composed image table dimensions, i.e. how many rows and columns it should have.
Have a look to how well my topology study renders fit the script purpose. πŸ˜‰

The resulting composed image

This second script is a bit more complex than my first one, making use of the Registry module to load and save options and the Draw.PupBlock() method to display a bigger GUI.

Of course it is released under the GNU GPL License and available online, download it from here.

Automatic parallax map generation with Blender

It has been a long time since I last wrote something here, during these months two new things happened that are worth to be mentioned: first of all I’m really close to graduation!
Well, actually I need to pass the last exam and spend a period of at least four months of internship, nothing is sure now but I’m in close contact with a game developing company… πŸ˜‰

The second thing is closely related to this post instead, a couple of months ago I began to convert, following M3xican’s advice, my OpenGL demos to object oriented C++.
What I have now is really not much, nevertheless my class library can load a Stanford PLY model, it is ES 2.0 compliant (this means it will be easily converted to “Pure” OpenGL 3.x), and it can already display both parallax mapping and depth of field!

I’m not going to publish any screenshot by now because I think it’s not the time yet, what I’m showing you is a easy script, my first one, which I wrote yesterday night using the Blender Python API.
What it does is really simple yet time-saving, you select a high-poly and a low-poly model, run the script from the Object->Scripts menu and watch Blender baking your normal and height map and then saving them.

Blender parallax maps

I have also set up an easy compositing nodes configuration to mix the two images in a single parallax map with height data encoded in the alpha channel.

Blender parallax maps nodes

You can download the script from here.
Everything is very simple (and funny!) with the astonishing power of Blender! πŸ™‚

A slighty smarter setup.py for PySoy r64

While working on PySoy I was really disappointed by the policy adopted by the setup.py script, anytime I launched it the result was a recompilation of *all* the sources, this was really annoying and slow.

Python transparent logo

What came after was just that I decided to hack it a bit and make it behave more like a standard build tool, that is to control the modification time of a source file in order to choose whether it is updated or need a fresh compilation.

The new policy for the script is very simple, but useful enough to save plenty of time.
It is aware of the following cases (thank you Arc for tips on how to deal with the last one):

  • a .c file is missing, pyrex should compile the .pyx file
  • a .pyx file is newer than the corresponding .c file, an update is needed
  • a .pxd file is newer than any .pyx file, a global recompile is needed

The last point is not optimal, of course, but it’s a lot simpler than specifying all the .pxd dependecies for any .pyx file, and, anyway, quite close to being optimal, because of the thick web of cross dependencies which actually exists in PySoy.

Here is the piece of code which performs the magic:

# Convert Pyrex sources to C if using Trunk
if version == 'Trunk' :
  import os
  from stat import *
  from Pyrex.Compiler import Main

  options = Main.CompilationOptions(defaults=Main.default_options, include_path=include_path)

  newestpxd = 0
  for dir in include_path:
    for pxdsource in os.listdir(dir):
      pxdsource_path = (os.path.join(dir, pxdsource))

      if os.path.isfile(pxdsource_path) and pxdsource.rsplit('.', 1)[1] == 'pxd':
        if os.stat(pxdsource_path)[ST_MTIME] > newestpxd:
          newestpxd = os.stat(pxdsource_path)[ST_MTIME]

  for pyxsource in pyrex_sources:
    compilation_needed = False

    if os.path.isfile(pyxsource.rsplit('.', 1)[0] + '.c'):
      ctime = os.stat(pyxsource.rsplit('.', 1)[0] + '.c')[ST_MTIME]
    else:
      ctime = 0

    if newestpxd > ctime:
      compilation_needed = True
    elif os.stat(pyxsource)[ST_MTIME] > ctime:
      compilation_needed = True

    if compilation_needed:
      Main.compile(pyxsource, options)

Well, actually Arc commited r65 too, simplifying the conditionals of the script even more, but this is another story. πŸ™‚
Anyway, I hope to have made another little step into making the life of our team a bit simpler. πŸ˜‰

My Summer of Code begins with PySoy r44

Summer of Code has started just today (even if currently it is only a “Spring of Code” πŸ˜€ ) but a little contribution of mine has already made his way inside the SVN repository of PySoy.

But let’s start from the beginning…
After having shown to Arc an early draft of an UML class diagram for the current code I decided to come back to work on some test code I had written in the afternoon.
It was just a classical spinning cube demo to actually compare PyOpenGL versus Pyrex speed, I don’t report the results here because they are quite identical, if I haven’t done any mistake it should be the absolute minimum complexity of the code which actually determined this result.
Anyway, even without this proof I firmly believe in the power and speed of Pyrex. πŸ˜‰

Going back to my commit, my changes affected a small yet important area of the code, I think we will remember it in the future. πŸ™‚

BEFORE
In src/windows-glx/soy.windows.pyx:

gl.glClear(0x4100)

AFTER
In include/gl.pxd:

[...]
# Constants
  ctypedef enum:
    [...]
    # glPush/PopAttrib bits
    GL_DEPTH_BUFFER_BIT
    GL_COLOR_BUFFER_BIT
[...]

In src/windows-glx/soy.windows.pyx:

gl.glClear(gl.GL_COLOR_BUFFER_BIT | gl.GL_DEPTH_BUFFER_BIT)

Was it better before or now? πŸ˜‰

Python 2.5 support in globs r46

Yes, I should have written about it when I actually committed the revision, but I forgot completely. πŸ˜€
About two months after Python 2.5 has been moved to the current repository of Arch I begun to think that maybe it was time to support the new release of our beloved language/interpreter.

Python transparent logo

I don’t know if it can be called “support”, but at least GL O.B.S. is now aware of it. πŸ™‚
The changes are very simple yet of some importance.
First of all, pysqlite is not needed anymore if you have the integrated sqlite3 module:

if sys.version_info[:2] >= (2, 5):
  from sqlite3 import dbapi2 as sqlite
else:
  from pysqlite2 import dbapi2 as sqlite

Moreover I make use of the updated API of the webbrowser module:

if sys.version_info[:2] >= (2, 5):
  webbrowser.open_new_tab(Globs.BROWSE_URL)
else:
  webbrowser.Netscape('firefox').open(Globs.BROWSE_URL)

Another addition, not related with the support of Python 2.5, is the check_ver function, which checks if a particular version of OpenGL is available on the machine running GL O.B.S., this have opened the possibility to add an OpenGL 2 only test like GLSL_Parallax into benchmarks r47.

Python and GL O.B.S. are getting better and better. πŸ˜‰

Accepted for Summer of Code

Dear Applicant,
Congratulations! This email is being sent to inform you that your
application was accepted to take part in the Summer of Code.

Yeah, one of my two proposals has been accepted!

My first proposal was about working on GL O.B.S. under the Python Software Foundation, unfortunately it was very likely going to be discarded.
I learned this from a mentor who contacted me, he wrote that my application was based on a personal program and that it would have been hard to find someone to mentor me, moreover I wouldn’t have contributed to the Python community. He also added that I could have been a good candidate for his project, he is, indeed, Arc Riley, Project Manager of PySoy.
And so I did, I wrote another application and, this time, it has been accepted. πŸ™‚

My work will be to integrate multi-texturing in the PySoy rendering loop and API, document API additions, test the whole under many different free software drivers and then implement some related techniques, like bump or normal mapping.

I’m really glad of this opportunity, I will learn many interesting OpenGL and Python topics and I will improve my design, teamwork and communication skills.
Thank you Google! πŸ˜‰