Electricmonk

Ferry Boender

Programmer, DevOpper, Open Source enthusiast.

Blog

Apache, FastCGI and Python

Thursday, February 12th, 2009

FastCGI is a hybrid solution to serving web applications written in a wide variety of programming languages. It sits somewhere between CGI, which spawns a new instance of the web application for each request, and the various web server modules (such as mod_php, mod_python and mod_wsgi) which take care of pre-spawning a pool of interpreters and web applications from within the web server, which will then handle requests.

FastCGI, too, can take care of spawning a pool of web application instances to handle requests. It can also facilitate communications between a web server and an external, already running web application. Unlike CGI, FastCGI does not spawn a new process for each request, and unlike the various web server modules it is not completely embedded within the web server. Instead it uses TCP/IP or Unix sockets to communicate to the web application. This makes it possible to create advanced setups such as spreading out requests over multiple servers, limiting web applications’ rights and system resources using the standard Unix tool-set, etc.

FastCGI is available for a large range of web servers, is fast and is powerful and versatile in its capabilities. It is however not well documented, nor easy to set up. This document covers the basic idea behind FastCGI, setting up FastCGI for Apache (v2) and hooking it up to a simple Python web application. Such a light-weight setup, which requires nothing more than a FastCGI Python library, can provide an alternative Python web development environment for people who feel that the Python web development frameworks (Django, Pylons, TurboGears and even Web.py) are too bloated.

Jump to this document.

Re: Unit testing is not (generally) useful

Tuesday, February 10th, 2009

There is a post up on Bill Moorier’s blog about the usefulness of Unit Testing. In it, he writes the following, which largely sums up his post:

The metric I, and others I know, have used to judge unit testing is: does it find bugs? The answer has been no. For the majority of the code I write, it certainly doesn’t find enough bugs to make it worth the investment of time.

I think we’re missing a couple of big advantages to Unit Testing here. Unit Testing catches very few bugs when writing a new program, this is true. Most developers test their code manually in much the same way as they would with unit tests. But this is not the true strength of Unit Testing.

First of all, Unit Testing forces one to write their programs so that they are testable. That usually means: well thought out APIs, consistent interfaces and more modular code.

Second, Unit Tests serve as documentation. They are prime examples of how code should be called, what side-effects one can expect and how various components work together. That’s assuming you write decent Unit Tests, naturally.

Finally, Unit Tests are invaluable when doing regression testing, which is where they truly shine. They are not for catching bugs in new code, they are for catching bugs in old code. Suppose you have a library which hasn’t changed in four years. Now you find a small problem or inaccuracy in one of the routines, so you decide to fix it. You test the new code, and it works. But when you run the Unit Tests, you suddenly find that on the other side of your application twenty tests fail where they passed with flying colors before you made the change. You just discovered a regression bug.

I have the feeling that Bill Moorier is talking about Unit Testing in the context of Test Driven development. I’m not much of a fan of Test Driven development, even though most of my points still hold true. However the Test Driven development proponents, much like the Extreme Programming proponents, somehow make it out to be a magically superior way of writing software. This I have to disagree with. I see no reason at all why Test Driven development and Extreme Programming in general would deliver better software than traditional software developing methodologies and, especially, common sense. There is no silver bullet. Bad software, in my view, is more often caused by bad or lazy programmers, time limits and seriously flawed requirements than the use of the wrong methodology.

Gimp Animation frame speed

Tuesday, February 3rd, 2009

The Gimp can do animations by simply using stacked layers as animations. You can preview the animation in the Filters → Animation → Playback. It’s rather to figure out how to change the time each frame is displayed though.

It turns out you have to change the layer name in order to change the time each frame is being displayed. To do this, open the Layers dialog, double click on a layer’s name, and add: (500ms) at the end of the layer name to make the current layer be displayed for 500 Miliseconds.

frames

Terminator – Splitting Terminal Emulator

Monday, January 12th, 2009

I use terminals a lot. Some years ago, there was a terminal emulator called Gnome-multi-terminal, which could be split horizontally and vertically, and thus giving optimum workspace usage when using many terminals. Gnome-multi-terminal wasn’t being maintained anymore (or at least not regularly) and started displaying some buggy behaviour in newer versions of Debian and Ubuntu.

I’ve searched for a replacement for a long time, but was never able to find one. Now I finally have: Terminator.

Here’s a screenshot:

.

Frustration-Free Packaging

Tuesday, November 4th, 2008

Amazon has a Amazon Frustration-Free Packaging initiative:


The Frustration-Free Package (on the left) is recyclable and comes without excess packaging materials such as hard plastic clamshell casings, plastic bindings, and wire ties. It’s designed to be opened without the use of a box cutter or knife and will protect your product just as well as traditional packaging (on the right).

I hate the plastic clamshell casings. They’re a terribly waste of natural resources for no good reason at all. And, like Amazon already says, they’re frustrating. More companies should start doing Frustration-Free packaging. Sometimes I feel like I’m paying twiice as much money for the packaging than I am for the product itself. An USB key and headphones do not need 500 grams of plastic.

Nice :)

Wednesday, October 29th, 2008

This is nice: I would just like to say….

Apache 2.2: Multiple authentication providers

Friday, October 24th, 2008

For a long time Apache only supported a single authentication provider per location. For instance, you’d have:

<Location />
    Require valid-user

    AuthType Basic
    AuthName "FooCoorporation"
    AuthBasicProvider ldap

    AuthzLDAPAuthoritative Off
    AuthLDAPURL ldap://192.168.1.1:389/ou=foo,o=electricmonk,c=nl
    AuthLDAPBindDN cn=ldapreader,o=electricmonk,c=nl
    AuthLDAPBindPassword PASSWORD
</Location>

In older Apaches, it wasn’t possible to add another Authentication provider. So, in the situation above, you can run into problems when your LDAP server dies on you, and you won’t be able to login to the / location anymore until the LDAP was fixed. Another problem with single authentication mechanisms is that there’s no way to add authenticated users if they’re not in the LDAP.

Since Apache 2.2 multiple authentication providers are now supported. This is nice, since now you can have an LDAP authentication provider with an htpasswd fallback authentication mechanism.

You can enable it by specifying multiple AuthBasicProvider providers:

    AuthBasicProvider ldap file

So the full Location section becomes, for example:

<Location />
    Require valid-user

    AuthType Basic
    AuthName "FooCoorporation"
    AuthBasicProvider ldap file

    AuthzLDAPAuthoritative Off
    AuthLDAPURL ldap://192.168.1.1:389/ou=foo,o=electricmonk,c=nl
    AuthLDAPBindDN cn=ldapreader,o=electricmonk,c=nl
    AuthLDAPBindPassword PASSWORD

    AuthUserFile /var/www/.htpasswd
</Location>

With language X, you’ll write code 10x faster!!1one

Wednesday, October 22nd, 2008

I’ve been hearing a lot of “With language X, you’ll write code 10 times faster!” kind of remarks in the last couple of years, where “language X’ can be substituted for the latest hip language or framework. I might even have shouted it about Python a couple of times myself. But does it really matter that much? On an average project, I spend maybe 30% on programming. The rest goes to planning, communicating, research, design, documentation, delivery, support, etc. And of that 30% a large portion is usually debugging, polishing, documenting (in the form of comments), making sure I’ve caught all the errors, finding weak spots in the code which might lead to bugs, checking if the security is okay, etc. Maybe silver-bullet-language-X helps in some of those cases, but I doubt it is very significant in the grand scheme of things.

I’m not really trying to make a point here or anything. Just an observation. Maybe something worth discussing.

Blocking VoilaBot

Tuesday, August 19th, 2008

There’s a web-crawler out there called VoilaBot, which is hammering my site with needless crawls and which appears to ignore robots.txt files completely. Apparently it’s a crawler for a french portal/search engine. If you need to block this bot from your site, there are two things you can do:

Firewall

If you’ve got a firewall on your box, you can deny access to the two IP ranges 81.52.143.0 / 24 and 193.252.149.0 / 24. That’ll get them off your back permanently. For Linux machines with iptables firewall, the following will do the trick:

iptables -A INPUT --source 193.252.149.0/24 -j DROP
iptables -A INPUT --source 81.52.143.0/24 -j DROP

htaccess

If you don’t want to firewall the bot, you can deny them access to your website by putting a .htaccess file in your web root directory with the following contents:

order allow,deny
deny from 81.52.143.
deny from 193.252.149.

Don’t trust VoilaBot to honour your robots.txt file; it won’t.

Dependency resolving algorithm

Thursday, August 7th, 2008

Here’s a little explanation of a possible dependency resolution algorithm. I’ve made this as basic as possible, so it’s easy to understand. If you’re at home in (graph) algorithms, this post is probably not of much interest to you.

Premise

Suppose we have five objects which all depend on some of the other objects. The objects could be anything, but in this case let’s say they’re very simple software packages (no minimal versions, etc) that depend on other packages which must be installed first. How does one find the right order of installing the packages?

(more…)

The text of all posts on this blog, unless specificly mentioned otherwise, are licensed under this license.