Log <-

Archive for the ‘python’ Category

RSS   RSS feed for this category

Breakpoint-induced Python debugging with IPython

Wednesday, June 25th, 2008

Most of the Python programmers out there will know about IPython. Most of them will also know about the Python Debugger (PDB).

IPython has an advanced version of PDB (spectacularly named 'ipdb') which does the same for PDB as IPython does for the normal interactive Python interpreter. It adds tab completion, color syntax highlighting, etc. The -pdb switch to IPython gives us access to the debugger automatically in the event of uncaught exceptions:

def ham():
	x = 5
	raise NotImplementedError('Use the source, luke!')
 
ham()

We now run this code using ipython -pdb

[todsah@jib]~$ ipython -pdb
In [1]: import test.py
<type 'exceptions.NotImplementedError'>: Use the source, luke!
> /home/todsah/test.py(5)ham()
      4         x = 5
----> 5         raise NotImplementedError('Use the source, luke!')
      6 

ipdb> print x
5

And as you can see we get dropped at a nice ipdb> prompt which allows us to use the additional power of IPython to investigate the problem.

Like most decent debuggers, we can also use PDB to set breakpoints. In Python, we do that in the code, rather than via external means. To do this, we import the pdb module and tell it to drop us into the debugger when execution hits the pdb.set_trace() line.

import pdb
def ham():
	x = 5
	pdb.set_trace()
 
ham()

We run it, and lo and behold, we get dropped into the pdb debugger:

In [1]: import test.py
> /home/todsah/test.py(8)ham()
-> raise NotImplementedError('Use the source, luke!')
(Pdb) print x
5

Buuuuut… as you can see from the (Pdb) prompt, this is the normal PDB debugger, not IPython's enhanced version. Wouldn't it be neat to be able to use IPython's debugger for breakpoint-induced debugging too? I spent some time looking around on the Interwebz, trying to find out how to do this, and to my surprise I couldn't find anything? So I dove into the IPython source and discovered IPython.Debugger.Tracer:

from IPython.Debugger import Tracer; debug_here = Tracer()
 
def ham():
	x = 5
	debug_here()
	raise NotImplementedError('Use the source, luke!')
 
ham()

Now when we execute this code, we're dropped into the IPython enhanced IPDB debugger:

In [1]: import test
> /home/todsah/test.py(8)ham()
      7         debug_here()
----> 8         raise NotImplementedError('Use the source, luke!')
      9 

ipdb> print x
5

We can use tab completion and all the other goodies IPython offers over standard Python now.

Why Python Rocks I: Inline documentation

Sunday, June 22nd, 2008

Okay. So what's cool about Python? I can't count the number of times I've had to show skeptics why Python is cool, what Python can do that their favorite language can't do. So I'm writing a bunch of articles showing off Python's Awesome.

First up: Documentation. I'm talking about inline documentation here: annotating modules, classes, methods, etc. Most languages have third party tools that parse the source code and extract documentation from comments. This is nice, of course, but the comments get out of date and you have to regenerate the documentation each time. Different people use different documentation generators (Doxygen VS. PHPDoc, JSDoc VS. ScriptDoc, etc) which, in turn, use different documentation standards, causing unknown chaos documentation even within the same language space. You may have heard some code monkeys say "The code IS the documentation". In Python, that's actually not far from the truth. Let's look at some of the things you can do with inline Python documentation.

(more…)

Links

Wednesday, June 18th, 2008

Here are some random links to interesting stuff:

FirePHP
FirePHP is a PHP debugging library and a Firefox plugin which allow you to output debugging information to the Firebug debugging panel. Since it doesn't intermingle debugging information with your page output, but writes in a special HTTP header instead, it's especially useful for AJAX debugging. It can also come in handy when you're trying to debug a server-side script which generates something else than a HTML page. A PDF or PNG file, for example.

OpenProj
OpenProj is a project management application written in Java and therefor platform independent. It has a lot of the features Microsoft Project has (according to the webpage; I have never used MS Project before, so I wouldn't know) such as Resources, Gantt Charts, Network Diagrams (PERT Charts), WBS and RBS charts, etc. There are also various different representations of tasks for resources. It doesn't really outshine Gnome Planner, but at least it's platform independent.

Typechecking Python module
Typecheck provides powerful run-time typechecking facilities for Python functions, methods and generators. Without requiring a custom preprocessor or alterations to the language, the typecheck package allows programmers and quality assurance engineers to make precise assertions about the input to, and output from, their code.

Here's a little code example:

@accepts(String, [Number], {str: Number})
def my_func(a, *vargs, **kwargs):
    pass
 
@accepts(String, Number, Number)
def my_func(a, *vargs, **kwargs):
    pass

pyBrainfuck

Saturday, March 22nd, 2008

For fun, I wrote a brainfuck interpreter in Python. Brainfuck is an esoteric (joke) programming language which is Turing-complete (given enough memory) with only 8 op-codes (instructions). It was designed to allow for the smallest possible compiler.

There are already some other Brainfuck implementations in Python, but they are either obfuscated or extremely slow. pyBrainfuck is optimized for speed by pre-caching loops and removing non-brainfuck opcodes.

PyBrainfuck can be used both as a stand-alone Brainfuck interpreter or as a python library. It can read from standard input or from a string (in library mode) and write to standard out or to a string buffer (in library mode).

pyBrainfuck is released under the MIT license. You can directly view the code for the interpreter at the Subversion front-end.

Python web app server in 48 lines

Saturday, November 3rd, 2007

I needed to write a little application which would be accessible from a remote client, and which wouldn't need any custom software running on the client. I needed a web application. But I didn't feel like setting up a whole Apache web server with mod_python and whatever.

Of course, there's CherryPy, but it feels a bit heavy for the very simple application I required. So I wrote TinyWAPP: Tiny Web App. A tiny web application server in Python in only 48 lines.

(more…)

IPython – Interactive Python shell

Monday, October 29th, 2007

(The latest version of this introduction to IPython can always be found here)

Python has an interactive shell, which you can start by simply starting
python:

[todsah@jib]~$ python
Python 2.4.4 (#2, Apr  5 2007, 20:11:18)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print('hello')
hello

This is a nice and very powerful way of using Python, but it's a bit
limited. So you might want to check out IPython.
IPython is also an interactive Python shell, but with lot's of stuff added,
such as tab completion, colors, dynamic object introspection, sessions,
command history, etc.

To start it, simply run the IPython command:

[todsah@jib]~$ ipython
Python 2.4.4 (#2, Apr  5 2007, 20:11:18)
Type "copyright", "credits" or "license" for more information.

IPython 0.8.0 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]:

You'll be dropped at a prompt (In [1]:) where you can enter python
commands, just like in the normal interactive python interpreter. Let's walk
through a couple of IPython's best features:

(more…)

Mako 'int' object is not callable

Sunday, October 28th, 2007

If you're trying out Mako (the templating language) and you happen to get this error:

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/CherryPy-3.0.1-py2.4.egg/cherrypy/_cprequest.py", line 551, in respond
    cherrypy.response.body = self.handler()
  File "/usr/lib/python2.4/site-packages/CherryPy-3.0.1-py2.4.egg/cherrypy/_cpdispatch.py", line 24, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "./pua.py", line 82, in index
    return Template('index', {'title': 'Title!'})
  File "./pua.py", line 35, in Template
    return(t.render(**vars))
  File "/usr/lib/python2.4/site-packages/Mako-0.1.8-py2.4.egg/mako/template.py", line 114, in render
    return runtime._render(self, self.callable_, args, data)
  File "/usr/lib/python2.4/site-packages/Mako-0.1.8-py2.4.egg/mako/runtime.py", line 287, in _render
    _render_context(template, callable_, context, *args, **_kwargs_for_callable(callable_, data))
  File "/usr/lib/python2.4/site-packages/Mako-0.1.8-py2.4.egg/mako/runtime.py", line 304, in _render_context
    _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
  File "/usr/lib/python2.4/site-packages/Mako-0.1.8-py2.4.egg/mako/runtime.py", line 337, in _exec_template
    callable_(context, *args, **kwargs)
  File "index_html", line 20, in render_body
TypeError: 'int' object is not callable

Check if your template lookup default_filters has an decoding for utf-8:

template_lookup = mako.lookup.TemplateLookup(
    directories=[path_templates],
    output_encoding='utf-8',
    encoding_errors='replace',
    default_filters=['decode.utf-8']
)

If it does, change 'decode.utf-8' to 'decode-utf8' (remove the dash). This will fix the error. No idea where it comes from, probably the utf-8 decoding doesn't exist. Me and Michiel now have both suffered from this problem, so there's bound to be more.

CherryPy on Apache2 with mod_python

Saturday, October 13th, 2007

(This article is also available here)

I've recently written a web application using Python using the following
libraries:

  • CherryPy v3.0.2

  • Mako v0.1.8

  • SQLAlchemy v0.3.7

CherryPy has a built-in web server which you can use during development and for
actually running the application. I already had an Apache with some old PHP
programs however, so I couldn't serve the Python web application using
CherryPy's built-in web server, cause I didn't want to serve it on a port other
than port 80. Fortunately, CherryPy applications can also be served with Apache
using mod_python.

Setting up to run it through mod_python turned out to be somewhat of a major
pain though. It took me a total of about 4 hours getting it to work. The
information on the CherryPy website about mod_python turns out to be incorrect,
incomplete and a little dated.

So in this article I'll describe how I eventually managed to set up my
application to work with both the built-in server as well as with Apache v2 and
which pitfalls to look out for.

(more…)

Giving up on PHP

Friday, October 5th, 2007

I have given up on PHP. I will still need to use it at my job, but in private I refuse to even touch it anymore.

Lately, I've been busy creating a framework, or a bunch of libraries actually, which where meant to negate some of PHP's worst qualities. It had better error reporting (the kind you can't just ignore), logging, sane naming schemes, better configurability, etc. But at every turn, it felt like PHP was actively trying to fight my efforts at overcoming its obscene shortcomings.

I will not rehash here those shortcomings of PHP, except that it's a pathetic excuse for a programming language written by a bunch of particularly bad programmers. The same goes for PEAR. They're simply horrid.

Some time ago I wrote a little web application in Python, and I actually had fun doing it. It felt liberating not having to fight your programming language every step of the way for once. I revelled in the professionalism with which basically everything in and for Python is constructed. It's invigorating to work with libraries that aren't written in a fundamentally broken way just so the authors can use every Design Pattern under the sun.

It's not strange that Ruby (On Rails) has become so popular. Even though Rails is severely limited in its abilities, people will gladly put up with it because of Ruby. Anybody speaking in favour of PHP compared to Python or Ruby simply hasn't worked with either Ruby or Python, or has never done anything significant in PHP. Or, like me, where so blinded by how much time they've invested in PHP, that they couldn't and didn't want to admit what a total piece of crap PHP really is.

Thank you, PHP, for finally exposing yourself as the leper you are. Our ways part here, and may I never run into you again. I'm running off with Python, and we'll live happily ever after.

GadflyB5: SQL Relational Database in Python

Wednesday, August 22nd, 2007

Gadfly is a SQL Relational Database that supports a large set of the SQL standard. It keeps the database in-memory while operating on it. It's also completely written in Python (with the use of some Python/C modules). I don't know why it's cool, but it is.