Log <-

Archive for the ‘sysadmin’ Category

RSS   RSS feed for this category

Blocking VoilaBot

Tuesday, August 19th, 2008

There's a web-crawler out there called VoilaBot, which is hammering my site with needless crawls and which appears to ignore robots.txt files completely. Apparently it's a crawler for a french portal/search engine. If you need to block this bot from your site, there are two things you can do:

Firewall

If you've got a firewall on your box, you can deny access to the two IP ranges 81.52.143.0 / 24 and 193.252.149.0 / 24. That'll get them off your back permanently. For Linux machines with iptables firewall, the following will do the trick:

iptables -A INPUT --source 193.252.149.0/24 -j DROP
iptables -A INPUT --source 81.52.143.0/24 -j DROP

htaccess

If you don't want to firewall the bot, you can deny them access to your website by putting a .htaccess file in your web root directory with the following contents:

order allow,deny
deny from 81.52.143.
deny from 193.252.149.

Don't trust VoilaBot to honour your robots.txt file; it won't.

Disable Ubuntu command not found

Sunday, February 17th, 2008

Ubuntu 7.10 has this feature where, if you mistype a command on the shell, it'll bother you with useless information about how you can install that application:

tiagoboldt@Niath:~$ gedit
The program 'gedit' is currently not installed. You can install it by typing:
sudo apt-get install gedit
bash: gedit: command not found
tiagoboldt@Niath:~$

It's pretty annoying if you do a lot of work on the commandline and each time you mistype a command you have to wait for up to a full second before the command fails. I keep thinking I accidentally ran some command which is now wiping my disk. I'm sure it's handy for noobs, but not for commandline junkies.

To get rid of it, type:

sudo aptitude purge command-not-found

And it won't bother you anymore.

Transmission 1.x on Ubuntu 7.10

Sunday, February 17th, 2008

For some reason Ubuntu 7.10 has an ancient version of Transmission. Version 0.74 or somesuch. Unfortunately, that version of Transmission contained some bugs so it's blocked by certain bittorrent trackers. In order to install a more recent version:

You can download a more recent version from the gutsy backports package pool.

  • Uninstall transmission:
    sudo aptitude purge transmission transmission-gtk
  • Download transmission-common 1.04
  • Download transmission-gtk 1.04
  • Install the packages:
    sudo dpkg -i "transmission-common_1.04-0ubuntu1~gutsy1_all.deb"
    sudo dpkg -i "transmission-gtk_1.04-0ubuntu1~gutsy1_i386.deb"

And you'll have a more recent version of Transmission.

VeeDee-Eyes.com: VirtualBox VDI images for download

Saturday, February 16th, 2008

VirtualBox has been getting more and more attention as of late. It is vastly superior to other virtualization programs due to the simple fact that it actually runs on my system. All the others either don't install (VMWare), crash (QEMU), or are a real bitch to set up properly (Xen). VirtualBox is Open Source, easy to install, fast, runs on Linux and Windows and it has both a very nice GUI and a very decent commandline interface. VirtualBox is what VMWare (for Linux) could have been, but better.

Now, WMWare has pre-built images online at their Appliances site. Qemu has the Free OS Zoo. Now there's one for VirtualBox too:

veeDee-Eyes.com:

The VirtualBox "Virtual Disk Image" Index

VDI images of pre-installed "Open Source" Operating System distros.
It's a computer inside your computer.

-Instantly run another operation system on your desktop in a window, on almost any computer.
-Implement full Linux functionality on an existing Windows Desktop or server.

I've been looking for this for a while. All it's currently missing are OpenSolaris and various BSD images. I'm sure these will be added rather quickly.

PS: It appears Innotek, the creators of VirtualBox, have been acquired by Sun. I'm not sure yet whether Sun's acquiring of Open Source companies is a good or bad thing. Guess we'll have to see where it goes. Perhaps they should get the benefit of the doubt since they did free the source of StarOffice (Open Office), Java (in part) and Solaris (in part). I'm just not sure whether Sun understands the Open Source philosophy and community.

DVTM – dynamic virtual terminal manager

Monday, February 11th, 2008

I do a lot of remote UNIX administration on various servers and even some desktop machines. This usually means that I'm doing a lot of SSHing into machines, often opening multiple sessions to the same machine at the same time. It quickly becomes tedious to keep having to type 'ssh fboender@some.machine.com' four times in order to get four sessions.

Of course there is Screen, but Screen's split screen abilities are very limited and don't really work for me. While scanning Freshmeat.net, I found a tool called DVTM: dynamic virtual terminal manager.

DVTM
DVTM is a kind of window manager for the console (or xterm) which allows you to quite easily split your terminal into multiple split screens.

DVTM has some limitations though:

  • Configuration is done using C code. (Though you'll probably never need to change anything; the defaults are quite sane)
  • It doesn't seem to support ANSI colouring. I use a red background at my prompt when I'm root, which I now have to miss.
  • It also doesn't seem to support line-drawing. This probably has something to do with the terminal type.
  • DVTM has no internal support for detaching and reattaching to running DVTM instances like Screen can. (But see the Tips and tricks section on the homepage).
  • There are a couple of small bugs, one of which is a problem with repeating keys when switching between split windows with the mouse.

But make no mistake: despite these shortcomings, DVTM is an absolutely fantastic tool. Some of the neat stuff it supports is:

  • Mouse support for switching between split windows / maximizing and unmaximizing, etc.
  • Maximize/unmaximize the currently active window (Default: Ctrl-g m / Ctrl-g g)
  • Various window layouts (you can probably add your own by hacking the source code)
  • Almost no dependencies except for ncurses.

The only thing I'd really like to see changed about DVTM are custom run-time splitting (for instance, Ctrl-g-MINUS to split the current window horizontally and Ctrl-g-PIPE to split the window vertically) and detaching/reattaching.

All in all, DVTM is an awesome tool. Go download it if you do a lot of remote administration.

Update: There also appear to be two bugs:

  • On the latest Ubuntu: Sometimes when exiting DVTM, it will hang for a while, and the cursor will blink like mad. An strace shows that IO Errors are occuring on file descriptor 3. I've filed a bug report with the author. This bug doesn't appear on Debian stable.
  • When using DVTM in combination with dtach, reattaching doesn't redraw the screen. This appears to be a problem with DVTM, but it would probably be easier to solve in dtach (which could just signal a screen resize event without actually resizing the screen). This is also the workaround for this bug: just resize the window. I'll see if I can file a bug report with either DVTM's author or dtach's author.

Update II: Here's a little patch/workaround for the first bug (exiting DVTM hangs for a while):

--- dvtm.c      2008-02-06 17:30:25.000000000 +0100
+++ ../dvtm-0.4/dvtm.c  2008-02-11 23:33:45.000000000 +0100
@@ -896,6 +896,7 @@
                Client *c;
                int r, nfds = 0;
                fd_set rd;
+               struct timeval tv;

                if(need_screen_resize)
                        resize_screen();
@@ -996,6 +997,13 @@
                        wnoutrefresh(sel->window);
                }
                doupdate();
+
+               // One microsecond delay so this loop doesn't hang context-switching
+               // when the last child dies and the SIGCHLD signal is prevented from
+               // being caught on time.
+               tv.tv_sec = 0;
+               tv.tv_usec = 1;
+               select(0, NULL, NULL, NULL, &tv);
        }

        cleanup();

Save to ~/patch.diff and in the dvtm-0.4 directory run:

~/dvtm-0.4$ patch -p0 < ~/patch.diff

Update III:

I was incorrect in stating that dvtm doesn't support ANSI coloring. It's just the bash prompt which doesn't work. Perhaps because there is a Xterm-styled window title ANSI sequence in there. Will update when I know more.

Update IV:

Marginally better patch:

diff -Naur ./dvtm.c ../dvtm-0.4/dvtm.c
--- ./dvtm.c	2008-02-06 17:30:25.000000000 +0100
+++ ../dvtm-0.4/dvtm.c	2008-02-12 22:08:54.000000000 +0100
@@ -982,7 +982,10 @@

 		for(c = clients; c; c = c->next){
 			if(FD_ISSET(c->pty, &rd)){
-				madtty_process(c->term);
+				if (madtty_process(c->term) < 0 && errno == EIO) {
+					/* client probably terminated */
+					client_killed = c;
+				}
 				if(c != sel){
 					draw_content(c);
 					if(!isarrange(fullscreen))

chkrootkit false positives filtering

Thursday, November 29th, 2007

Chkrootkit is a tool that searches for rootkits, trojans and other signs of break-ins on your system. Like most security scanners, it sometimes generates false positives. Chkrootkit doesn't have a native way to filter those out. From the FAQ:

[Q:] chkrootkit is reporting some files and dirs as suspicious: `.packlist', `.cvsignore', etc. These are clearly false positives. Can't you ignore these?

[A:] Ignoring some files and dirs could impair chkrootkit's accuracy. An attacker might use this, since he knows that chkrootkit will ignore certain files and dirs.

This is true, but getting an email every day is simply too annoying, and makes me skip chkrootkit generated emails on occasion because "It's probably a false positive anyway". So here's a small guide for setting up a filtering of chkrootkit's output.

First, we create a file /etc/chkrootkit.ignore which will hold a bunch of regular expressions that will match everything we don't want to be warned about. For instance, I've got a machine that needs to have a dhcp client installed. Chkrootkit keeps on generating emails with these lines:

eth0: PACKET SNIFFER(/sbin/dhclient[346])
eth1: PACKET SNIFFER(/usr/sbin/dhcpd3[1008])

So what we do is create the file /etc/chkrootkit.ignore and put the following in it:

/etc/chkrootkit.ignore

^eth0: PACKET SNIFFER\(/sbin/dhclient\[[0-9]*\])$
^eth1: PACKET SNIFFER\(/usr/sbin/dhcpd3\[[0-9]*\]\)$

In order to test if the rules we created are correct, we put the two lines with false positives in a separate file (/tmp/chkrk-fp.txt) and run the following:

test:

[root@sharky]/etc# cat /tmp/chkrk-fp.txt | grep -f /etc/chkrootkit.ignore
eth0: PACKET SNIFFER(/sbin/dhclient[346])
eth1: PACKET SNIFFER(/usr/sbin/dhcpd3[1008])

The lines that should be filtered out of the chkrootkit output should appear here. If nothing appears, or if not all of the lines that you want to filter appear, there's a problem. Refine your regular expressions in /etc/chkrootkit.filter until it works.

Now we need to modify the chkrootkit cronjob so that the false positives are filtered. To do this, we edit /etc/cron.daily/chkrootkit. Below is a patch that shows what should be changed. You can apply the patch with the 'patch' command, or you can manually add the lines that start with a '+', replacing the lines with a '-'.

--- /home/root/foo      2007-11-21 11:53:58.532769984 +0100
+++ /etc/cron.daily/chkrootkit  2007-11-21 11:54:00.689442120 +0100
@@ -1,27 +1,28 @@
 #!/bin/sh -e

 CHKROOTKIT=/usr/sbin/chkrootkit
 CF=/etc/chkrootkit.conf
+IGNOREF=/etc/chkrootkit.ignore
 LOG_DIR=/var/cache/chkrootkit

 if [ ! -x $CHKROOTKIT ]; then
   exit 0
 fi

 if [ -f $CF ]; then
     . $CF
 fi

 if [ "$RUN_DAILY" = "true" ]; then
     if [ "$DIFF_MODE" = "true" ]; then
-        $CHKROOTKIT $RUN_DAILY_OPTS > $LOG_DIR/log.new 2>&1
+        $CHKROOTKIT $RUN_DAILY_OPTS | grep -v -f $IGNOREF > $LOG_DIR/log.new 2>&1 || true
         if [ ! -f $LOG_DIR/log.old ] \
            || ! diff -q $LOG_DIR/log.old $LOG_DIR/log.new > /dev/null 2>&1; then
             cat $LOG_DIR/log.new
         fi
         mv $LOG_DIR/log.new $LOG_DIR/log.old
     else
-        $CHKROOTKIT $RUN_DAILY_OPTS
+        $CHKROOTKIT $RUN_DAILY_OPTS | grep -v -f $IGNOREF || true
     fi
 fi

Next, we try running chkrootkit, to see if anything shows up:

[root@sharky]/etc/cron.daily# ./chkrootkit
[root@sharky]/etc/cron.daily#

There is no output, so our false positives are now being ignored.

SSH + SOCKS5 = Universal proxy

Wednesday, November 14th, 2007

I didn't know it, but (Open)SSH supports setting up a Socks5 proxy:

-D [bind_address:]port
  Specifies a local ``dynamic'' application-level port forwarding.
  This works by allocating a socket to listen to port on the local
  side, optionally bound to the specified bind_address.  Whenever a
  connection is made to this port, the connection is forwarded over
  the secure channel, and the application protocol is then used to
  determine where to connect to from the remote machine.  Currently
  the SOCKS4 and SOCKS5 protocols are supported, and ssh will act
  as a SOCKS server.  Only root can forward privileged ports.
  Dynamic port forwardings can also be specified in the configura-
  tion file.

Socks5 is pretty neat, as it allows you to proxy stuff without the server having to know anything about the way the client works. For instance, if we give the following command:

$ ssh -N -D 127.0.0.1:8080 todsah@sharky.electricmonk.nl

We can now tell all kinds of clients such as web browsers and instant messaging clients that there is a Socks5 proxy running on the localhost at port 8080. SSH will forward all connections made to port 8080 to the sharky.electricmonk.nl host (all encrypted of course).

So, say we tell Pidgin that it should connect your MSN account through the Socks5 proxy at localhost:8080 by opening the Accounts (Ctrl-A) → Click MSN account → click ModifyAdvanced tab → Proxy options → Proxy type = SOCKS5 and setting it to Host: localhost, Port: 8080. Now, when we reconnect to our MSN account, all MSN traffic will be routed over an encrypted SSH tunnel to the sharky.electricmonk.nl host, and will enter the public Internet from there.

This works great if you don't trust the network you're currently on, but don't have access to a VPN for instance. You also don't have to specify a single forward for each application/port like you have to do when you use ssh -L. You can use the same SOCKS5 proxied port with multiple applications, as long as they understand SOCKS5.

Virtualization Security

Thursday, October 25th, 2007

Theo de Raadt on virtualisation security:

> Virtualization seems to have a lot of security benefits.

You've been smoking something really mind altering, and I think you
should share it.

x86 virtualization is about basically placing another nearly full
kernel, full of new bugs, on top of a nasty x86 architecture which
barely has correct page protection. Then running your operating
system on the other side of this brand new pile of shit.

You are absolutely deluded, if not stupid, if you think that a
worldwide collection of software engineers who can't write operating
systems or applications without security holes, can then turn around
and suddenly write virtualization layers without security holes.

> Anything we can do to increase security, *including* setting up VMs (of any
> flavor) is an improvement [that also increased hardware utilization].

This last sentence is such a lie.

The fact is that you, and most of the other fanboys, only care about
the [that also increased hardware utilization]. The yammering about
security is just one thing — job security. You've got to be able to
sell increased harwdare utilization in a way that does not hang you up
at the end of the day.

Of course, de Raadt is right… in his own tiny little world at least. Running services which would normally run on multiple machines on multiple hypervisored instances on a single host machine would indeed be less secure than running them from multiple physical machines.

But running multiple applications on virtualized machines which would normally run on a single machine is more secure, simply because it adds another layer of protection.

But, as usual, de Raadt's complete ineptitude when it comes to communications totally negates any point he's trying to make and only serves to rile up people against his cause.

It's the chrooted story all over again. Yes, chroot isn't completely secure. Yes, chroot isn't meant as a security construction. Yes, running multiple services on a single machine that would normally be run on several separate physical machines is less secure. That doesn't mean chroot (and virtualisation for that matter) can't add an extra layer of security if used properly!

Theo de Raadt's problem is that he views security the way cryptography experts view cyphers: as an absolute. But security isn't like math. It's not absolute. There are right and wrong ways of doing security. De Raadt is like that security consultant who says: "You must have randomly generated passwords consisting of at least eighteen characters, lower and upper case, numbers and symbols, nothing repeated twice, completely unique and changed every week, or your being insecure!", all the while ignoring the fact that that kind of password policy will only force people to write down passwords on a yellow-note under their keyboards. In theory, they're right. In practice, they're wrong. These people become blinded by their own viewpoint. Just as these so-called security consultants are blinded by their belief that strong passwords equal security, so is Theo de Raadt blinded by his belief that virtualization doesn't improve security.

Perhaps it's time to stop listening to de Raadt, and start listening to people with a broader overview of the situation.

CherryPy on Apache2 with mod_python

Saturday, October 13th, 2007

(This article is also available here)

I've recently written a web application using Python using the following
libraries:

  • CherryPy v3.0.2

  • Mako v0.1.8

  • SQLAlchemy v0.3.7

CherryPy has a built-in web server which you can use during development and for
actually running the application. I already had an Apache with some old PHP
programs however, so I couldn't serve the Python web application using
CherryPy's built-in web server, cause I didn't want to serve it on a port other
than port 80. Fortunately, CherryPy applications can also be served with Apache
using mod_python.

Setting up to run it through mod_python turned out to be somewhat of a major
pain though. It took me a total of about 4 hours getting it to work. The
information on the CherryPy website about mod_python turns out to be incorrect,
incomplete and a little dated.

So in this article I'll describe how I eventually managed to set up my
application to work with both the built-in server as well as with Apache v2 and
which pitfalls to look out for.

(more…)

DDoS attack paralyses Estonia

Sunday, September 9th, 2007

Hackers Take Down the Most Wired Country in Europe:

At exactly 11 pm, Estonia was slammed with traffic coming in at more than 4 million packets per second, a 200-fold surge. Globally, nearly 1 million computers suddenly navigated to a multitude of Estonian sites, ranging from the foreign ministry to the major banks. It was a larger-scale version of what had happened to the Postimees, except that the entire country's bandwidth capacity was being squeezed.

The story is a little over the top, there's a lot of speculation and I doubt Estonia is 'the Most Wired Country in Europe', but it's still an interesting read. I wonder if the Europian Union has any plans on how to counter such calamities. Basically all that's needed is a good line of communications via which immediate action can be taken to stop traffic as close to the root as possible, I guess.