This article is the first in my new series "Python everywhere".
As this is the first article in this series I would like to explain what the series is all about.
As an avid Python user and developer I want to share my observations whenever I find Python applications doing not-so-unusual things, Python applications running on embedded devices. In the end I want to point out just what the name of this series suggests: Python is everywhere and can be used for everything.
So, straight ahead to the first issue: the conficker scanner.
When reading an article about a detection mechanism for the conficker worm on heise Security [german] I was myself wondering a few things, but wanted to give it a try. So I followed the link to the article Detecting Conficker, by Tillmann Werner. Before clicking the link I was wondering whether I could get this tool running on GNU/Linux using wine, or another method.
After downloading the ZIP file and unpacking it I thought I was dreaming. There were two Python files, along with a COPYING file.
So, even though before having a look at the code I wanted to know the COPYING conditions, and again I saw something unexpected: it's licensed under the GPLv3, great!
As there are some computers running a proprietary operating system from Redmond on this network I immediately gave it a shot. I started the script (scs.py), and after fulfilling its requirements (namely the impacket Python module) I ran it on the local network and it worked without any problems. No conficker found on this network, after all my flatmates have their systems secured - good.
So there you have another use-case for Python: detecting malware over the network.
Kudos should go to Tillmann Werner, not only for this piece of Python code, but also for his work on the honeynet project and, together with Felix Leder, the great analysis of conficker. Keep up the good work, and thanks for proving Python can also be used for this task.
In this article I would like to inform you about my newest pet-project: pyttpd.
pyttpd is my effort of implementing a webserver in Python, with a focus on security (through privilege separation), extensibility and scalability.
I started this project because I was not entirely happy with the lack of flexibility and support for privilege separation by popular webservers. Whilst both lighttpd and Apache httpd provide means of running processes under different users these usually require hacks like suexec. Additionally I am somehow curious about how a fully-fledged webserver implemented in Python would perform compared to the mentioned daemons.
Security through extensive use of Privilege Separation
Whilst it is common for daemons to initially run as a privileged user and drop privileges as soon as possible it is possible to make more extensive use of setuid and friends.
pyttpd's design aims at creating one privileged process, which only binds to privileged ports and spawns subprocesses.
All subprocesses have specific tasks, such as routing between all processes, protocol-specific parsing of incoming requests and handling processing of those requests.
The point is that all these processes do not run as "www-data" or another common account, but that a logic separation takes place on a per-host basis. This means that if the webserver is hosting www.example.org and webapp.example.org those will be running under different system accounts, making it hard to interfere with each other. This method should also enable the use of MAC mechanisms such as SELinux or SMACK more efficiently.
The design choice of having separate processes for each vhost comes with another benefit: users (or customers) "owning" a vhost could potentionally be allowed to modify parts of the vhost's configuration (excluding UID, GID, and other security-relevant options) on their own.
Early status
Right now pyttpd is in a very early planning stage, with no code to show yet. I am still in the middle of the process of writing down all ideas that come to my mind, weeding some out and documenting the others.
So what do I have to show you then? Well, the the concept section of the documentation is online now and I am planning on extending it in the next few days and eventually start writing code rather sooner than later.
Your ideas...
...and opinions are what I am really interested in. If you are interested in this project I would love if you got involved in some way. Feel free to create tickets at pyttpd's project page if you have an idea you feel is worth adding or if one of my ideas is flawed, create a comment here or send me an email.
I just wanted to inform you that I am in the process of adding AdSense ads to this blog.
However, I am planning on having a one-ad-per-post policy, whilst not placing any ads on the front page.
More details on this topic will follow in the next few days.
UPDATE:
I have now integrated AdSense into this blog. As promised the front page does not contain any ads, but all other pages do. Ads are shown as a widget so they are not in-text and thus should not disturb you whilst reading.
I am proud to announce that python-argvalidate has hit Debian unstable yesterday.
This does not only mean that you can install argvalidate on Debian-based systems more easily now, but also that python-argvalidate has met the strict criteria of the Debian Free Software Guidelines, and as such has been confirmed to be Free Software.
Also, I wanted to let you know that I am maintaining the Debian package itself, which means that updates to python-argvalidate itself will be included in Debian as fast as possible, usually within two days.
There has been a lot of discussion on whether Free Software is more secure than proprietary software, but I have an additional argument that shows how the use of Free Software can improve system security.
Now you probably expect me to come up with a pure technical reason showing superiority of Free Software, but I am taking another path this time: let's talk about user trust.
Software updates on pure Free Software operating systems
Firstly, let's take a look at how security updates are deployed on pure Free Software operating systems.
All major GNU/Linux and *BSD distributions come with built-in update systems for all Free Software packages provided by the distribution. Once a security update to any piece of software is released you will get a notification saying updates are available and that you should install them and after doing so your system should be in a quite secure state.
Updater-applications of proprietary software
Okay, proprietary software does often come with its own update mechanisms, giving you pretty much the same functionality. But here is my point: users tend to block these updater-applications from accessing the network with personal firewalls and similar tools, which basically disables the updater and most likely will cause the system to be more vulnerable rather sooner than later.
But why do people even consider blocking an updater-application?
When I asked people why they blocked these applications a lot of reasons came up, but it usually boils down to lack of trust for proprietary software vendors. People seem not to trust software vendors for a variety of reasons. The most common reasons seem to be that they are either worried about the disclosure of private information or the lack of a license for a piece of software, combined with the first reason.
So people actually seem to be scared by what a piece proprietary software could do, and when not having their software licensed people seem to be even more scared. When I ask people why they are running these programs, even though they do not trust their vendors they usually shrug and I get replies that can be summed up as "it just works", "but I don't have a choice" and "I am used to software X".
The solution is Free Software
Free Software can be a solution to all these problems. The most important thing people should be aware of that they do not have to fear Free Software vendors. After all, when using Free Software you do have a valid license and you can, at least in theory, check exactly a program is doing to your system. Also, if you are not capable of doing such checks yourself, you can rest assured that other people are doing such checks and give back to the project in question, improving your favorite Free Software applications.
Free Software is a choice, and there is hardly any proprietary piece of software which cannot be replaced by a free equivalent. Free Software is my first choice, not only for me but for my family too, with great results so far.
Rest assured you can trust in what your software is doing, free yourself today - use Free Software.
Just as I wanted to start writing an article here and I entered the URL of this blog into my browser I got no response from the webserver, zero, nothing.
First I thought the PHP fastcgi process for this virtual host died, but a quick check on another virtual host suggested that something else was going on.
So I guessed the lighttpd process itself must be experiencing problems of some sort, but after doing a "netstat -nat" I knew what was going on:
tcp6 1 1 83.65.62.72:80 61.135.190.248:12474 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.234:39671 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.253:39211 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.234:55160 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.230:25836 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.231:16865 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.232:24266 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.240:38441 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.243:17726 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.241:38206 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.251:23892 LAST_ACK
tcp6 1 1 83.65.62.72:80 61.135.190.225:29675 LAST_ACK
Plus "a few" more of those. Now I'm not entirely sure whether it's just some systems misbehaving or actually an attack, but my feelings told me this could have been intentional after all.
I did a quick whois on one of those IP addresses and came up with the 61.135.0.0/16 network which is owned by China Network Communications Group Corporation.
As the connections were made from pretty much every host in that network I had two choices: sit it out or block it.
I came to the conclusion that blocking the entire subnet from connecting to this system, at least temporarily, might be a viable solution and so I did.
However, afterwards I am asking myself whether I really had to block an entire 16-Bit network, so I am asking you: how do you handle such situations usually?
Even though I planned providing a release candidate first, which can be seen in the project's Mercurial changelog I have released python-argvalidate 0.9.0 today. Tarballs can be obtained from the Python Package Index (pypi), as usual.
So what's has changed compared to 0.8.2
Basically a lot of code has been rewritten or reviewed and modified. The most important change lies in argvalidate's internals though.
Instead of relying on Python internals, those are the func_* and func_code.* attributes of every function argvalidate uses another mechanism for obtaining the information about functions now, the inspect Module. In short this means that argvalidate should be more resistant to possible changes in the format of these attributes, because the inspect Module that ships with Python should be changed accordingly and it is unlikely that its public interface is going to break.
Which leads me straight to the next point regarding argvalidate: the public interface. The most visible change was the deprecation of the method_args, func_args and return_value decorators. These decorators are still present, but will be removed in one of the next upcoming releases of argvalidate. The functionality of method_args and func_args has been combined into the accepts decorator, whilst the returns decorator replaces return_value.
accepts now includes some magic for automatically determining whether a function or method has been decorated. Initially the plan was on using inspect's ismethod function, which did not work out as when the decorator is called for methods the method has not been bound to the class yet and looks like a simple function to ismethod. However, the magic added checks the first argument's name and decides that functions having either "self" or "cls" as their first parameter are methods, whilst everything else is a simple function.
There is more to say about accepts. Internally some checks which were done at every call to the decorated function in the past are now done only once, when the decorator is invoked. This should reduce the overhead argvalidate adds when the function (and thus the internal wrapper doing the checks) is called, but most likely will slow down decoration itself. However, decoration is only done once, whilst a function is likely to be called several times in its lifetime, which should give us an overall improvement of speed.
The documentation has been updated accordingly and can be found at the usual place.
What's up next?
There is no real further roadmap for argvalidate right now. No issues were left open, except for Python 3 testing & support. Further releases are going to be made when needed, that is when you submit a bug report and it is fixed.
I just wanted to let you know that this blog (actually all webpages I am hosting) are now accessiable via IPv6. Additionally, my mail-server now also accepts IPv6 SMTP and IMAP connections, allowing communication with the IPv6-world.
The setup uses SiXXs as tunnelbroker, with AMIS being the SiXXs PoP in use.
If you experience any problems with the services I am providing via IPv6, please let me know, either via a comment to this article or an email to ipv6@sp-its.at.
Welcome to my new blog, "freedom blog reloaded".
Now with this first article I would like to elaborate on the name of the blog, the purpose and what you are likely to find here in the future.
Okay, let's start straight ahead with the name of the blog. Freedom in the blog's name refers to Free Software, which is going to be the main topic of the articles you will find here.
I would like to keep you informed about my involvement in the Free Software community and hopefully provide you with some useful information when it comes to configuring and running Free Software.
Now you might still ask what the "reloaded" part in the blog's name is about. Well, I have done some blogging in the past, but due to various reasons didn't have the time to provide my readers with a constant flow of articles, but this should change now. I am planning on regularly keeping you informed.
On to the last thing I wanted to write about: the kind of articles you are likely to find here in the future.
I am planning on writing posts on development in the Free Software community, updates to the Debian GNU/Linux packages I either maintain or co-maintain, the projects I am working on and last but not least some tips and tricks when it comes to day-to-day operation.
Lastly, as this is a blog dedicated to Free Software it's a good idea to let you know that this blog is being run on a Free Software stack completely and I am using Free Software only to write articles.
The setup is as follows: Running on a Debian GNU/Linux system is lighttpd, my webserver of choice, and builds, along with PHP5 and MySQL, the base for running Wordpress, a blogging system written in PHP.
For writing articles I am using, guess what, a browser, namely Iceweasel (also known as Firefox to non-Debian users), running on my Debian GNU/Linux workstation.
I guess that's it for now. As a last note I would like to point out that even though comments have been disabled for this article I will enable them for all posts where discussion makes sense.
-- Stephan