Static website generators

The last month I was supposed to work on a OpenStack related project, but for administrative reasons it has been delayed and I've tried to do small tasks to be able to finish them quickly and start the work on the main project when the issues get solved.

As the delay has been longer than expected last Wednesday I've realized than on the last weeks I did a lot of small system administration tasks:

• With a co-worker I started to work on a GNU/Linux version of our firewall based on Shorewall to handle the rules and conntrackd and keepalived to make it highly available (I had to stop my work on the Debian GNU/kFreeBSD based firewall a long time ago, and this summer the old firewalls' hardware started to fail, so a migration from Linux to Linux makes sense now, as it will be faster and a future migration will be simpler, as we will have a cleaner set of rules and better documentation),
• I installed and configured an instance of a web based File Exchange server (F*EX),
• I installed and configured an instance of a pastebin clone,
• I installed and configured an instance of ProFTPD that works only as a SFTP server using virtual users (without shell access),
• I also installed an instance of a web based event management system called indico that is being used to manage a conference and probably will be used for other events in the future,
• I installed and patched some plugins in our Trac servers,
• I tried a groupware system called SOGo that we will probable deploy in a week or two,
• And updated and fixed configurations of some other services,

With all the changes I did I noticed that I had to do something with our Intranet server; it is just a reverse proxy for a lot of different web services and its main page was one static HTML page with links to them, nothing else.

In the long term maybe we will replace it with something based on Drupal or Lifeay, but for now I just wanted something to be able to organize the links and provide some information about the services for the new users without having to write HTML (I really like Agile Documentation Tools that let me focus on the content and forget about the markup), and started to look at some of them.

My first idea was to use ikiwiki, as it has all the features I was looking for: I can use Markdown or reStructuredText to write the contents, the source pages are easily handled on a Version Control System, it supports the use of templates for the HTML, etc., but it seemed to me that using ikiwiki was like killing flies with a cannon (that's a Spanish say, I guess it's easy to understand it in English, ¿no?) and I decided to review other tools to build static web sites.

To make a long story short, I selected some tools that met my requirements and looked nice on their demo sites; after my first review I thought that Hyde was going to be my bet, as it uses technologies I'm already familiar with, but after trying it I saw that I was going to have a problem with documentation (the current Hyde version lacks it) and it was going to be more complicated that using ikiwiki.

Before giving up I decided to review simpler tools, just in case, and after looking some of them I ended up using poole, a simple python script (the source is just one file and it only requires python-markdown to work).

Before moving to the content I tried to adapt a couple of free themes to be used by the tool, but I didn't liked the result, so I went back to the plain style provided by the tool and added a logo and a background.

With that simple look and feel I started to work with the content, splitting it into eight markdown files and a python macro to include a file that has all the links used on the site.

While trying to make the main page look good I noticed how little I know about CSS, but using search engines I was able to build a two column block into the main page and publish the contents and with the help of some CSS enabled co-workers I changed the look and feel of the site in about 30 minutes.

In summary, if you want a really simple website, you know a little bit of python and don't want to spend much time learning how to use a website generator then Poole is a good option. If you want something more complex I still think that ikiwiki is a good option, but YMMV.

Posted Sat 01 Oct 2011 23:49:39 CEST

Encrypting a Debian GNU/Linux installation (take 3)

After my followup to the Tuesday post I've received some additional comments and I'm writing this entry to close the subject... ;)

One of the comments was from Gunnar to tell me that the followup setup was the same provided by the automatic partitioner of the Debian Installer since 2007.

I was unaware of that because until some weeks ago I never tried to install a system with encryption support and when I did it on my laptop I used the manual setup because I wanted to keep the MacOS X partitions.

Anyway my followup blog entry made sense anyway, as I just wanted to comment my thoughts about the advantages and disadvantages of each partitioning schema.

I also received a couple of messages proposing the use of three layers to keep the flexibility of the original setup and the simplicity of the second; the setup is as follows:

• Layer 1: use LVM on a physical volume,
• Layer 2: create a logical volume and format it as an encrypted volume,
• Layer 3: use LVM on top of the encrypted logical volume and put there the file systems that you want encrypted.

With the LVM at the lower level you get the advantages of my setup (mix encrypted and unencrypted partitions, the crypted volume can use multiple physical volumes, etc.) and the advantages of the second setup (only one key for all the encrypted file systems).

I believe that this setup is a little too much for a laptop, but can be a good option if you need encrypted file systems on a server.

Posted Thu 26 Feb 2009 08:30:16 CET

Encrypting a Debian GNU/Linux installation (followup)

Yesterday I received a mail message from a Debian user called Ekrem Erdem about my previous post, proposing a different partitioning schema that I found interesting.

The basic idea is to swap the order of the technologies, that is, use LVM on top of an encrypted partition instead of encrypting logical volumes.

I never thought about this schema because I always use LVM on servers and that is one of the fist things I setup (just after software RAID-1, if the machine has two hard drives); when I was evaluating how to setup my system for encryption I started with the LVM setup and never looked back.

The advantage of this setup is that there is only one pass phrase (the one used to unlock the encrypted partition, sda4 in my case), eliminating the need of derived keys (i. e. my swap setup) or key files (I use them to mount snapshots of the encrypted partition non interactively).

On the negative side I believe that this setup looses some flexibility:

• On my original model crypted and unencrypted partitions can coexist on the same volume group, while the new setup requires a different volume group for unencrypted volumes.

• If the user wants to have multiple partitions each one can use a different pass phrase or key file.

• If a logical volume is expanded through multiple physical volumes the new setup requires a key for each physical volume, while the original setup only needs one key.

Anyway if the plan is to encrypt all the file systems on a laptop the proposed setup is simpler and, IMHO, as safe as my configuration (remember that my keys are related).

I'm not going to change my setup now (it works great), but I'll probably try this one in the future if I need an encrypted setup on a different machine.

Posted Wed 25 Feb 2009 00:42:30 CET

Encrypting a Debian GNU/Linux installation on a MacBook

A couple of weeks ago I updated my Debian Sid setup on the MacBook to use disk encryption; this post is to document what I did for later reference.

The system was configured for dual booting Debian or Mac OS X using refit and grub2 as documented on the Debian Wiki; I don't use the Mac OS X system much, but I left it there to be able to test things and be able to answer questions of Mac OS X users when I have to.

The Debian installation was done using two primary partitions, one for swap (I used a partition to be able to suspend to disk without troubles) and an ext3 file system used as the root file system.

The plan was to use the Debian Installer to do the disk setup and recover the Sid installation from a backup once the encrypted setup was working OK.

Backup for later recovery

My first step was to install all the needed packages on the original system; basically I verified that I had the lvm2 and cryptsetup packages installed.

The second step was to backup the root file system; to do it I changed to run level 1 and copied the files to an external USB disk using rsync.

My third step was to boot into Mac OS X to reduce the space assigned to it; I had a lot of free space that I didn't plan to use with Mac OS X and I thought that this was the best occasion to reassign it to the Debian file system.

Encrypted Lenny installation

Now the machine was ready for the installer. As I formatted the system a couple of weeks ago I used a daily build of the Lenny Debian Installer, now that Lenny is out I would have used the official version.

I booted the installer and on the partition disk step I selected the manual method; I left sda1 and sda2 as they were (the Mac OS X installation uses them) and set up sda3 and sda4 as follows:

• sda3: 256 MB, use as ext3, mount point: /boot
• sda4: 86 GB, use as physical volume for LVM

Note that I decided to put /boot on a plain ext3 partition to be able to use grub2 as the boot loader (if we put the kernel on an LVM logical volume we need to use lilo as the boot loader).

Once sda4 was adjusted as LVM I entered on the LVM setup and created a LVM Volume Group (VG) with the name debian, using sda4 as the physical volume.

Once the VG was defined I created a couple of Logical Volumes (LV):

• root: 82 GB
• swap: 2 GB

I left some space unallocated to be able to create LVM snapshots (I use them to do backups, I'll post about it on the next days).

Once the LV were ready I finished with the LVM setup and went back to the partitioner to configure the Logical Volumes:

• debian-root: use as physicals volume for encryption
• debian-swap: use as pascal volume for encryption, encryption key: random

Once both encrypted volumes were ready I entered on the Configure the encrypted volumes menu and the installer formatted the volumes for encryption and asked for the debian-root pass phrase.

Back on the main partitioning menu I set up the debian-root_crypt encrypted volume:

• debian-root_crypt: use as ext3, mount point: /.

I didn't need to touch the debian-swap_crypt, it was configured automatically as swap because I choose a random encryption key.

At this point I was finished with the partitioning; to finish I installed a minimal system and rebooted to try the system.

As I had changed the disk layout I had to re-sync the partition tables from refit; once that was done I was able to boot from the newly installed system.

Setting up suspend to disk

I was using s2disk to suspend the system; to test if it still worked with the new setup I installed the uswsusp package and adjusted the resume device on the /etc/uswsusp.conf to /dev/mapper/debian-swap_crypt.

After my first try I noticed that the resume step failed with the encrypted swap partition because it was using a random key, which means that the swap contents are unrecoverable after a reboot.

Looking at the cryptsetup documentation I found that the solution was to use a derived key for the swap partition instead of a random one.

The command sequence was as follows:

# disable swap
swapoff -a
# close encrypted volume
cryptsetup luksClose debian-swap_crypt
# change the swap partition setup on the /etc/crypttab file
sed -e -i 's%^debian-swap.*%debian-swap_crypt /dev/mapper/debian-swap debian-root_crypt cipher=aes-cbc-essiv:sha256,size=256,swap,hash=sha256,keyscript=/lib/cryptsetup/scripts/decrypt_derived,swap%' /etc/crypttab
# open the encrypted volumes with the new setup
/etc/init.d/cryptdisks start
# enable swap
swapon -a
# update the initrd image
update-initramfs -u

After executing all those commands the suspend to disk system worked as expected.

Recovering the original system

If I were going to reinstall the system completely I would have finished here, but in my case I wanted to recover my original system setup (except the minimal changes required to use the encrypted passions, of course).

To recover my old installation I backed up some files (/etc/fstab, /etc/crypttab, /etc/uswsusp.conf and the current /boot contents to be able to boot in case of failure with my old kernel) from the current installation, after that I recovered all the files from the initial backup (except the ones just saved) using rsync again and regenerated the initrd images of my old kernels:

update-initramfs -u -k all

After that I rebooted and everything worked as on my original installation (except for the disk encryption, of course).

Posted Sun 22 Feb 2009 00:11:52 CET

Redmine

I've been using Subversion and Trac for some years now, and I have encouraged its use at work since the last couple of years, with the undesired effect of having to maintain four different Trac installations with different database systems (SQLite3 and PostgreSQL), plugins (more than 15 on the big servers), authentication systems (htpass files, LDAP and a database based system) and tons of projects published (two internal servers have 64 and 16 projects, one of the client system has 33 projects and there is only one single project installation, but it is living at a client's system).

Yesterday night, while reading Planet Debian I found a post from John Goerzen about tools to replace Trac, including the option to use Git as the project VCS.

In the post he talks about different options, mainly projects that I would categorize as issue tracking systems (mantis, roundup, etc.), but it also talks about Redmine, a project management system implemented using the Ruby on Rails framework that is similar to Trac.

As it looked interesting I downloaded, installed and executed an instance in about 15 minutes (I love the systems that support sqlite3 for this quick tests, not having to touch real database servers speeds up simple tests a lot).

I played a little bit with the system and I believe that I will spend some more time testing it at work next week, as it looks quite promising; the standard version has almost all the features I'm interested in without the need to install additional plugins and it can do most of the things I was missing from Trac to do lightweight project management.

I evaluated ]project-open[ to use it together with Trac for our internal project management tasks, mainly because we miss important features from Trac, like having clean systems to view the tasks of a user in all projects or a clean way to do the project planning using tickets and gantt charts. Of course there are ways to do it, but the plugins I've tried are not as good and simple as I would like.

The problem with the use of ]project-open[ is that I don't really like it for us, as it has tons of features that I feel we don't need nor will use and, on a first try, the system seemed difficult to deploy and maintain, probably because my lack of knowledge about OpenACS and TCL.

In fact we still don't have ]po[ running at work because I was unable to to integrate the authentication system with our LDAP server on my first tries and have had no time to investigate further since then.

The good thing about trying Redmine is that if we don't end up using it at least I can take the most of this opportunity by looking at Ruby on Rails and the Ruby Programming Language, at least from the administration side, as I have never looked at it seriously.

Posted Sat 01 Mar 2008 09:59:45 CET