On digitized books

For years, Project Gutenberg and related endeavours have been seeking to produce digital copies of books that are no longer under copyright. The Gutenberg people have already digitized 17,000. Purposes for doing so include making machine-readable copies available for those with disabilities, allowing for their use with e-book readers, and even in more creative applications – like printing books onto scarves, so that you can read them on flights from the UK to the United States.

In the grand tradition of huge companies incorporating the results of smaller enterprises, many (if not all) of the Gutenberg books are now available through Google Book Search. Figuring out which Jane Austen book a particular passage stuck in your memory is from has thus become a far simpler task. For years, I have been using The Complete Works of William Shakespeare, provided by MIT, to search through plays.

Admittedly, not many people want to sit in front of a monitor to read an entire book. With the development of electronic paper that has high resolution, high contrast, and no requirement for power consumption while displaying static information, perhaps this will all become a whole lot more useful.

Preferences, re: ponderings

The sky over Woodstock

A comment was posted earlier that has a certain resonance. While there is no greater online sin than blogging about blogging, I will trespass for a moment – always with the aim of pleasing you better, dear spectators. The comment:

You know, if you took the amount of time you spend on a week’s worth of these mediocre mumblings and used it to write one thing, it might be good?

This is a possibility I have wondered about myself. I feel a constant urge to write, but it may be better directed in a less haphazard direction. At the same time, about 100 people a day read the blog; I am willing to bet that is more people than will read my thesis, in total, between now and the end of humanity.

The issue, then, is not the medium, but the message. What would it be more socially useful to write? Before seeing that comment, I was going to write tonight’s post about metallurgy and the possibility that we are living in a ‘composite age.’ Hardly my area of expertise, and hardly an area of interest of most people who I know to be reading the blog.

The format of this blog is intentionally somewhat experimental, as well as somewhat scatterbrained. All told, I am skeptical about whether I can impose a pattern upon something as ephemeral as daily posts, but constructive criticism and suggestions about profitable directions to take would be most appreciated.

PS. We had our 30,000th visit today.

Creepy stuff

Who runs http://www.vroomfondel.co.uk? Also, why do they keep scanning through my blog? They do so through this page and seem particularly interested in anything involving NatWest: the bank where I foolishly opened an international student account.

Is anyone else getting several hits a day from these people? I don’t know who they are, but their URL is registered at the following address:

c/o Net Rank
Suite 1c, Western Way,
Exeter,
Devon
EX1 2DE
GB

Suffice it to say, I do not appreciate their attentions, at least so long as I don’t know who they are or what they are doing. If you run your own server, it is easy enough to ban people referred directly from their strange login site. Just add this to your .htaccess file:

RewriteCond %{HTTP_REFERER} vroomfondel\.co.uk [NC]
RewriteRule .* – [F]

Depending how automated the system is, that might foil it. If it is run by a group of people working through that portal, they will be able to find another way to access your site, regardless of the above addition to your .htaccess file.

A small request for those commenting

I would be much obliged if, when commenting, you would call yourself something other than ‘Anonymous.’ Anything at all that distinguishes you from other commenter who don’t want to leave names, aliases, or initials would be wonderful. At present, threads with multiple commenters, all called ‘Anonymous’, are likely to become rather difficult to understand.

With regards to the need to provide an email address, this is to help prevent spam comments. Using your real email address, will over, time reduce the probability of your comment getting eaten as spam. That said, I do have the ability to see which email addresses people have listed. If you really don’t want me to know who you are, you can always use something like “nottelling@history.ox.ac.uk” or whatever strikes your fancy. Doing so will somewhat increase the probability of your comment being marked as spam, but if you aren’t doing anything else dodgy – like linking to virus laden websites – you should be fine regardless. The system is also clever enough to learn, over time, that comments from a particular computer are safe.

I very much enjoy getting comments and engaging in discussion here. Along with other roles, the blog is a device through which I hope to refine ideas and positions, on the basis of intelligent criticism. As such, all substantive contributions are appreciated.

As always, any technical problems with the blog should be reported on the bug thread.

The awesome power of exponential growth

This blog now has 1/5000th as many registered users as Wikipedia. That may sound trivial, but it should be noted that at the present rate of growth (12.5% per day – welcome Mark), we should have one million in just 99.5 days (by November 13th).

In just 174 days or so, all 6.5 billion human inhabitants of the Earth should have signed up. Don’t be the last!

Apology for cursory treatment

Coming home to 900 blog posts, all laid out in BlogLines, is impossible. My apologies to you all, but your hard-earned thought committed to webservers have mostly been dismissed at a glance. It is a very concrete demonstration of the limitations of all human beings, and the hopelessness of capturing any significant share of human knowledge over the course of our lives.

Something to try over the weekend: cryptography by hand

For about three and a half hours tonight, I awaited essays from next month’s tutorial students in the MCR. Having exhausted what scaps of newspaper were available, I fell back to reading a copy of Dan Brown’s Da Vinci Code, abandoned by some departed grad student.

Two hundred and sixty pages in, and unlikely to proceed enormously further, I note somewhat pedantically that there have been no codes presented. At best, there have been a series of riddles. The book would be interesting for its historical asides, if I could consider them credible.

Rather than go on about that, I thought I would write an incredibly brief primer on how to actually encrypt a message:

Crypto by hand

In the next few paragraphs, I will show you how to use a simple cryptographic device called a transposition cipher. If you really want to learn it, follow along with a pen and paper. As ciphers go, it is very weak – but it is easy to understand and learn. For starters, we need a secret message. The following is hardly secret, but it will do for a demonstration:

“DAN BROWN IS A DUBIOUS HISTORIAN”

Next, we need an encryption key. For this type of cipher, we need two or more English words that do not use any letter more than once. It is quicker if they have the same number of letters, but I will use two with different numbers of letters to demonstrate the process:

“DUBLIN PINT”

Write the first word of the key onto a piece of paper, with a bit of space between each letter and plenty of space below:

“D U B L I N”

Now, add numbers above the letters, corresponding to their order in the alphabet:

“2 6 1 4 3 5
D U B L I N”

Now, add your message (hereafter called the plaintext) in a block under. If necessary, fill out the box with garble or the alphabet in order:

“2 6 1 4 3 5
D U B L I N
D A N B R O
W N I S A D
U B I O U S
H I S T O R
I A N A B C”

Note how each word of the first keyword now has a column of text underneath it. Starting with the first column in the alphabetical ordering (B, in this case) copy out the column, starting at the top, as a string of text. Make sure you understand what is happening here before you go on. The first column, read downwards is:

NIISN

Now, add to that string the other columns, read from top to bottom, in alphabetical order. You can leave spaces to make it easier to check:

NIISN DWUHI RAUOB BSOTA ODSRC ANBIA

Clearly, each column section should have the same number of letters in it. Make sure you’ve got the transcription right before going on. Note that the string above is the same letters as are in the original message, just jumbled. As such, this system isn’t smart to use for very short messages. People will realize fairly quickly that “MKLLINAIL” could mean “KILL MILAN.”

Moving right along…

Take the strong you generated a moment ago, and put it into a block just like the one you made with the first keyword, except with the second keyword. This time, if you need letters to fill out the rectangle, make sure to use the alphabet in order. You will need to remove the excess letters when working backwards to decrypt, so you may as well make it easier.

“3 1 2 4
P I N T
N I I S
N D W U
H I R A
U O B B
S O T A
O D S R
C A N B
I A A B”

Now we have the message even more jumbled. The final encryption step is simply to copy each column in that grid out, from top to bottom, in alphabetical order according to the second keyword:

IDIOODAA IWRBTSNA NNHUSOCI SUABARBB

Note: the shorter the key, the longer each column will be. The above string is your encrypted text (called cyphertext). This final version is a jumble of the letters in the original message. Remove the spaces to make it harder to work out how long the last keyword is. If you like, you can use that put that string through a grid with another word. Each time you do that, you make the message somewhat harder to crack, though it obviously takes longer to either encode or decode.

To pass on the message, you need to give someone both the cyphertext and the key. This should be done by separate means, because anyone who has both can work out what kind of cipher you used and break your code. The mechanisms of key exchange and key security are critical parts of designing cryptographic systems – the weakest components of which are rarely the algorithms used to encrypt and decrypt.

To decode it, just make grids based on your keywords and fill them in by reversing the transcription process described above. I am not going to go through it step by step, because it is exactly the same, only backwards.

If anyone finds out about the credibility of Mr. Brown’s historical credentials, it won’t be my fault.

One word of warning: this system will not keep your secrets secure from the CIA, Mossad, or even Audrey Tautou. This cipher is more about teaching the basics of cryptography. If you want something enormously more durable that can still be done by hand, have a look at the Vignere Cipher.

PS. It is rumored that this very blog may contain a tool that automates one form of Vignere encryption and decryption. Not that it is linked in the sidebar or anything…

[Update: 27 July] Those who think they have learned the above ciper can try decrypting the following message:

BNTAFREEHOOI-LTOSIRISOTWD-FTNWAOEYSOXT-ERASEAAAKGVE

The segment breaks should make it a bit easier. The key is:

SCOTLAND HIKE

Good luck, and please don’t post the plaintext as a comment. Let others who want to figure it out do so.

Strange and annoying WordPress bug

I am abandoning the What You See is What You Get (WYSIWYG) editor that is built into WordPress (they call it the ‘visual rich editor’). It has the extremely nasty habit of randomly inserting literally hundreds of [em] tags and [/em] tags into pages with complex formatting, such as my academic C.V. Usually, it closes every tag that it randomly opens, so the formatting isn’t visibly affected. As soon as you try to change some small thing, however, everything goes insane. Going back through and fixing all of these mangled pages is a big pain.

WordPress also has serious trouble dealing with [p] tags and line breaks.

I hope the cause behind this was identified in the recent bug hunt and will not trouble people after the next major release.

Essential free Mac software

After a year of using a Mac primarily, I have come to appreciate this excellent operating system. I have also come to understand some of the gaps in it, particularly insofar as the software and tools that it includes are concerned. The following, then, is my short list of essential (free) Mac programs. Naturally, they are geared towards the kinds of things I personally do all the time.

1) Adium – instant messaging program

The MSN Messenger client for Mac is quite terrible. It is unstable and badly out of date. The freeware program Adium talks not only to MSN, but to AIM, ICQ, Google Talk, and many other instant messenger services. You see one contact list for people on all the services you’ve listed and the software works well and in a stable fashion.

Make sure to get the Hobbes icons. The one of him dancing, to indicate the presence of a new message, is especially endearing.

One word of warning, all the different preferences can be a bit daunting when you start out. Leave them on the defaults and don’t worry about them.

2) Fetch – FTP client

An FTP program essential to anyone who runs websites. This one is much less temperamental than Cyberduck, which I used for many months before being introduced to this superior alternative. You can apply for a free educational license on the Fetch homepage.

3) Firefox – web browser

Hands down the best web browser for any platform, the Mac version of Firefox is an essential item. I hang on to Safari because it sometimes runs complex Java more reliably than Firefox does (I am thinking specifically of the photo upload script for Facebook). I hang onto Opera because the built in bittorrent support is very useful. With those caveats, Firefox is what I use 99% of the time. At a later point, I should write a list like this of the essential Firefox extensions (SessionSaver, AdBlock with Filerterset.G, and Flashblock come to mind instantly).

4) Google Earth – interactive atlas

Not essential, perhaps, but free and definitely great fun. The built in demonstration tour is worth a look. It shows off the terrain mapping nicely with Mount Saint Helens.

5) jEdit – text editor

Even with MS Office installed, there is no program in Mac OS that can cleanly edit files that must be text only, without formatting. I am talking about things like manually editing HTML files, PHP scripts, htaccess files, and the like.

6) KisMAC – wireless network detector

Particularly if you are running Tiger (OS 10.4), this free utility is helpful for dealing with wireless networks in more sophisticated ways than are possible using the WiFi implementation built into the OS.

7) MacJanitor – maintenance program

If you have a laptop that you leave closed or in sleep mode when you are not using it, chances are some of the timed maintenance scripts that are meant to run under Mac OS are never doing so. By default, they run in the early morning, but that will only happen if your computer is on. This program lets you run them manually, a good idea for maintaining system performance.

8) Password Safe – password utility

The Java version runs under Mac OS and is very helpful for keeping track of the passwords of things you use quite rarely. It is better than Keychain because you can install the Java version on a USB key and then use it on Macs, PCs, and Linux machines.

9) Remote Desktop Connection – system tool

I have no idea why this is not included by default in the operating system. Either this or one of the open source equivalents is necessary to connect to Windows based terminal servers.

10) Skype – VOIP program

Particularly if you have a Mac laptop with a built-in mic, Skype is an exceptionally convenient way to keep in touch with people inexpensively. I really wish more of my friends used it.

11) VideoLAN – media player

This open source video player can deal with the widest range of file types of anything I have used on the Mac. DivX files that simply will not play in Quicktime or Windows Media Player open without trouble, and it has fullscreen mode – a feature that is bizarrely lacking in other Mac video software.

One item that I won’t put on the official list is a third party MD5 hash checker. Only people who need to check the integrity of downloaded files will need one and it doesn’t really matter which one you choose. Just don’t trust the one built into Disk Utility (at least not for .iso files).

PS. The essential non-free software is basically MS Office (OpenOffice does not cut it when you need to collaborate with people using Office) and Photoshop 7, CS, or CS2.