I’ve had a good time at Ghost, with the position offering me the opportunity to learn a range of new tools and techniques (Sass, Git, Lemonstand, et al), but I feel that I’ve somewhat outgrown it (too much of a focus on frontend technologies), and the desire to move onto bigger and better things lead me to School Stickers.

I’m sure that once I start there I will be posting more about the company and my role, etc.. but for now, lets just say they have a really cool system for promoting good behaviour in schools, which you can find out more about at Carrot Rewards and Classroom Carrots.

Anyway, now that I’m leaving, Ghost Design has a position open for a new web developer, so get your CV and portfolio ready, and you could be taking over my job.

But what’s the fun in that? So rather than do the sensible thing, and just upload the site, I instead chose to start fresh and build the site all over again from scratch.

Why? Well, there are a couple of reasons.

The first is that I had spent a good portion of the last week updating the limeBase theme on which the Lime Blast site was based. These updates included the latest versions of HTML5 Boilerplate, improved CSS structuring, improved media query responsiveness, and quite a lot of tweaks to the WordPress specific functionality.

The second is that the majority of time spend building the initial site was experimentation to see what was possible. I’m still new to WordPress, and while I’m much better at building for it than I was four months ago, I know I’ve still got a lot to learn. I’m also starting to understand HTML5 and CSS3 much better than I did before, but the downside of this was the labrynth of HTML and CSS that I had left behind. Pretty much the same thing happened with the Ghost Design site.

The way I see it, this site is my most important showcase – if I can’t show the world how good I am on my own site, then how good am I going to be working on someone else’s site?

It would appear, however, that whoever it was that hacked into my server and turned it into a spam-sending machine, had other ideas.

(This post is a bit long and rambling, so feel free to skip to the end if I start to bore you.)

A bit of background

This time last year, if you had asked me about running my own Web server, I would have told you that I had no interest in it. I was perfectly happy with the cPanel based shared hosting that I was been using, and didn’t see the need for anything else.

This all changed during my time at Propeller Communications, where I was introduced to version control. My first taste of a version control system was Mercurial, and while my own experience of it was rarely bad, and the bundled TortoiseHG was a joy to use, it didn’t take long for me to realise that the de-facto industry standard was Git (thanks, mainly to GitHub), so upon leaving Propeller, I made the switch.

The list of benefits afforded to users of version control is long, but the one benefit that really caught my attention was the ability to push changes I had made on my local machine directly to the server. No longer did I have to use FTP to upload the correct files to the correct place, while remembering to removing files that weren’t needed any more. I simply had to run one command, and everything was taken care of for me.

But, in order to reap the benefits of Git, I needed my sites to be hosted somewhere that supported Git – and to date I’ve yet to find a shared host that does. So it was time to say goodbye to cPanel, and say hello to SSH.

First tentative steps

Prior to setting up my live server, the one the hackers took a fancy to, I had built two local development servers.

The first, which was as much an experiment as anything else, was in built out of an old PC in the office at Ghost Design. The process involved booting off the Ubuntu Server (10.11) CD, selecting all the options I wanted (LAMP, DNS, SSH, etc..), then, after watching the progress bar complete, I installed Webmin to help manage it. This, I felt, went quite well, and allowed me to gain more confidence using the Linux command line.

The second, which I use as a development server in my flat, was built out of a PC that I had been using as a Windows-based media server. The install of Debian 6 was a much more involved process because I had decided to install all the software I wanted manually. I also avoided installing Webmin, as I wanted to learn how do things properly.

The success of these two servers had filled me with confidence, and so on the 8th December 2011, with a small loan from my mum, I ordered a dedicated server from Hetzner. Because a development server should be as close to that of the production server as possible, I went for Debian 6 again, and for the most part, followed the same instructions as I had for my home server.

Skip to the end

On the 30th January I received an email from Hetzner stating that an ISP had reported my server for sending spam. I forwarded this email to Phil, who suggested that an incorrectly configured mail server might be at fault. After removing EXIM, I thought that would be the end of it, but three days later I received another abuse report.

A full week later I was still receiving abuse reports, and crying out to Phil for help. I have no idea how he managed it (via the use of the occult no doubt), but he tracked down the culprit: a whole bunch of unexpected files located in three of the sites/vhosts I was hosting.

Two of the sites were based on WordPress. I vaguely understand how the open source nature of WordPress, combined with an out of date install and some lax permissions, could allow someone to search the source code for exploits, then search the Web for an exploitable server. But the third was a static HTML site, meaning whoever had done this had been able to get access to it from one of the other two sites, meaning, potentially, the entire server was compromised.

To stop the immediate issue of spam being sent, I had to turn the server off, and following Sheepy‘s advice, I’m going to “Nuke it from a great distance and start again”.

Prevention is better than cure

So what can I do differently to prevent this from happening again? I think my main issue was that of permissions. You can afford a level of flexibility, and a more relaxed attitude to permissions on a development server, because, for the most part, it isn’t accessible to the outside world. For obvious reasons, the same isn’t true of a production server.

I’m also going to make sure that any software I use on the server is kept up to date, thereby increasing the chance of exploits being fixed.

Anyway, I’m going to reinstall the server soon, and I’m still hoping to write the server newbie post, so watch this space.