Tagged with "programming"

The Problem

The recent RockYou.com password problems have spawned plenty of debate online about the best way to store passwords and build a site securely.

Part of being a good, security-conscious web developer is paranoia, and it's apparent that the RockYou.com developers could have used a little more of it. They made two mistakes in their work, not one. Their first, and most obvious one, is that they had a SQL injection hole somewhere. Their second was their assumption that their measures to protect their data were enough to do so.

A healthy dose of paranoia would have led their developers to make the opposite assumption - that whatever they did to protect the data, sooner or later someone would be able to access it.

The result of this second mistake is that, rather than simply announcing a security hole has been found and closed, they have had to deal with the fact that the passwords of more than 32 million people have been exposed, in plain text, to an unknown number of people. As most people use the same password for multiple places, and most will be unaware that this has happened, we can safely assume that the access details of millions of email accounts are in the open and unchanged. That's a bad day in code-land by anyone's standards.

Hashing

The solution to the problem is to first assume that all data will be exposed at some point to an intruder of some sort. Once you assume that, it becomes important to ensure that the damage resulting from that exposure is minimal.

Which brings me on to hashes. Hashes are one-way functions that generate a representation, usually a number, of the data put in to them. They always generate the same hash from the same data, and there is no simple way to reverse the process.

This makes them incredibly useful for password storage. Instead of storing a user's password, you can store the hash of the password. When a user logs in again, instead of checking the password they type in against the one you have stored, you calculate the hash of the password they type in and compare that to the stored hash.

There are lots of different hashing algorythms, the most commonly used being MD5 and SHA1.

Are Hashes Secure?

Unfortunately, ensuring passwords are stored securely isn't as simple as just using storing a simple hash of a password. Two of the strengths of hashes are also their largest potential weakness: they are small to store and quick to generate.

To generate SHA1 and MD5 hashes of every word in English, for example, takes moments. To store that amount of data is also trivial. To generate hashes of all combinations of letters and numbers, plus a few commonly used punctuation marks, up to say 8 characters, is much slower but still doable without any special setup or equipment.

Tables of precalculated hashes of data like this are easily found online or easily generated. If you have a hash of some data (like a password) and you want to see what that data originally was, you can compare the hash to the entries in your precalculated table. If you find a match, you have discovered the data that was originally used to generate the hash - the password you were trying to find out.

So basic password hashing is, essentially, useless for the majority of users. It is a simple process to compare hashes of basic passwords to a table of precalculated hashes and thereby "dehash" passwords en masse.

Some people recommend nesting hashes as a way to make add complexity and therefore more security. Unfortunately, to generate tables of nested hashes is almost as easy as plain hashes by themselves, and no more secure.

Add Salt!

The solution is to hash more than just the user's password, and this process is called "salting". For example, instead of storing a hash of a user's password, you could store the hash of their email address and their password together.

This is effective because tables of hashes of generated data of more than about 10 characters start to become problematic to generate and store. At around that point, tables must be generated based upon dictionaries and known words, rather than on programatically generated lists of all possible passwords in a range.

The average length of "email plus password" is easily in the region of 25 characters. Not only that, but if someone worked out that you were using hashes of "email plus password", they would still need to generate a new table for every password they wanted to dehash.

This level of complexity, added to a reasonably strong password policy, ensures that if (or when) your user data is exposed, the work involved in extracting usable passwords from it is going to stop all but the most determined attackers. Not only that, but even they will find extraction of data in bulk prohibitively difficult.

What makes a truly great developer? Some might say a positive attitude. Some might say a high-sugar, high-caffeine, high-bacon diet. Some might say an absence of sunlight and as many monitors as a desk can support.

Certainly, everyone has anecdotes about developers they've worked with who they thought were brilliant. Unfortunately, most of the time that judgement is made not based on code quality, or hitting of deadlines, but on less relevant criteria, like whether or not the developer knew the names of their colleagues, how many lines of code they output or how confident they sounded when talking about their work.

Unfortunately, the best developers don't always come across positively. While this list may not be applicable to every development environment, here are a few of the traits to look out for to spot a great developer.

Pessimistic

Great developers are almost always pessimistic with regard to their work. That doesn't mean they're not upbeat, lively or even cheerful - just that they will always be thinking about what can go wrong and how it can be dealt with.

They'll assume that at some point they'll need to undo work already completed, that hardware will fail, that all security will be compromised, and that your office will burn to the ground. The really brilliant ones will assume that will all happen on the same day. And they won't be happy until there is a specific, actionable, testable - and fully tested - plan for dealing with these sorts of issues. Even then they won't be completely happy.

Pessimistic developers will be the ones that find constant flaws in ideas, and the important thing to remember when working with them is that they're not doing that to tear down other people's ideas - they're doing it to ensure that the ideas that turn into projects are properly thought through and that as many problems as possible have been anticipated in advance. That neurotic, paranoid, pessimistic attitude is exactly what you should be looking for if what you want from your developers is robust, secure, reliable code.

By contrast, an optimistic developer will be more likely to simply assume code will work, or that it is secure, or give a deadline for a project without considering all the potential pitfalls.

Likely to be heard saying: "And what happens when that goes wrong?"

Lazy

Laziness is not usually viewed as a desirable trait, and in this case I don't mean turns-up-late-and-pretends-to-work laziness or just-move-that-logic-to-the-view laziness - both entirely unwanted. I mean a desire to not do tasks that are repetitive, or to waste time doing things a machine can do for you, or even to avoid future work by writing better code now. A lazy developer is one that builds a reusable code library, or wants a fully automated build process rather than a manual copy-and-paste one, or wants comprehensive automated unit testing, or writes code to be scalable even though that wasn't a requirement (rather than revisit it later).

As a bonus, a lazy developer is also usually one who will try and keep a project focussed on its core goals, rather than try and cram more work into the same time, providing a buffer against feature creep.

For example, when writing a category structure, a lazy developer might be likely to assume a many-to-many relationship between parent and child categories, even though the project specification says it will be a one-to-many relationship. Why? Because it might be needed one day and it would be better to write it that way from the start than to revisit it later.

Likely to be heard saying: "We could automate that."

Curious

Good developers are often rather like Gregory House. They're very easily bored by repetitive work (see laziness) and spend most of their time ploughing through it looking for an interesting and challenging (and hopefully new) problem to solve. The less time they can spend on the repetitive, the higher the frequency of the challenges.

Curious developers will be constantly looking for new problems to solve, and better ways to solve previous problems. They'll be the ones encouraging new ways to work and constantly tweaking and trying to improve existing systems. They'll also be the ones most conscious of existing problems in the current working environment, and trying to correct those problems. Curious developers will usually have a wide breadth of knowledge, not just of their primary language(s), but of supportive, associated and alternative technologies.

Curious (or easily-bored) developers are often the least stuck in their ways - the most open to change. They may well need convincing of why a new way of working is better (and that's no bad thing) but as long as it's an improvement, and likely to release more time to spend on the interesting problems, they'll embrace it with a minimum of resistance.

Curiosity also breeds creativity, another highly desirable trait in any developer. A strong desire to work out what has caused a problem and how to solve it is highly likely to motivate someone to continue once obvious avenues are exhausted. It is that sort of mentality that fosters "outside the box" thinking and creative coding.

Possibly the most useful attribute of a curious developer is that desire to find and cure a problem rather than just paper over the crack.

Likely to be heard saying: "Maybe there's another way to do this."

Meticulous

Many great developers are sticklers for detail. They will demand consistency in their work and the work of their team (they're likely to care about common code standards and naming conventions, for example). They'll want unit testing and peer review of code. They'll want everyone in their team to comment on and document code. They are likely to be fussy about version control log messages.

They'll also be fussy about details in communication, and happy to ask what might seem like obvious questions, simply to be sure they have properly understood. This is especially true of things like bug reports. While they may not be terribly motivational communicators, they will usually be able to explain concepts clearly and effectively. That clarity is a tremendous advantage in any development environment, especially if teaching and learning are encouraged.

Likely to be heard saying: "I just have a couple of questions ..."

Translations

• Russian / Русский