Tagged with "security"

The Problem

The recent RockYou.com password problems have spawned plenty of debate online about the best way to store passwords and build a site securely.

Part of being a good, security-conscious web developer is paranoia, and it's apparent that the RockYou.com developers could have used a little more of it. They made two mistakes in their work, not one. Their first, and most obvious one, is that they had a SQL injection hole somewhere. Their second was their assumption that their measures to protect their data were enough to do so.

A healthy dose of paranoia would have led their developers to make the opposite assumption - that whatever they did to protect the data, sooner or later someone would be able to access it.

The result of this second mistake is that, rather than simply announcing a security hole has been found and closed, they have had to deal with the fact that the passwords of more than 32 million people have been exposed, in plain text, to an unknown number of people. As most people use the same password for multiple places, and most will be unaware that this has happened, we can safely assume that the access details of millions of email accounts are in the open and unchanged. That's a bad day in code-land by anyone's standards.

Hashing

The solution to the problem is to first assume that all data will be exposed at some point to an intruder of some sort. Once you assume that, it becomes important to ensure that the damage resulting from that exposure is minimal.

Which brings me on to hashes. Hashes are one-way functions that generate a representation, usually a number, of the data put in to them. They always generate the same hash from the same data, and there is no simple way to reverse the process.

This makes them incredibly useful for password storage. Instead of storing a user's password, you can store the hash of the password. When a user logs in again, instead of checking the password they type in against the one you have stored, you calculate the hash of the password they type in and compare that to the stored hash.

There are lots of different hashing algorythms, the most commonly used being MD5 and SHA1.

Are Hashes Secure?

Unfortunately, ensuring passwords are stored securely isn't as simple as just using storing a simple hash of a password. Two of the strengths of hashes are also their largest potential weakness: they are small to store and quick to generate.

To generate SHA1 and MD5 hashes of every word in English, for example, takes moments. To store that amount of data is also trivial. To generate hashes of all combinations of letters and numbers, plus a few commonly used punctuation marks, up to say 8 characters, is much slower but still doable without any special setup or equipment.

Tables of precalculated hashes of data like this are easily found online or easily generated. If you have a hash of some data (like a password) and you want to see what that data originally was, you can compare the hash to the entries in your precalculated table. If you find a match, you have discovered the data that was originally used to generate the hash - the password you were trying to find out.

So basic password hashing is, essentially, useless for the majority of users. It is a simple process to compare hashes of basic passwords to a table of precalculated hashes and thereby "dehash" passwords en masse.

Some people recommend nesting hashes as a way to make add complexity and therefore more security. Unfortunately, to generate tables of nested hashes is almost as easy as plain hashes by themselves, and no more secure.

Add Salt!

The solution is to hash more than just the user's password, and this process is called "salting". For example, instead of storing a hash of a user's password, you could store the hash of their email address and their password together.

This is effective because tables of hashes of generated data of more than about 10 characters start to become problematic to generate and store. At around that point, tables must be generated based upon dictionaries and known words, rather than on programatically generated lists of all possible passwords in a range.

The average length of "email plus password" is easily in the region of 25 characters. Not only that, but if someone worked out that you were using hashes of "email plus password", they would still need to generate a new table for every password they wanted to dehash.

This level of complexity, added to a reasonably strong password policy, ensures that if (or when) your user data is exposed, the work involved in extracting usable passwords from it is going to stop all but the most determined attackers. Not only that, but even they will find extraction of data in bulk prohibitively difficult.

The web has been around for many years, and since the early days, websites have been requested by users and delivered to them using HTTP - the HyperText Transfer Protocol. HTTP is the means by which web browsers and web servers communicate. Unfortunately, HTTP is what we call a "stateless" protocol - meaning, effectively, that when you visit a page on a website, and then another page, the website you are visiting has no way of knowing it is the same person that visited the two pages.

This makes things like user login systems difficult to manage. You can't use IP addresses to track users for a variety of reasons, most notably proxy servers that might be used by several people (many hundreds in the case of AOL) to access the web. This is where cookies come in handy.

A cookie is a simple, tiny, text file. It is stored on your PC, and is incapable of performing any tasks or functions. It is simply a text file containing data in text form, just like this (an example cookie from Google):

Cookies, once stored on your PC, are tied to a specific web address. That can be a domain (eg google.com), a sub-domain (eg www.google.com), or even a folder (eg www.google.com/folder/). When you revisit a web page, your browser checks to see if any cookies that are stored on your PC are valid ones for the page you are visiting. If they are, the information contained within them is sent back to the server.

Which means, essentially, that a website can only get from a cookie information it put there in the first place. Not all that dangerous in the vast majority of cases. With the above cookie on my PC, any time I visit a page within the "www.google.com" subdomain my browser will send the contents of the text file above to Google.

However, in some circumstances this tool can be put to work in a wider context. Companies like "DoubleClick", that provide advertising (just for example), place data (usually images and links) on web pages. When you see an advert (or any image, or iframe, or even JavaScript), that item may not necessarily be sent from the same server as the rest of the web page. It may be sent from another server. This is perfectly normal in the case of advertising providers. However, along with the image, or text file, many advertisers also usually send a cookie set to use their main domain (eg "DoubleClick.com").

The next time you request an advert from DoubleClick (i.e. the next time you visit a web page with advertising provided by DoubleClick, or whoever the advertising provider is), your browser may send back data you received in a cookie from a previous time you viewed a DoubleClick advert. One advertiser cannot read cookies from another, as web servers will only send cookie data out when you request a page from within the domain of the cookie.

What this allows the advertising provider to do is to identify that someone who visited one page also visited another page. It can't identify you personally, but it can tell that someone who viewed a review of "Smokey and the Bandit" on one site then went on to order 6 cases of Jack Daniels on another. Assuming both sites carry advertising from the same provider (increasingly rare). They provide basic thematic profiling - the above example, if repeated by thousands of people, would tell them that people with excellent taste in movies also have excellent taste in drinks. They might use that information to help provide more accurate advertising - Jack Daniels ads on the "Smokey and the Bandit" review, for example.

Can cookies ever contain private information? Yes, they can. A web site can only place information in a cookie that it already has, and only that web site will be able to retrieve that information, so if you have given that web site personal information about yourself, they could store that in a cookie. Is that dangerous? Not in the slightest.

If you want to avoid being tracked by advertisers (and remember, any tracking they do is not personally identifiable to you), there are measures you can take. Most competent browsers allow cookie control (at the end of this article is a guide to preventing your browser from accepting third party cookies). Some allow you to accept and reject cookies individually, or based upon domain. Some will allow you to reject "third party cookies" (the above, cookies sent with advertising, would count as third party cookies), or all cookies altogether.

Personally, I have my version of [url=http://www.opera.com/]Opera[/url] set to reject third party cookies without notifying me. I've had no problems at all using the web since, and running ad-aware every few months turns up no more than one or two tracking cookies, usually ones that have turned up when I've used Internet Explorer for brief testing purposes.

Many people call cookies "Bugs", which makes them sound rather insidious. Misunderstanding and paranoia have made cookies an issue, but for no valid reasons. Most sites have privacy policies now, and I recommend you read a site's privacy policy if you are worried about any cookies they might use. More and more people are just refusing to accept cookies altogether now, but remember - if you reject every single cookie, out of hand, it is ultimately only you that will suffer, as you are just making it harder for the average site to improve your user experience.

How to prevent your browser from accepting third party cookies.

Internet Explorer

1. Click in Tools > Internet Options.
2. Select the "Privacy" tab.
3. Click "Advanced".
4. Select "Override automatic cookie handling".
5. Check that you are set to "Accept" (or "Prompt") First Party Cookies, and to "Block" Third Party Cookies.
6. Though it is up to you, I recommend you also make sure "Always allow session cookies" is checked.
7. Click "ok" to close the "Advanced Privacy Settings" Window.
8. Click "ok" to close the Internet Options pop-up.

Opera

1. Press "ALT-P" or click Tools > Preferences.
2. Select "Privacy" on the left.
3. Make sure "Enable Cookies" is ticked.
4. Next to "Normal cookies", check that "Accept all cookies" is checked.
5. Next to "Third party cookies", check that "Refuse all cookies" is checked.
6. Click "ok".
7. (There are other settings in there you may wish to experiment with until you are happy with your cookie setup).

Mozilla Firefox

1. Click on Tools > Options.
2. Select "Privacy".
3. Expand the "Cookies" section.
4. Check that, of the four boxes, only the top two, "Enable cookies" and "for the originating web site only" are both ticked.
5. Click "ok".