It is amazing how many people hire online marketers without the faintest idea of what online marketers actually do. Search engine optimisation (SEO) is fairly simple - SEOs will try and improve your site's performance, usually by trying to leverage their knowledge of how search engines work and tricks they can use to make sites seem more relevant than they actually are to specific keywords.

Marketing online, though, need not have anything to do with search engines. Search engines are irrelevant - good positions and traffic are a by-product of effective online marketing.

Unfortunately, after educating a client on what online marketing is, they usually assume that if they pay you a few hundred pounds, you can make their site compete with the very best out there.

• Client:
  'Ok, I see. Great positions aren't necessarily worth much unless there are customers searching for those keywords.'
• Me:
  'Right. We want high traffic, but not if it's not going to be bad for your bottom line. Traffic that doesn't convert to sales just costs you money. Same applies for phrases people never search for. No point being number one for the phrase "fish banana druid" - it's likely to get you as many customers as peeing on people that walk past your shop will.'
• Client:
  'Ok, so if I pay you, say, £300, how long before I'm at number one for this list of relevant phrases?'
• Me:
  'You wouldn't get in a boxing ring with Joe Calzhaghe after jogging a couple of miles and doing a few push-ups, would you?'
• Client:
  'Well, no.'
• Me:
  'Exactly. To compete with the big dogs, you need to think bigger. Your site is a 10 stone weakling at the moment, and the aim is to turn it into a champion. It needs to be Rocky Balboa. You won't get the top spots quickly - this takes time and hard work. And it's not cheap.'

People are obsessed with money. Absolutely obsessed. Even more so in a company environment. The chances are the most of the time, the person you are talking to at a client (or potential client) company is not the top dog. They have to justify their decisions, and they certainly have to justify what they spend.

The problem is that the way most people look at SEO (and they are thinking SEO, not marketing - it's up to you to show them the difference) is that they're going to pay a certain amount of money for the top spots for certain keywords. You can guarantee they've been told another company will guarantee 10 number 1 positions for $50.

This is where ROI comes into play. ROI stands for "Return on Investment". Paying $50 for a $0 return is a bad idea - but people do it all the time, because it's cheap. Paying $5,000 for a $50,000 return is a great idea - but people gasp at the very idea they could spend that much in the beginning, despite the potential.

In order to measure a return, you need to use tracking. If you're focussed on natural search, measure natural search traffic. See how many people come to the site, and where from. See where they go in the site. See if they view products, add them to a basket, and complete sales. See if they view products then come back weeks later to buy them. Measure that over time and you can tell a client exactly what effect your marketing campaign is having - and you will be able to show them what they are getting for their money. Usually, telling a client you are going to do this will also put their mind at ease - much easier to spend money on someone when that person tells you how they're going to measure their success. Most companies involved in SEO and online marketing focus on positions, not results.

• Client:
  'That's good to know. If I can see what's going on, I can give hard numbers to my boss. I'd rather tell him we have 10% more visitors and 20% more sales than tell him we're in top positions for our target phrases but traffic has gone down.'
• Me:
  'Woohoo! You've taken your first step into a larger world.'

The other thing to bear in mind with money conversations is that most companies think of their site like a brochure. They think of it as a print-like cost, where they pay a fixed sum and that's it. They put the site up, leave it, and expect results. They should be thinking of a site like a salesman. A salesman that never sleeps, rarely gets ill, and can handle virtually unlimited enquiries. As such, they should be thinking of the money they spend more like a wage.

• Client:
  'We're spending $200 a month on our site now for hosting. Are you saying we should be spending a lot more?'
• Me:
  'What would you pay a salesman with the figures your site has, ignoring PPC?'
• Client:
  'Probably $3000 a month.'
• Me:
  'Then that's what you should be spending on the site. As the figures get better, spend a little more. Remember that that needs to include redesigns, hosting and other costs.'

(Note: PPC is something of a difficult subject to bring in to a monthly spend on a site. You should have a monthly spend on PPC, but it should be managed as a separate entity.)

The same traffic you are monitoring to see where site visitors are coming from and what they are doing when they reach the site can also give you some good places to start making changes. Break the traffic down by area, by language, by time of day (user time of day, not server time of day), and track who converts to a sale and who doesn't. Track people through the sales process, and watch which links they click to navigate and buy products.

This will tell you a huge amount about the current users of the site. It will show you quick wins, opportunities, and highlight problems. Forget search - if on your first day marketing a website you can spot that there is a problem with the site checkout process and get it fixed, you could double sales from existing users. That's a good start to any campaign.

Look at language and area closely as well. If a site is getting traffic from the US, but only sells to the UK, look at similar companies only serving the US and strike a deal with them. You direct US traffic to them, they direct UK traffic to you, and you both do slightly better.

Check browser usage stats, especially if the site is a tables-based dinosaur. The chances are that it is an inaccessible mess. Get it cleaned up! Semantic markup is key - it allows user agents (browsers, search engine spiders, screen readers) to attach specific meaning to different areas of a page. Unlike with tables, semantic markup allows you to differentiate between a header and normal content, or to identify an address. Accessible coding is likely to draw attention, and should help you retain a higher percentage of your visitors, and should help reduce the running costs of your website (lower bandwidth bills and quicker turnarounds on redesigns, for example, both save you money).

• Client:
  'I don't care about different browsers though - they only make up 1% of my traffic. Everyone else uses Internet Explorer'
• Me:
  'Perhaps it is because your site doesn't work in other browsers that that number is so low. Even if you do have 99% of users on the same system, the other 1% is still important. Techies use different browsers and operating systems. Techies are also often the people who are asked by their families if they know a good site to buy something from. Many directory editors are in the same boat, and techies can create links to your site.'
• Client:
  'Ok, techies are important. But do I need to care about blind users and all that accessibility stuff?'
• Me:
  'Yes, of course. It's a legal obligation for one thing, but users with sight problems make up a far larger proportion of your audience than you might think. They have a voice too - and it's far harder to undo the damage some adverse publicity can do than it is to make a site work properly in the first place. Finally, search engine spiders are blind users with no JavaScript support.'

Dynamic sites are slightly trickier to improve. Most of the time, they are restricted, with the original authors not allowing access to the website code. Even if access to the code is allowed, changes may be overwritten later or worse cause immediate problems on the site. That said, making a site easier to use is important, and often dynamic sites are not easy to use.

Look at the pages users visit in the site, and how they get there. Look at the products they buy and spot themes. Use that information to make the important sections and products easier to find and organise. For example, if listing products, don't make people click through 4 levels of navigation to find them - improve the product navigation. Once they get there, allow them to reorder the page according to what they consider important, be that name, price, manufacturer - whatever is possible.

Remember also that people like to tell other people about things they find. If a user likes something on your site, they may email the address of the page they are on to a friend. Most people use forms to set the ordering criteria of a page. That means that the user will be sending a friend a URL that will show that friend something different to what the user currently sees. Make life easy for your users - use URLs, not forms, wherever possible in a site.

• Client:
  'I am curious about one thing. We're already really well ranked for the name of our main product, and lots of people search for it. The people that visit our site tend to buy the product. But I can't help feeling that there should be more people coming from the engines. Any ideas?'
• Me:
  'Yes. The Overture Search Term Suggestion Tool', capitalising my speech for no good reason, 'shows that millions of people search for that phrase. I can see you have a top spot. And your traffic is surprisingly low, but converting well.'
• Client:
  'So I'm not imagining it then - we have a problem?'
• Me:
  'Yes, we do. When a user looks at search results, they scan the first two or three words of each link. Your link says "Arthur Jackson Ltd. Sheds and other garden products." That comes from your page title.'
• Client:
  'And that's bad?'
• Me:
  'Most people will only glance at "Arthur Jackson Ltd". You need to show them, in the first two or three words of your page title, that you have what they are looking for. And you're not doing that. The user has no reason to click on your link ahead of all the others they see.'

Titles are tricky. They're important to the user, they provide the text for bookmarks, they appear in search results, and search engines use them as part of ranking algorithms. You need for fit branding into a title, and describe a product, ideally also incorporating a call to action. Tricky stuff. But not impossible.

First, consider the brand. Most companies think their company name should be the first thing in a page title, even if the rest is unique for each page (as it should be). However, unless the company has a household brand name, the company name is irrelevant to the searcher. They're looking for a product (or the answer to a question), so show them you have it.

Next, remember that as titles are used as the text for bookmarks, links and appear in search engines, they should, when taken out of context, by themselves, leave no doubt what a page is about.

A good example of a title is:

• "Norwegian Blue Parrot - Buy Norwegian Blue Parrots from Mr. Praline's Pet Shop".

You've included the all-important product name twice in the title, along with a call to action, a hefty dose of branding, and not added irrelevant information. It's a title that tells the user straight away what the page is about. No messing around.

• Client:
  'Ok, the titles need sorting, but what about the content of the site? I keep hearing that "Content is King".'
• Me:
  'Content is, ultimately, King. Sites with lots of great content will, over a decent time period, far outperform sites with no original content. But content doesn't just have to be on site ...'

Product is important. The object you sell though is only half of the picture. A user will want support from you. They will want information. They may want news. All of this is part and parcel of the package a company offers. Your site needs good, visible support (including a phone number), as well as plenty of good, original information. Guides to products, online manuals, FAQs, advice - there are always areas, in any industry, where content can be added.

Content need not be solely posted on the website either. Big news should be released as a press release, and there are plenty of services that will distribute press releases for you. These will be reproduced all over the web, allowing more and more people to hear of the company. Most press release services will allow you to embed a link to a site in a press release, generating more direct traffic as well.

When writing content, or advising on the writing of content, remember that it is not about keywords. Sure, keywords are important, but there is more to it than simply stuffing as many keywords into text as possible. Content needs to answer questions - to provide information. It needs to give a user what they are looking for, and they need to feel that it has done that. Content that is written for SEO can read very badly with too many keywords in, and can mean that although more people see an article, most of them leave the site straight away to find a better one.

A good way to add content to a site is a blog, or a news section. Aside from adding plenty of information, this gives a great opportunity to connect with the user. Consumers are constantly being targeted, from every angle, by companies anxious to take their money. Sometimes they get trodden on. When adding content to your site, stay on the side of the average consumer. Recently, in the UK, the energy companies all raised their prices dramatically. Sites that allow users to compare fuel prices almost all missed a great opportunity to have themselves noticed - not one of them posted a decent news item denouncing the changes as unnecessary or over the top. They all simply commented on the change factually.

While on the one hand, some of these companies may be unable to comment in this fashion (and many companies have strict policies regarding neutrality and customer perception), at least one should have been able to stand out by taking a clear, customer-supporting position on the issue. That is the kind of thing that gets companies noticed and remembered, and spotting opportunities like that is key to a good marketing strategy.

Not all content need be inflammatory of course. It does need to be unique in some way, however. It can be controversial, but it could also be definitive - the ultimate and complete guide to a topic. Controversial content is interesting to the user, and definitive content is just plain useful - either makes for good content for any website.

Users go through different stages when buying products, and one of the early ones is a research stage. There is always a good chance that a user will come back to the same place that helped them or impressed them when they were doing research to buy what they were looking for. This is branding - associating specific ideas and feelings with your company. You want your users, when they revisit the web to make a purchase, to think of your company first.

Which brings us nicely to our last, and most important point. Why would a customer think of any company first, ahead of any other. Content will help, yes. A nice design might even make a difference. More than anything else, though, customers pay attention to the company that stands out from the crowd - the company that is different, that offers them something nobody else does. Often known as a Unique Selling Point, or USP, this is the thing that makes you memorable, or if ignored helps you blend into the crowd.

• Client:
  'But we don't have a USP. How do we get one?'
• Me:
  'Well, hang on one minute. You say you don't have a USP, but is there nothing about your product that makes it better than the alternatives?'
• Client:
  'Well, we sell Norwegian Blue Parrots. They're all the same, really. Although a rather large proportion of our competitors appear to sell mostly dead ones.'
• Me:
  'There you go then. Your USP is that your product is, in fact, not dead.'
• Client:
  'By that reasoning, a USP could be almost anything, when put in the right light. And when did we turn into a Monty Python sketch?'
• Me:
  'Be quiet about the Monty Python thing. Yes, though, a USP can be virtually anything. It can be quicker delivery than competitors, better products, better customer service, a freephone enquiries number, or simply the people that run the business. Almost every business has a USP - although most of them don't know what it is.'

Many businesses don't know their own USP. They can't tell you, when you ask, what makes them different. Many of them will just say "because we're better than the others", but can't explain why. Usually, however, a quick chat will reveal what makes them stand out. Whatever the USP is, it needs to be clear and obvious on the website. The customer can't miss it, because if they don't know what makes one business different from another, they're not going to remember it.

• Client:
  'What about search? You've not told me how to get my site to the top of the search engines!'
• Me:
  'Let's review, shall we. You've changed your site substantially, so that it meets current standards and you can sell to more of your users. You're showing your clients why you are better than your competition. You've started releasing press releases, and adding content to your site. You're championing the cause of the common man, increasing link numbers and getting people talking about your business. And you know how your users find your site, and what they do when they get there.'
• Client:
  'And?'
• Me:
  'You're positioning yourself as a great resource for your market. Your search engine rankings will come as a direct result of everything else you are doing. You're going to perform well in search, as a direct result of good marketing.'
• Client:
  'I'll get my chequebook.' (Hah. As if.)

Anchor text is the text used to link to another site. In this example - Google Web Search - the anchor text is "Google Web Search".

Atom

Atom is a file format used for web feeds. It is a type of XML document, and is used in syndication.

Black Hat

Black Hat is the term used to describe techniques used by some search marketers to promote websites. These techniques are those that go against guidelines published by search engines, and in many cases their use can result in a site being penalised or removed from search engine listings. Black Hat is the opposite of White Hat.

ccTLD

A ccTLD is a country-code top level domain. .uk, for example, is a ccTLD, as are .au (Australia), .de (Germany), .fr (France), .ca (Canada) and .nz (New Zealand).

Click-through Rate

See CTR.

Cloaking

Cloaking is a technique used to show content to a search engine and different content to a user. The content shown to the engine is usually designed to help a page rank very well for a certain phrase or word, and the content shown to the user usually designed to maximise the conversions from that page. Search engines dislike this technique and many sites are banned for using it. It is a Black Hat technique.

Conversion

A conversion is when a website user completes a specific goal. With some sites that can be to complete a sale; with others, to sign up to a newsletter; and with others to make an enquiry.

Cookie

A cookie is a small text file stored on a website user's computer. It identifies a repeat visitor to a site, often with a unique code, allowing people to shop online and removing the need to log in to sites repeatedly. Cookies are often considered dangerous by less experienced web users. You can find out more about cookies in Are Cookies Dangerous?

CPA

CPA stands for "Cost-Per-Action", and is a form of advertising model. The idea is that an advertiser pays a specific amount for each successful conversion, be that a sale or a signup.

CPC

CPC stands for "Cost-Per-Click", and is a form of advertising model. The idea is that an advertiser pays a specific amount for each visitor referred to their website, regardless of whether that user converts to a sale or not.

CPM

CPM stands for "Cost-Per-Mille", and is a form of advertising model. The idea is that an advertiser pays a specific amount for every thousand times his advert is seen on a site, regardless of how many of the users who see the advert click on it and visit the advertiser's site.

Crawler

See Spider.

CTR

CTR stands for "Click-through Rate". It is an indicator of the percentage of people who see an advert who actually click on it. For example, if one out of every hundred people who view an advert click on it, the advert with have a CTR of 1%.

Directory

A directory is different to a search engine in that it organises the sites it lists in categories. Sites are usually added by hand, rather than found using a spider, and often a small fee is charged for this addition.

Data Center

A data center is a large collection of computers that hold information for a search engine. Major search engines have several of these around the world. Their purposes is to process search queries.

Doorway Page

A doorway page is a page designed specifically to rank well in search engines. Often a visitor to a doorway page will not notice they have visited one, as they will be sent straight on to the target page instantly. Use of doorways is a Black Hat technique.

Feed

A feed is a file that users can download that contains information about recent updates and additions to a website. Often these feeds are used for syndication purposes. Using feeds and programs designed to use feeds, users can often keep up to date with many hundreds of websites.

FFA

FFA stands for "Free-For All". It is usually used in conjunction with links pages that allow anyone and everyone to add a link to the page.

Google Dance

The Google Dance is the name for the process Google used to go through very regularly when it updated an algorithm. As various data centres around the world were progrssively updated, people would be able to make the same search several times in succession and see different results each time. The Google Dance does not happen as often now, but can still be seen when major changes are made to the Google infrastructure or algorithms.

Hit

A "hit" can mean one of two things.

• When searching the web, a hit can be a result found by a search engines that matches the search criteria.
• In analytics, a hit is when a file is requested by a server. Some people have used hits as a measure of website traffic, however hits to a server include images and repeat visitors, and are a poor indicator of traffic. One thousand hits very rarely equals one thousand visits.

IBL

IBL stands for "Inbound Link", and refers to a link pointing to a website from a separate website (unlike an internal link, which refers to a link within one website pointing to somewhere else within the same site).

Impression

Impression is the word used to describe a single viewing of something. A page impression would mean a single view of a web page. In advertising, one impression is a single view of the advert.

Keyword

A keyword is simply a word used to describe a page. It can also be a word used by someone trying to find a site, entered into a search engine.

Keyphrase

A keyphrase is very similar to a keyword, except that it is a phrase made up of several words.

Keyword Stuffing

Keyword stuffing is the practice of repeating a keyword (or keywords) far too many times throughout a page. It may be that the keyword is repeated so many times in the text that as a result the text reads badly. It may be that it is repeated lots of times in meta tags, or elsewhere in code, or it may be a combination of these things. Common practice in the late 90s, this is now considered a technique that may harm a site more than help it.

Link Building

Link Building is the process used to increase the number of links to a website. This can include submitting a website to directories, creating more content for a website, link rental, and many more techniques. Most search engines now use link data extensively in their algorithms, and so link building has become far more common.

Meta Data / Meta Tag

Meta Data is information held about a page or document. It is usually held invisibly within the page, and may include a description of the page, a list of relevant keywords, or the name of the author. For a full explanation of common meta tags, and how to work out which ones are worth using, please read Meta Tags.

Page Title

A page title is an important part of a page - it is usually the part of the page that appears as a link in search results. It is usually visible in the title bar of your browser while you are viewing a page.

PageRank / PR

PageRank is an algorithm, developer by Larry Page and Sergey Brin, founders of Google. It allows you to find the "best" pages of a group of pages by looking at how the pages link to each other. The more links a page has, the better it is considered, and the more important its links, in turn, are considered. PageRank is named after Larry Page.

Pay Per Action

Pay Per Action advertising is the same advertising model as CPA, in that an advertiser will pay every time a user completes a specific action.

Pay Per Call

Pay Per Call advertising is a subset of Pay Per Action, and is the same advertising model as CPA, in that an advertiser will pay every time a user calls a specific number.

Pay Per Click

Pay Per Click advertising is the same advertising model as CPC, in that an advertiser will pay every time a user clicks on their advert.

PFI

PFI stands for "Pay For Inclusion". Some engines will charge sites to be listed at all in their results (notably Yahoo for many years). Prices vary greatly, and some engines charge annually, where others charge a one-off fee. This is a far more common feature of directories than search engines.

PPC

See Pay Per Click.

Robots.txt

A robots.txt file is a simple text file that contains instructions for search engine spiders. It can tell specific spiders to slow down, or not to index specific area of a site. For more information, please read robots.txt.

ROI

ROI stands for "Return on Investment". It is a measure of the success of any marketing campaign. A marketing campaign that cost ?10,000 but made ?3,000 would obviously have a low ROI. A marketing campaign that cost ?10,000 but made ?100,000 would have a high ROI.

RSS

RSS is a type of XML file, and is the most commonly used file format for website feeds.

SEM

SEM is an acronym of "Search Engine Marketing". SEM is a broader topic than SEO, and can include, for example, an online PR campaign or PPC (and other forms of) advertising.

SEO

SEO is an acronym of "Search Engine Optimisation", and is the art of altering a website to improve a site's performance in search engines (note: an improvement in performance does not equal an increase in traffic!).

SERPs

SERPs is an acronym for "Search Engine Result Pages".

SEs

SE is an abbreviation of "Search Engine".

Site Map

A site map is a page, or set of pages, on a website, designed to help users and search engines find their way around a site.

Spam

Spam has many different meanings on the web. The most common meaning is related to email, where spam describes unwanted email, often commercial in nature, and often sent out indiscriminately to millions of people at once. In a search engine context, spam refers to pages that are listed out of place. This can mean pages that are found for keywords unrelated to their content. It can also mean pages appearing unnaturally high in search engines. These pages are often promoted using Black Hat techniques, especially cloaking and doorway pages.

Spider

A spider, also often called a "crawler", is a program created by a search engine to index pages on the web. It visits pages on the web, collects their content, and finds links within that page. It then adds the links found on that page to those it intends to crawl.

Splash Page

A splash page is an introduction page to a website, often created using flash. They are much derided, as they slow down access to a website and often provide no useful information to the user.

Stop Word

A stop word is a word that is ignored by the search engines. It is a word that appears so often on the web as to be useless to a search engine. Examples include "a", "and", "I", "you" and "it".

Syndication

Syndication is where a website makes information available for others to use. In the majority of cases, the information available is a list of the content most recently added to the site (a feed), to allows visitors to keep up to date easily with new content added to many sites.

Text Link Ad

A text link ad is a type of advert on a website, placed in return for a simple monthly fee. These types of advert can have a positive effect on a website's SEO campaign, and can directly generate traffic to websites.

TLD

A TLD is an acronym for "Top Level Domain". .com, .org, .net, .biz, .info, .name and .pro are all examples of TLDs. They are usually global TLDs, unlike ccTLDs, which are country-code domains.

URL / URI

A URL (Uniform Resource Locator), sometimes (more correctly) referred to as a URI (Uniform Resource Identifier), is in basic terms a web address. For example, "http://www.addedbytes.com" is a URI.

Visit

A visit is different from a Hit or an Impression, in that it indicates a single person's visit to a website. A visit may include many page impressions, and many hits.

White Hat

White Hat is the term used to describe techniques used by some search marketers to promote websites. These techniques are those that adhere to the guidelines published by search engines. White Hat is the opposite of Black Hat.

XML

XML is a file format designed to create files that are easy to share and understand.

del.icio.us describes itself as a social bookmarking site, for it allows all users of the net to share their bookmarks with others. Unlike similar enterprises that went before it, users' shared bookmarks are not listed only under their name. Each bookmark also has a set of "tags" associated with it. These tags are words that identify what that page is about. An article on, well, security in PHP would have the tags "php" and "security". Maybe even "webdev" and "programming" as well.

Each user can pick their own tags for their own bookmarks. You can also browse all available bookmarks by tag - meaning that if you wanted to see all bookmarks about PHP, you would simply browse to the PHP tag, and voila - there you would find all bookmarked pages about PHP.

What's Wrong with Directories?

Tags are an intelligent way of organising data. Regular directories work on a similar basis to a filing cabinet - items are stored within folders within drawers, and often only to be found in one place. Of course, that fails to make use of the power of databases and computers. Tags, on the other hand, allow a single item to be found in all the places it should be, rather than just the one place that is the single best fit.

This site, for example, is listed at DMOZ under "Web Design and Development: FAQs, Help, and Tutorials" - a good fit, but it also contains writing on internet marketing, browsers, usability and accessibility, and there are resources available as well as a blog. DMOZ cannot reflect this with its rigid and antiquated structure. At del.icio.us, however, it is associated with the following tags: css, php, design, web, programming, blog, webdev, blogs, development, webdesign, reference, cheatsheet, resources, code, mysql, tips, apache, web-design, tools, tutorial, tech, tutorials, computer, web-dev, html, database. And that's just the front page - specific articles are all listed with their own tags. This means that sites and pages listed by users of del.icio.us are classified and organised in a much more effective and user-friendly way.

There are more serious flaws in the directory model though. The most significant problem with web directories is the editors themselves. A web directory requires editors in order to function, and these can either be paid employees or volunteers. If your directory has paid editors working for it, you are left with no serious choice but to charge a fee for submission, in order to cover your editors' wages. That system scales pretty well - if the directory succeeds and becomes popular, the fees for the extra submissions should be enough to cover the wages of the extra editors required to process those submissions.

A volunteer system does have advantages over the paid-editor model. Because volunteers are not paid, a listing in a directory with volunteer editors can be free. This means that non-profit information sites and low-traffic sites can be listed in the directory (a fee for submission will usually prevent that), and means that editors can go out and find sites themselves to be listed.

Both of these systems have their problems. Volunteer editors are volunteers - making it much harder to hold them accountable for laziness or incompetence. DMOZ - a directory with around five to ten thousands volunteer editors - is a great example of this: submissions are often not processed for many months, if at all. Also, because it is a volunteer position, uncrupulous folk are far more likely to accept bribes to list or de-list sites - they stand to lose very little if discovered - and there are plenty of people who claim that a great many editors do just that. The system can also scale badly - if tens of thousands of submissions suddenly require processing, it can be very difficult to source the hundreds or thousands of editors needed to manage that influx.

The paid system leads to an exclusive directory, which by definition becomes one that is missing out on a huge amount of quality content. Yahoo's fees of hundreds of dollars for a submission have always seemed to many to be completely disproportionate to the benefit of a listing and for many people are higher than the cost of hosting a site or the income from it. As a result, for a long time (and this is still true to a great extent) Yahoo has been an incomplete directory, lacking the in-depth listings required by today's discerning web surfer.

Why is del.icio.us better?

del.icio.us is different to both of these systems. It is similar to a peer-review system, in fact. One person bookmarking one page can count as a vote for that page. As the user will have added tags as well, their vote tells the system that one person believes that the page in question is related to each of the tags they listed. After a few hundred people have bookmarked the same link, you'll begin to see some tags used more than others, giving you an idea of how closely the target page relates to each of those tags.

What this has created is a kind of directory with a distributed editting system. The editors are volunteers, but because of their sheer numbers it it much harder for any one editor to affect listings. If one editor is lazy, it does not matter - there are thousands more covering the same topic. If an editor makes a mistake, and lists a page or site under the wrong tag, it doesn't matter - the huge numbers of other editors will make up for it.

Spam is likely to become a huge problem for del.icio.us. To a degree, there is already spam within the index, and some work has already been put in to preventing spam. del.icio.us's robots.txt file prevents indexing of the whole site, so no link popularity benefits from spamming the index directly. However, del.icio.us can still generate plenty of traffic and by virtue of the RSS feeds it generates can generate link popularity from other sites.

Luckily, there are plenty of signals they could look for to weed out spam. Their database can already tell them what tags are related, and what sites. If a user starts to list unrelated sites with tags unrelated to those sites, they may well be a spammer. If lots of new users suddenly join and all bookmark the same page instantly, using the same tags, again that may well be spam. IP tracking and the registration system (that requires a valid email and features a turing test) should make automated spam far harder. Ultimately, it may be del.icio.us's own success that makes spamming virtually impossible. With enough users on the site, a spammer may need to create hundreds, even thousands, of fake users to have a site listed in the "popular" section, or listed highly for a specific tag.

What next?

The intention of del.icio.us is not (at the moment) to become or create a directory. It is by pure fluke that they have created a site and a system so able to perform the same function as a directory but without the problems associated with that. It may well be that other similar sites will spring up whose aim is to build a directory, especially those involved in search, with a large user base. I would be greatly susprised if Google, MSN and Yahoo were not already watching del.icio.us very closely and with great interest. I would be equally surprised if none of them bought or created a social bookmarking product in the next few months, as the power of a distributed editting becomes apparent.

Update (14th December 2005)

It appears that I was rather close to the mark with my guess at what would happen next for del.icio.us - they have just been bought by Yahoo. It will be interesting to see how Yahoo integrate del.icio.us with their other services. I only hope they don't mess it up like so many promising sites before!

You'll need to use two files to set up a caching system for your site. The first, "begin_caching.php" in this case, will run before any other PHP on your site. The second, "end_caching.php" in this case, runs after normal scripts have run. The two scripts effectively wrap around your current site.

You can achieve this wrapping effect one of two ways. The first way is to simply use the include() function and add them manually to every script you run. Unfortunately, this method can take some time, but is arguably more portable than the alternative.

The alternative relies on adding the following two lines of code (modified to reflect the correct path to the two PHP files needed) to your htaccess file. This is my preferred method, just because it requires no modification to existing scripts, and can very easily and quickly be turned off (just by commenting out the relevant lines in the htaccess file).

php_value auto_prepend_file /full/path/to/begin_caching.php
php_value auto_append_file /full/path/to/end_caching.php

Next, we move on to the scripts that do the work. There are several stages to caching a document:

1. Receive request for page
2. Check for the existence of a cached version of that page
3. Check the cached copy is still valid
   • If it is, send the cached copy
   • If not, create a new cached copy and send it

To begin with, the script below contains a few basic settings. Here, you can set the directory you want to save cached files to (I would recommend keeping that directory outside your web root directory or at least protecting it from view through a normal browser). This script will need to be able to create files in this directory, and you need to allow this by setting the permissions of the directory. The permissions depend upon your server set up, so you may want to start by setting them to 777 while testing the script, and then reduce them to the lowest levels possible once the script is working.

You can also set the time, in seconds, a cached file should be considered valid for after creation, and set the file extension for saved files. It would be wise to not name them ".php", just for safety's sake.

<?php

    // Settings
    $cachedir = '../cache/'; // Directory to cache files in (keep outside web root)
    $cachetime = 600; // Seconds to cache files for
    $cacheext = 'cache'; // Extension to give cached files (usually cache, htm, txt)

    // Ignore List
    $ignore_list = array(
        'addedbytes.com/rss.php',
        'addedbytes.com/search/'
    );

    // Script
    $page = 'http://' . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI']; // Requested page
    $cachefile = $cachedir . md5($page) . '.' . $cacheext; // Cache file to either load or create

    $ignore_page = false;
    for ($i = 0; $i < count($ignore_list); $i++) {
        $ignore_page = (strpos($page, $ignore_list[$i]) !== false) ? true : $ignore_page;
    }

    $cachefile_created = ((@file_exists($cachefile)) and ($ignore_page === false)) ? @filemtime($cachefile) : 0;
    @clearstatcache();

    // Show file from cache if still valid
    if (time() - $cachetime < $cachefile_created) {

        //ob_start('ob_gzhandler');
        @readfile($cachefile);
        //ob_end_flush();
        exit();

    }

    // If we're still here, we need to generate a cache file

    ob_start();

?>

The file starts by generating an MD5 hash of the page that has been requested. It will use the complete requested URL, and the MD5 hash will be a 32 digit number, unique for each file. It then checks for the existence of this file.

If the file exists, it checks to see when it was last updated. If the file is older than the allowed time, it acts as though no cache existed (carrying on and generating a new file). If the file is still valid, it simply displays it.

There is also, in the settings, a list of pages to ignore when caching. This can be search results, comments pages, a news page or news feed - anything that should always be up to date. Simply add anything you do not want cached into here, and it will not be cached. You can add directories, or parts of URLs - the above simply searches for a text string. In the example above, I have left out the "http://www" portion of the URL, as this can be missed out by some visitors.

Finally, the two lines in italics above are both commented out. You can, if you like, uncomment these, and that will use outbut buffering to gzip your content before sending it to users, making your site even faster for them. Please note, though, that output buffering with gz encoding is not available in versions of PHP previous to 4.0.5.

Which brings us to the second file, "end_caching.php". At the end of the first file, if no cache exists, we start output buffering. This means that rather than send the page to the user, we are saving it for use later. In the second script below, we take the contents of the output buffer, and write it to a file.

<?php

    // Now the script has run, generate a new cache file
    $fp = @fopen($cachefile, 'w'); 

    // save the contents of output buffer to the file
    @fwrite($fp, ob_get_contents());
    @fclose($fp); 

    ob_end_flush(); 

?>

Important: If you do not have "register_globals" set to off in php.ini, make sure you add the following to the beginning of "end_caching.php" (straight after the "<?php" line) to aid security. This will ensure that an attacker cannot visit "end_caching.php" directly and overwrite an important file on your site (or read its contents).

    $cachedir = '../cache/'; // Directory to cache files in (keep outside web root)
    $cacheext = 'cache'; // Extension to give cached files (usually cache, htm, txt)
    $page = 'http://' . $_SERVER['HTTP_HOST'] . $_SERVER['REQUEST_URI']; // Requested page
    $cachefile = $cachedir . md5($page) . '.' . $cacheext; // Cache file to either load or create

And there we have it. If a cached document exists, it is shown to the user, and if not, one is created.

Finally, you need to make sure the cache remains reasonably clean. Over time, out of date or redundant files could build up, and these should be removed regularly. For this reason, I usually set up an automated script to delete all cache files once a week (or less often, depending on the traffic of the site), but this will depend greatly upon the server software you are using.

The script below is one example of a script to delete all cache files. You will need to set the cache directory at the beginning before running the script. You can either use this manually, visiting the page through your browser whenever you want to empty the cache, or run it automatically. An example of a CRON job used to run this script automatically is below the script (the " >/dev/null 2>&1" bit at the end of the crontab prevents the server emailing me every time the script runs). Please note that this last script will be cached too, unless you specify otherwise!

<?php

    // Settings
    $cachedir = '../cache/'; // Directory to cache files in (keep outside web root)

    if ($handle = @opendir($cachedir)) {
        while (false !== ($file = @readdir($handle))) {
            if ($file != '.' and $file != '..') {
                echo $file . ' deleted.<br>';
                @unlink($cachedir . '/' . $file);
            }
        }
        @closedir($handle);
    }

?>

curl http://www.your_domain.com/empty_caching.php >/dev/null 2>&1

META tags are a way to describe a page in HTML, invisibly to the user. Many search engines either do not use them at all, or give them little weight. However, they still have their uses and can provide a boost to your search engine placement.

The trick to using them well is to understand what they do, and providing the best possible information within them. It is important to realise as well that changing or adding META tags will not turn your website into a gold mine overnight, but as part of a well formed SEO strategy, they can certainly help.

There are many people who say you should only ever add two or three META tags to your site. There are those who say you should add hundreds. The simple fact is that there are many that could be appropriate to your site, and you should judge each of them on its individual merit.

META tags all go within the HEAD section of your site. That is to say, within the <head> and </head> tags.

Index

• The Title Tag
  Absolutely invaluable.
• The Description Tag
  Useful if used correctly
• The Keywords Tag
  Becoming less useful, but may be worth adding.
• The Robots Tag
  Can be useful in the right circumstances.
• The Content-Type, Content-Style-Type and Content-Language Tags
  Can be useful in the right circumstances.
• The Refresh Tag
  Can be useful in the right circumstances.
• The Pragma Tag
  Can be useful in the right circumstances.
• The Revisit-After and Expires Tags
  Usually useless, unless you actually want to put the spiders off for a certain period of time.
• Useless Tags
  Useless all the time, unsurprisingly.
• ICRA Label
  Useful

The Title Tag

The TITLE tag is NOT a META tag. But it does contain metadata, and it is the most important tag on a page and is closely related to them, so I am including it here.

Title tags are displayed in the top of a browser window, and are often used as a link from search engine results listings, so form them well. They should be descriptive and short (ideally under 70 characters), and they are also often used as bookmark titles, so it is important that you ensure your primary keyword phrase for a page is here, and that the title makes sense all by itself.

The Description Tag

This is one of the few META tags that can be considered important. The text within this is displayed by some search engines as the description to your site. A description tag should usually be kept to under around 150 - 200 characters and it is important to ensure that this tag reads well, and that it describes the page accurately.

There is no point in telling the user that your page contains thousands of pictures of Alicia Silverstone in lacy underwear if when they arrive on the page they see nothing but a sales pitch for tinned goulash. An extreme example, perhaps, but does demonstrate the point that it is better to have visitors who are interested in your product or content than those who aren't. Numbers are unimportant if they don't convert to sales, and this will help to qualify your visitors before they arrive.

The Keywords Tag

Fairly self explanatory, this tag is used to list keywords for your page. These are words you think are relevant to your page - words that if entered into a search engine should return your site. Search engines do not pay much attention to this, if any, as it has been abused for many years, but some do still use it to some small extent, so you may consider it worth adding.

Try to limit yourself to as few keywords as possible (the less keywords you list, the more weight each will likely have), certainly no more than 25, and list them with nothing more than spaces between (some people use commas, however this is no longer necessary). There is also no need to repeat the words listed.

As has been widely reported on the web, this tag is not used by many engines, if at all, and you would be wise to spend your time optimising and improving your site in other ways rather than waste time on this particular tag, in my humble opinion.

The Robots Tag

The ROBOTS META tag is one that is very often used when it should not be. The four variations listed above are four of the more common variations in use, and each accomplishes a different task. Never use this tag unless you wish to prevent a search engine spider from doing something. That's what it's there for.

The first of the examples listed above is completely worthless. If you have it on your site, go and delete it. That tag does nothing more than tell a search engine spider to behave exactly as it normally does. It does not benefit a site, does not get you crawled faster or more often, and will not suddenly make your site more popular than Google.

The second of these can be useful, for example on printer-friendly pages (where the content on the page is a duplicate of the original). This tag tells a search engine spider not to list the page it is viewing, but to follow the links away from the page anyway. The third of these is the reverse of the above, and tells a spider to list a page in it's results but not to follow the links on the page. Both of these have their uses, but these are very rare, so think carefully before adding these before you do.

The last tag tells a spider not to index a page or follow the links on it. It is extremely rare that you would want to use this (why would anyone want a page on the web that people cannot find?) but is included for the sake of completeness (some people use this for login pages or other similar pages they do not want listed).

There are more instructions you can add to this tag, the most notable of which is NOARCHIVE. This simply tells a search engine spider not to serve archived copies of the page to people viewing the search engine results (for example, Google offer a cached copy of sites in search results, and this will prevent Google from doing so). The tag to add to only prevent search engines making archived copies of your site publically available is:

The Content-Type, Content-Style-Type and Content-Language Tags

These are again quite common on some sites, and again have their uses. It is a wise idea if you are using an unusual language or style to mention it here, but by no means essential, as with most META tags. The W3C provide a more comprehensive resource for [url=http://www.w3.org/International/O-charset.html]character set information[/url], so if you do wish to use this, I recommend that as a good place to start reading.

Meta tags that use the "http-equiv" attribute rather than the "name" attribute, like these, allow you define within a document something that would usually be defined in HTTP headers (sent by your server). If you have no control over the headers sent with your web pages, but still need to define a content type or content style type (and so on), these are the tags you are looking for.

The Refresh Tag

Most useful on a chat page, or when a page has moved, this instructs a browser to refresh the page after a certain interval of seconds. If the second half of the content attribute is a URL, the refresh will take the user to the URL specified rather than simply refresh the current page. This can be, and sometimes is, used mischievously to prevent a user from clicking their back button to leave a page, something likely to annoy visitors enough that they may never return.

The Pragma Tag

Not very widely used, this tag asks a browser not to cache a page. Though this can be useful if a page on your site is frequently updated (for example a news site or a forum), it will often just increase your bandwidth bills and slow down your users' browsing experience. There is also no guarantee that a browser will pay attention to it.

Interestingly enough though, Microsoft recommend that if you do want to use this, you add the tag in a second HEAD at the end of the document, like so:

The Revisit-After and Expires Tags

There are a huge number of sites that say you should add the first of these to your site, because it tells search engine spiders how often to index your page. Which is a common misconception. The tag was created by SearchBC, who have said they no longer use it. Originally, it was created as a tool to suggest to the spider how often a page should be indexed. Few have ever been able to agree on the format of the tag. At the end of the day, remember that the search engines do not care how often you want them to index your pages - they will index as and when they feel like it. Some are clever enough to have a rough idea of how often you update your site, and will make use of that. Some are not that bright, and will come around when the mood takes them.

Assuming you are happy for the spiders to index your site as often as possible, as most people are, you would do well to leave this out. The spiders will return to your site as often as they deem fit, and the only way to influence the frequency this occurs at is to just keep adding new content on a regular basis.

The "Expires" tag tells browsers and search engine spiders when the document should be considered expired. This is worth using, of course, if there is a date on which the relevant document will be no longer valid. However, at this time, the search engines will often drop the page from their index - you should use the "Expires" tag only if this is what you want.

Useless Tags

A select few engines sometimes make small use of a select few of these, but most of these (and the others to be found on this [url=http://www.bauser.com/websnob/meta/useless.html]list of useless META tags[/url]) are better placed on a page, or not used at all. Most of these are added automatically by HTML editors, and some are added by over-zealous META tag addicts. In my opinion, these are best avoided, as they do little more than clutter up your code.

ICRA Label

Last but not least, something a little more unusual. The ICRA (Internet Content Rating Association) is an ideal I am happy to support, as they provide a means for helping webmasters to identify their content as suitable (or not) for certain age groups.

Simply put, you can visit their [url=http://www.icra.org/_en/label/extended/]label generator[/url] and tell the generator what your site contains. That data can then be used to help keep any content not appropriate for young eyes away from them. The data is used by some search engines and some browsers can be set to avoid pages without labels.