Vie's Inn of Wonders' Awards

Rewarding excellence in Content...

Criteria: the good, the bad and the ugly.

*

*

*

I have been a member of the award scene for quite some time now, certainly by current standards. During this time I have run my own successful award program, as well as having been an evaluator for, and even owner of, a variety of rating indexes.
And the thing that caught my attention the most is how many award program owners have great difficulties in writing good, understandable, logical and, very importantly, positive criteria.

Before we start with measuring the quality of criteria, we should first grasp the concept of criteria and the components thereof. This one is rather simple as most award owners can grasp this...
Criteria can be divided into 3 blocks:

• disqualifiers / pre-qualifiers
• criteria
• scoring

Let's take a closer look at defining those first...

Disqualifiers or pre-qualifiers (or even general criteria) are basically a list of items the award owner finds to be a no-pass… they end the evaluation on the spot.

Criteria or specific criteria are the elements the award owner likes to see in the applicant's website, and which he will base his evaluation upon. These are the elements that determine the score and hence what award will be granted, and even if an award will be granted at all.

Scoring is retaking those specific criteria (this is really important) but by adding grades next to it. Positive grades and deductions are placed here. You could place these alongside the criteria also, but it could be argued that, by making this distinction, you can be more elaborate in the criteria and more concise in the scoring. This is a perfectly valid argument, but not necessarily the best way to go for each award program.

Now that we have explained the divisions in criteria, we can immediately define the first errors some award owners make in the build-up of their criteria sections:

• each criterion belongs clearly in either disqualifiers or criteria... if it ends the evaluation it's a disqualifier, if you give it points it's a criterion. Doubles do not exist, and mixtures of criteria and disqualifiers do not exist either.

• Criteria and scoring have to be identical... perhaps less elaborated but nonetheless identical.. if items show only on one but not the other, you create confusion.

• You do not need more divisions than those 3 mentioned above... there are no more divisions possible, so it's pointless to have more and it only confuses applicants.

• For theme criteria there is not much difference in this... theme criteria only divide the criteria by giving parallels to something else, a theme. It doesn't touch the fundamentals of this division.

Now let's discuss the build up of criteria or format.

I think it should be obvious to everyone that a criterion that will end the evaluation and one that you score upon can not be presented in the same way. Unfortunately it does happen in quite a lot of award programs...

Let's begin with the easiest one: the criteria.
Criteria are the scoring elements. So basically it doesn't really matter if an applicant has the criterion or not, does it? Sure it will cost them points, but that's about it...
So if we can all accept the above, then we should also be able to agree that words like "must", "should", or "have to" do not belong here. These words create absolutes, and that criteria are not!
The only logical and acceptable means to properly display criteria in the way they're supposed to be perceived by the applicant is in a questioning structure.

Examples:

Do not say:
- Your site must have an easy navigation. (10 pts)
But rather say something like:
- Does your site have an easy navigation where each page can be reached in less than 3 mouse-clicks? (10pts)

Now about the disqualifiers.
This is something I have fought for in all my rating indexes... the need for positiveness in disqualifiers. After all these are the elements we do not like to see and will end the evaluation, and unfortunately these are the elements where we are often the most blunt and impolite. And because of this we might scare good applicants away.
Is this hard to do? Well I won't lie to you... yes it is. It's the hardest thing ever. But I've created some tips I feel might help you in overcoming this heavy task. Are these the only ones? Absolutely not… But if you don't have an idea of where to start these might help...

1. Write full sentences. Short sentences have an easier way of being interpreted as rude and impolite than a full sentence.

2. Avoid the use of the exclamation mark… this is absolutely unnecessary and it's very impolite.

3. Do not overuse the same verb. If you constantly use must/must not, it also tends to become rude... there is an abundance of verbs available that can be used to say the exact same thing: ought to, have to, need to,...

4. Instead of constantly focussing on the bad elements, try to view it the other way around: how do you like to see it?

5. And if all the above are exhausted: instead of punishing, show it's outside your control: you are willing to evaluate them, but you can't.

Examples:

Do not say:
- no horizontal scrolls!
But rather say something like:
- A considerate site owner will make sure his site is free on any scrolls in an 1024x768 screen resolution, so that all visitors may look at the site the same way.

Or

Do not say:
- Only English sites allowed!
But rather say something like:
- I would love to visit all sites on the web, but since I can only understand English, I am limited in evaluating only those.

I truly believe that if you as award owner try to keep the above divisions and writing formats in mind, you will have good logical criteria that are also positive in nature. And this can only be beneficial to you and to the quality of your award program.

* *

Intro  Disqualifiers  Criteria  Graphics  Self Test  Application  Winners*
Status  Statistics  Evaluator  Affiliations  Ethics  Privacy  Copyright  Site Map