This is a quote from an insider bloggerabout human quality reviewers at Google…“The documents are used by Google QualityRaters to aid them in classifyingqueries, measuring relevancy, and rating thesearch results. To do so, the Quality Rater mustunderstand how Google works and this documenthas a bunch of that. Let me pull out some ofthose details in easy to read bullet points.
Three Query Types:Navigational: someone searching for asite, such as a search for IBM.Informational: someone searching forinformation on a topic of interest, such asfinding out more information on Danny Sullivan.Transactional: someone searching whenseeking to purchase something either online oroffline, such as searching for „buy ipod touch.‟
Quality Rating Scales:Vital: This is the highest score a web pagecan receive for a query. A vital result comesfrom a query that is most likely navigationaland the resulting page is the official web pageof the query. When searching for „ibm‟, thevital result would be www.ibm.com.
Useful: This is the second highest score a webpage can receive for a given query. A usefulrating should be assigned to results that“answer the query just right; they are neithertoo broad nor too specific.” One of theexamples given for a useful rating would be asearch on meningitis symptoms with a resultingweb page of:http://www.webmd.com/hw/infection/aa34586.asp
Relevant: This comes after a useful rating, andis used for results that return less useful results.The guidelines say the result is often “lesscomprehensive, come from a less authoritativesource, or cover only one important aspect ofthe query.”An example would be a review of laptopcomputers, but the review only takes fivecomputers and not all computers within its class.Since it is not a fully comprehensive review, itwould be rated as relevant and not useful.
Not Relevant: This rating is used for pagesthat are not helpful to the query but aresomewhat still connected to the original query.Classifications of a not relevant page would be“outdated, too narrowly regional, toospecific, too broad” and so on.One of the examples give is a search for the„BBC‟ that returns a specific article from BBS; itis too specific and is not relevant to the queryat hand.
Off-Topic: This is the lowest rating a pagecan receive for a query. If the returned pageis completely not relevant to the query, itwould be given a rating of “off topic.” Anexample given is a query on „hot dogs‟ thatreturns a page about doghouses.
Categories For Results That Can’t BeRated: Not everything can be rated, andthose must be classified somehow. Thecategories for those types of results include:Didn’t Load: For pages that return a 404error, page not found, product not found,server time out, 403 forbidden, login required,and so on.
Foreign Language: This is given to a page thatis in a “foreign language” to the “targetlanguage” of the query. English is never aforeign language, no matter what.So, if you search in Chinese for something and aHebrew page is returned, it is a foreignlanguage, but if an English page is returned, it isnot a foreign language. There are exceptions tothe rule.Unratable: When the rater cannot rate it forany other reason.
Spam Labels: Now for the really goodstuff, spam labels. This is a new addition to thequality raters guidelines and is fairly small. Thelabels include:Not Spam: The not spam rating is given topages that “has not been designed usingdeceitful web design techniques.”
Maybe Spam: This label is given when youfeel the page is “spammy,” but you are not100% convinced of that.Spam: Given to pages you feel are violatingGoogle‟s webmaster guidelines.
Flags: Flags are for pages that requireimmediate attention, such as:Adult contentMalicious code on pages
That is a brief overview of some of the manypoints in the document. For more, see thearchived document and for somehistory, check out Google Blogoscoped.Here is an additional copy of this documentat Huomah.com”source – http://searchengineland.com/the-google-quality-raters-handbook-13575