3-4 months ago, we had set the goal to have 100% of every article reviewed by 2 human editors. Today, that objective has been reached.
It wasn’t as easy as we thought because an entire sub-system platform had to be built to manage what we now call the “QC Queue” and summary & detailed QC reporting. We had a simple QC system for about a year now, but it was only used to spot check performance. The first 1,000 articles have now floated through the 2nd generation article QC process and March 2007 will be the first month with 100% of all articles that are accepted will be human reviewed by (2) separate editors.
This means our in-house editorial team can now be evaluated on a quantity and quality performance metrics rather than quantity alone. It also means less errors in future approved articles, leading to a greater level of trust with the various stakeholders in our site.
To give you a little insight into our internal QC system, every error has been divided up into MAJOR and MINOR errors. Errors that can be fixed in a few seconds by the full-time QC editor are fixed and not reported as errors.
Major errors might include: Leaving SmartQuotes in the article summary, title left in the article body, blatant wrong category, end of body spaces left, blatantly bad spelling or grammar issues, excessive BR tags left in, author name mismatch between by-line and resource box name, failed to add co-author if mentioned in the resource box, broken links, first paragraph link left, excessive links beyond our editorial guidelines, etc. If an article has to be rejected after it was accepted, it’s a major error on the editors part.
Minor errors might include: Title casing wrong, lack of spaces after a period, or periods left in HREF statements causing a link to be broken.
An error can be classified as either minor or major depending on severity.
At our current pace, we’re allocating about 3,000 hours of annual labor to be the 2nd set of eyes on every article and that rate will climb as the quantity of articles climb.
We’ll be dramatically enhancing the new QC platform over the next 5 weeks as we learn how to become more efficient with it.
Sometime in April, I expect we’ll have some kind of small quality incentive for our editors so that the current 83% non-error rate goes into the 95-99%+ territory.
Another large advantage this new system nets us (besides a major boost in confidence/trust from our users) is that we’re not reliant on editors who have to put in years with us to keep track of the 7-8 pages of single spaced guidelines that every article is compared against before approved. We can also insert any editor who has earned our trust to take a portion of the QC Queue so there is no single point of failure (such as the QC editor having a day or week off).
Ahhhh. That is a sigh of relief. :) We can now hire in packs of 2-4 editors at a time, bring them through our training system, and then not worry about mistakes left un-caught because 2-sets of eyes are on every article.
In other words, this new proprietary QC platform will help us SCALE while not losing quality out the door as volume increases.
Is it a perfect system? nope. Will we continue to make mistakes? Yep. Will this new system widen the gap between our competitors and us? I hope so.