I mentioned sourcing as the solution to the credentialing problem in my last post. And in thinking about some of what I wrote there, it occured me that Wikipedia's greatest problem has to do with "goal management" in the community. Wikipedia not only suffers from the lack of a meaningful goal statement itself (it defines itself as "a multilingual, web-based, free content encyclopedia project", but that doesn't really set measurable goals for the community), but also suffers because, in my expectations, the personal goals of many of its contributors do not align well with the overall project goal.
Really, why do people edit Wikipedia? And I don't mean "what draws people to Wikipedia". I want to know why people stay at Wikipedia. What addicts people to this site? And how does the personal pursuance of whatever personal wants these people are satisfying through their participation help make Wikipedia more of a "free content encyclopedia"?
I suspect that one of the major motivators is the desire to show off one's own knowledge. Quite a few people (including a number of the so-called "active contributors" who are currently claiming the right to VestedContributor status) are exercising this motive. And that's fine -- up to a point. There are at least two problems that come out of participants who are acting out of this motive. One of them is the tendency to claim ownership rights on one's own contributions. For these people, their contribution list, and especially their list of "featured articles" or otherwise specially-recognized content, becomes a substitute for penis size, and they thereby become unreasonably possessive of the articles they craft. They may also attempt to subvert the quality recognition processes in order to ensure that their own content is more likely to be favored with designations of quality, again in order to boost their own importance.
Another is the tendency to doggedly defend what they "know to be true" even when doing so is not defensible. Sometimes it's just due to lack of sourcing -- I could write a lot about computers, or electrical systems, or any of many other topics, based on my work experience, but I couldn't source most of it because I've learned what I know by osmosis, and don't have reference materials to back it up. Quite a lot of Wikipedia's content is of this origination. If Wikipedia had a dedicated corps of fact-checkers and fact-sourcers, this wouldn't be a problem; such articles would merely be treated as unverified drafts, presented as such, and eventually cleaned up to the next level. The problem is that that doesn't happen. The lack of such a corps is why the English Wikipedia has about a 9:1 crapticle:article ratio right now. As long as most of the content on Wikipedia is in the nature of "verbal diarrhea" (which so much of it is) this won't easily change.
Another major motivator is the urge to belong. Especially in the past year or so, Wikipedia has become one of the "places to be" on the Internet, and a large number of people have been attracted to Wikipedia for reasons that have rather little to do with Wikipedia being an encyclopedia. I have, in the past, referred to this as "playing the Wikipedia MMORPG". By and large, these people are teenaged boys, and virtually all of them are vandalism patrollers, some to the exclusion of all else. They have brought with them a number of undesirable dynamics: an expectation of hierarchy, a quest to achieve rank, a tendency toward dogmatic, unthinking application of rules, a propensity toward hazing those below them in their perception of the hierarchy, and a near-complete failure to understand the real purpose of Wikipedia in anything but the most superficial of ways. (That is to say, they usually claim to understand that Wikipedia is an encyclopedia, but have no idea what that means, and they have no grasp of board policy principles, only the specific statements made in individual policies.) I strongly suspect that PDD spectrum disorders are pervasive in this element. The sad thing is that these people are eminently useful to Wikipedia: they tend to be organized and follow clear process well, many are good at analyzing detailed data, and they generally do not mind doing repetitive tasks over and over again. We should put them to work sourcing articles; instead, we expect them to manage interpersonal conflict, which by and large they are really really bad at, and to deal with vandalism and trolls, which requires more diplomacy and discretion than the average member of this group can produce.
Another motivation, not entirely unrelated to the above, is to be important, powerful, influential, or even just noticed. I think this explains the Essjay situation to some degree. It can also explain a lot of trolling (since trolls typically do what they do in order to be noticed) as well as the addiction to drama that seems to pervade certain segments of the Wikipedia community.
Other motivations include the desire to overthrow the copyright system, or even to overthrow capitalism, a genuine interest in creating free content, and altruistic desires to create something of value for mankind. These people tend to be more likely to have an appreciation for the broader principles of the project. And they tend to run into trouble because they're the minority in the community.
The main failing, in my opinion, is Wikipedia's failure to erect a structure that can be used to channel the people in the second group into useful work. I am reminded of the death of the CommuniTree (see Shirky's article). The near-total lack of rules and structure on Wikipedia at the time these teenaged boys started flowing in meant that these children could not be channeled into useful work by an existing structure; had there been one most of them would have either flowed into it willingly or else rejected it and left. Because no such structure existed, they created one. And the one they created is designed not toward the purpose of creating a quality encyclopedia meeting certain fundamental core values, but instead toward the purpose of playing the Wikipedia MMORPG. Many of the current problems in the Wikipedia community is that there is a large, active subcommunity which views itself as having vested rights and which is pursuing goals that are at least somewhat at odds with the foundation goals of the project.
Fundamentally, Wikipedia needs to contemplate what the motives of its editors are, and find ways to discourage those whose motives are unproductive or which tend to work at cross purposes to the project's purposes. I don't think much effort has been put into understanding why people edit Wikipedia; in most cases we don't know at all, and where we have any clue it's based on voluntary responses to a survey, which often fails to reveal self-serving or otherwise "undesirable" motives that respondents don't want to admit to.
The conclusion I keep coming back to is that the wide-open unrestrained editing model that Wikipedia started out with simply will not lead to long-term reliable quality. Worse, the community structures that have evolved in recent months seem designed to entrench the current miasma of low-quality articles; I fear that without major change Wikipedia has reached the endpoint of its evolution.
It's a shame that Meatball is down at the moment; there's a lot of good stuff there that is pertinent to this discussion.
Sunday, March 11, 2007
Subscribe to:
Post Comments (Atom)
Why do you stay at Wikipedia?
ReplyDeleteI'm working on a similar explanation, while butting heads with RDH on the Wikipedians LJ. Starting points: 1. A Group Is Its Own Worst Enemy (Shirky); 2. that link I keep losing, finding and posting to wikien-l about how a hierarchy will evolve where you're not looking.
ReplyDeleteBasically, I'm going to crib shamelessly from your articles but writing in dense cryptic koans ;-p
(The fundamental misconception of the troll-critics (Wikitruth, Wikipedia Review) is that all the problems with Wikipedia are (a) fixable, therefore (b) not fixing them is evidence of evil. They don't seem to understand that "assume good faith" is a restatement of "don't assume malice when stupidity will suffice." And they appear to have far too little experience of just how stupid smart people can be.)
One thing we need more than anything else is freaking stable versions. Once we finally get that long-promised feature implemented we can make some serious progress towards improving the quality of articles. As it is right now, we spend too much time playing whack-a-mole with vandals and terrible edits.
ReplyDeleteStable versions doesn't need a single line of new code - just the will of the community to define and implement it. We have page protection and can send new editing to a sub page, or any number of other implementations. Or how about mav's old idea: send vetted articles to nupedia, with new revisions at wikipedia.
ReplyDeleteThe hard part is defining what a stable version is. We manage to process tens of thousands of article deletions per year. I see stable versions as a similar effort.
What's lacking here isn't code but leadership.
P.S. Another great post Kelly.
ReplyDeleteKelly said a lot in this post, but what resonated most with me is the need to better channel our workforce into the most useful work. Vandalism cleanup might be gratifying, but the work people are doing today is often of no better quality than the work that done by bots, in fact it's usually worse because the humans are inconsistent. I don't know that we can eliminate the Wikipedia MMORPG phenomena, but I believe we can channel it to be more effective for our real goals.
ReplyDeleteIt's been nearly a year since Jimmy announced the drive for quality. Our article count continue to soar, but I'm not seeing a lot of evidence for increased average quality.
Responding to Cyde: Stable versions (in whatever form they come in) will be a huge advancement. However, they are just one component and will not magically solve our problems. If nothing else we need to have new work available for the swarms of people who are squashing obvious vandalism who will hopefully be put out work by stable versions. Right now there is no active organized effort to source articles, for example. We don't need to wait to make a lot of improvements.
So I agree entirely that channeling people away from antivandal work and into other forms of work will benefit Wikipedia in several ways; both by using their labor more efficiently, and by altering their conception of Wikipedia. There are, however, serious structural obstacles to achieving this.
ReplyDeleteThe first is the advancement and status structures we have in place. This is really too obvious to spend much time on, but most of the MMORPG players you describe are seeking advancement in some form, and RFA is the outlet for that. And, in a self-reinforcing cycle driven by the fact that most voters there are soon-to-be-candidates, RfA demands high edit counts and "process experience" (i.e. hanging around AfD or DRV). The result is pretty obvious; and this is, incidentally, the reason why I tend to support the "require at least some demonstrated interest in writing an encyclopedia" movement; although RFA ideally wouldn't be the forum that shapes editor's activity patterns, it is, and I see little choice at the moment but to try to harness it.
The second structural obstacle is the community infrastructure that's developed around the vandalism game, and which is lacking in other maintenance fields (subject-specific content editing has it to a degree). First, vandal-patrolling is often social (IRC based); there's plenty of time to chat between reverts. There's a community there to provide mutual reinformcement; cf. the pile o' barnstars on an active patrollers userpage. Second, it's oversubscribed. This introduces a competitive aspect (beat the other guy/the bot the the revert/the RFA promotion...), but also provides a sense of accomplishment; all the obvious vandalism is getting reverted, and you're a part of it.
Contrast this to any of the massively backlogged article maintenance queues. There's no avenue to promotion, no recognition (no IRC channel for central communication, nobody sees you "beat them to the wikification", articles that are finished merely vanish). Active cleanuppers may not even know who their compatriots are, while active RC patrollers and AfDers see each other and talk to each other every day. The queues are undersubscribed, which contributes to isolation and a sense of futility. This is, obviously, another self-reinforcing problem.
In the past, I considered trying to create a community structure for wikification, cleanup, sourcing, etc. that resembled the vandal-patrolling setup; I no longer consider this viable. A better option, I think, is to emphasize membership in topic-specific editing communities and wikiprojects, and encourage those communities to take responsibility for problem articles in their area. The radical extreme here would be to split the encyclopedia into smaller subject encyclopedias, each with its own project infrastructure and (importantly) user and admin lists. This is obviously a bad idea for a number of reasons, but it illustrates the sort of change I'm talking about. Within topic communities, status tends to be based more on quality of work than number of edits and talkativeness, and if that ethic could be extended to localized cleanup tasks as well as content, there could be progress. The advancement structures thing is even thornier, and lord knows everyone has their own answers to that, but I think that there, too, the path may ultimately lie in a more general decentralization of the project, deemphasizing central fora like RFA and relying increasingly on local communities and structures.
I think I fit into a different category....One I'd like to see more of. I've stayed on for almost a year as I see a day without learning something new is a poor day. I can't think of anything more boring than writing about things I know (purchasing,midwifery, MRP,FMEA,PPAP, database programming and other dull subjects) - preferring to write about what I am ignorant becoming knowledgable in the process.
ReplyDeleteYes the site is replete with those whose goal is to vandalwhack and opine on deletion to get an RFA badge and the legions who know that the world needs to know the sexual proclivities of every person on TV... but there are others.
The difficult thing is to not lose the attraction to new, quality editors, of lack of entry hurdles while preventing the site from becoming myspace with citations. I don't think that Larry has the right idea with Citizendium as this looks ( from the exit comments of academics ) to be doomed as a poor fit for purpose.
Stable versions is a good start but some method to keep the rubbish out would be better. There is an increasing amount of quality (FA/GA) articles but, with the increasing popularity of the site, an faster increasing pile of unreferenced, stubby articles on nothing of importance.
It's almost like there are two wikipedia's coexisting on the same database.
I think I fit into yet another category....
ReplyDeleteI participate in Wikipedia for recreational joy. For me, Wikipedia is like a crossword puzzle, or, more accurately an online adventure game with many concurrent co-players. It's an intellectually challenging and fun game to play. That's why it's so addictive to me. As an extra bonus, i get to practice my English (i'm not a native speaker).
Some people express disappointment with Wikipedia, and even anger at it, for not standing up to what it presents itself to be. I delight in Wikipedia simply for what it is; i give less weight to what it should be and what it claims to be.
I do have my ideas about what Wikipedia ought to be. I try to observe closely the rules of Wikipedia (Notability, Attribution and a Neutral Point of View). For me, much of the appeal of games comes from observing rules. Exploring their boundaries, but not crossing them. There are infinitely many games, each with its own set of rules. There's nothing inherently worse or better in the rules of football compared to the rules of basketball. But when you play basketball, play basketball, not football. If everyone played the same game according to their own rules, there wouldn't be much fun in it, would it?
~~~~ ;)