Wednesday, November 29, 2006

Not dead yet!

Things have been a bit screwy around here lately; therefore, I haven't had time to finish up the post I'm working on right now. Maybe in a few more days.

At least I finished A Clash of Kings.

Friday, November 24, 2006

Lots of stuff, all mixed together!

I recently commented on the idea of automated page expirations. Someone mentioned the oft-promised dewiki stable versions proposal in respect to this. Almost immediately after hearing about stable versions I realized how powerful and useful they would be for Wikipedia. They were promised for delivery "soon"; the timetable was presented as sometime in October, maybe even earlier, right after single-user login, which was supposed to be available "very soon" in August. Well, it's November, almost December, and single-user login isn't here yet, and neither are stable versions. My patience is starting to wear thin....

Another commenter suggested that Wikipedia has too few editors to have "content administrators". Wikipedia has thousands of editors. I find it hard to believe that there aren't enough people in those thousands who would be willing to do that sort of work if asked. Wikipedia does a poor job of managing its volunteers, and especially of advertising ways that volunteers can help the project. There is some sort of weird belief that it's wrong to ask people to do things for the project, or expect people to hold to their promises to do things, once made. Are Wikipedians afraid of commitment? (Is this a consequence of the relatively young age of most Wikipedians?)

The discussion of this brough to mind another idea, one Greg Maxwell had: have a bot go through the database, marking (at a relatively slow rate) pages which lack sourcing with a tag that declares that the article is not sourced and will be deleted if not sourced within one month. Any page which, after that month, is still unsourced is then deleted. The tagging rate has to selected to balance the desire to get all articles sourced in a reasonable time (there are something like 250,000 articles which lack sourcing on the English Wikipedia) while at the same time not putting too much demand on the editors who know how to properly source material. Pretty much everyone I've talked to about this thinks it's a good idea. I believe the only reason Wikipedia is not currently doing this is the lack of a working enwiki replica on the toolserver (another issue the developers seem not to care about very much).

The Esperanza dispute that I wrote about was deftly deconflagrated by Kim Bruning, who quite wisely recognized that no consensus could possible arise from that train wreck of an MfD and closed it early. I understand that he was criticized for doing so by a few sore losers. Kim's a tough cookie, though; he can take the heat. I can only hope that in the long run some good comes out of the secondary discussion.

I've had several people ask me recently if they should run for ArbCom. Why are people asking me this? Maybe they should listen to what I had to say in Episode Five of Wikipedia Weekly about the elections and people who think they might want to run for Arbitrator. Still, I'm not happy about the current slate of candidates: of the thirty-one candidates who have declared so far, I really think that only two or three are qualified and suitable. There's maybe a half dozen more who I don't know well enough to know whether they're suitable or not; the rest are, in my eyes, unsuitable in some way or another, some very much so. There's five seats open, which means that the chances of electing an unqualified candidate are quite good. And while frankly I think the ArbCom is losing relevance, it still remains an important entity; it would certainly not be a good thing for the ArbCom to become home to one, or worse yet, several, people who have no business sitting in judgment on their fellow editors. (By the way, the people who've asked me have generally gotten back a generalized "meh". Nobody who I really think belongs on ArbCom has asked me if they should run. Fortunately, nobody who I really think shouldn't be on ArbCom has either: I haven't had the displeasure of telling someone that they really shouldn't run.)

In a related topic, Werdna mentioned to me that he's working on an essay on how to improve Wikipedia. He's pretty much spot on on a lot of the issues. I'm looking forward to seeing what his solutions are, beyond exhorting readers to behave differently.

Onto another topic entirely: in last week's Wikipedia Weekly we briefly talked about universal wiki markup languages. Some listener mentioned Wiki Creole to Andrew, who shared the link with me. This project is working on developing a subset wiki markup that would be supported across multiple platforms and allow editors moving from wiki to wiki to use the same basic markup set no matter where they are. A MediaWiki implementation is promised. I noticed they've not made the MediaWiki mistake of using quotes for both bold and italics, but they still have the equals problem (both of which introduce unlimited lookahead, although in the equals case it's at least bounded to end of line instead of end of document, and the equals problem can be avoided if you have reasonable error recovery semantics, which MediaWiki doesn't).

Happy Thanksgiving, everyone (in the US, at least). Don't forget to check out next week's Wikipedia Weekly on Monday!

Tuesday, November 21, 2006

Of WikiCouncils

My gnomes have mentioned to me that there has been, since the October board retreat, some discussion about some sort of "WikiCouncil", consisting of respected members of the various communities, brought together to discuss issues and advise the Board. The exact details of the proposal vary depending on who makes it, and from moment to moment, but in general there seems to be some significant support for this amongst most of the vested parties at the Foundation level, with one notable exception: the Board's newest member, Erik.

I have long supported having a council of this sort. A council of Wikimedians, chosen by the communities to represent them, and wielding at least some influence over Foundation matters, would go a long way to restoring the sense that the community has real input into the operation of the Foundation—something which has, of late, been largely lacking. In the long term and in the ideal case, I actually think that an assembly of delegates elected from the various project communities (with the number of delegates from each community being decided by some fair and agreeable method and the method of election left largely up to each community) actually replacing the Board as the ultimate controlling authority would be the best model for long-term governance of the Foundation. (This assembly would meet once a year, immediately before, during, or immediately after Wikimania, and elect a Board, which would then govern between these annual meetings.) The Foundation is probably not ready at this point in its existence to move to such a governance model, however, and I do not today advocate it. However, efforts to establish a possible precursors to such an entity would be a good thing, in my eyes, not only for the potential it would offer for a long-term move toward true community governance of the Foundation, but also the shorter-term value of better community input into the strategic vision of the Foundation with today's governance (which quite frankly has almost no obligation to the community at all).

I freely admit that I have not read any of the discussion surrounding this concept on the Wikimedia public mailing lists; those lists are full of sound and fury, signifying almost entirely nothing. In almost every discussion on a public Wikimedia mailing list, the discussion is dominated by the noisy fringe, and in general the reasonable people bow out of the discussion because they do not wish to argue pointlessly with these fringe elements, who in many cases border on trolls. Nor have I read any of the private discussions—even when I was still involved in Wikimedia activities I only had access to the ComCom's mailing list, and certainly never to the vaunted internal-l or private-l, which is supposedly where all the real dirty business takes place. The Wikimedia Foundation has long been the home to some very ugly turf games and internal politicking, and I see no reason why that would have gotten any better with Erik's election; in fact, I would expect that it has gotten much worse. (I cannot begin to imagine how ugly the politics of the executive search and the discussion regarding Tim Shell's replacement must be.)

What I find interesting, though, is Erik's strident opposition (and I characterize it as "strident" based on what my gnomes are telling me; I have no direct experience with it myself, as I generally avoid talking to Erik, for reasons I may share at a later date, and as mentioned I have not read the mailing lists). I can only assume that this is because Erik feels that such a council would diminish his own importance. Sadly, this should not be a factor that a responsible member of the Board of Trustees of a nonprofit organization would use in decisionmaking for that organization, but I have seen decisions made for the Foundation in the past with equally questionable motives, as well, so I shall not waste too much time applying the tar and feathers to the Board's newest member. But I most definitely do call into question his motives for opposing what is quite clearly a very sensible means of establishing a viable channel for community feedback into the Foundation, something which both the Foundation and the communities need, in order to ensure that the Foundation is actually serving the communities' interests and that the communities feel that they have some real stake in the operation of the Foundation. It is my understanding that virtually everyone at the board retreat—save Erik—supported some sort of council. And yet, now that the retreat is over, Erik is building a grassroots campaign against what would normally appear to be the sort of thing the grassroots would want. (Is this grassroots, or astroturf? Hm.....)

Anyway, very curious situation. Makes me glad that I'm just an observer, not a stakeholder.

Monday, November 20, 2006

How to make MediaWiki better

MediaWiki, at least when used to support Wikipedia, could use at least two additional features.

The first is a workflow system. Right now, there are four outcomes that can occur when someone goes to edit an article. First, the user may be refused permission to edit at all. Second, the edit may be accepted and stored. Third, the edit may be rejected due to an edit conflict. Fourth, the edit may be rejected because it contains content deemed inappropriate (e.g. links to a "bad image" or to a spamblocked URL). I would like to have the option to add a fifth option: defer the edit until it has been reviewed by a content administrator. (Edits made by a content administrator would possibly bypass these checks.) This could be used to deal with editors who are on probation, or for edits which insert certain URLs which are frequently, but not always, spam (e.g. youtube, myspace). Right now, Wikipedia cannot add youtube.com to the spam link list because doing so makes editing [[YouTube]] impossible. This would allow people to edit articles containing questionable content, subject to a review process. (A generalized improvement of the methodology by which certain content is unacceptable would be useful too.)

The other addition would be the ability to add an expiration date to an article. This would be useful in the case that an article contains content which will be known to be out of date at some future date. Upon expiration, the article will generate a work queue item. Users with privileges on a work queue will be presented (upon request) with items on the work queues they have access to and asked to resolve them; in this case by updating the page. This could also be used to handle, e.g., AfD closures, RfA closures, proposed deletion, and lots of other things. I believe there's already a tasks extension, which is not implemented on the English Wikipedia for reasons that are not known to me, but I think this goes somewhat beyond what is offered by that extension.

I'm starting to think that I'd be better off reengineering MediaWiki rather than continuing the port, keeping these (and other) considerations in mind. MediaWiki's design is rather wedged, and I don't see any real value in continuing off of that design.

Friday, November 17, 2006

Wikimedia needs a TechCom

One of the things I've noticed in the past year and a half or so of watching Wikimedia operate is the way that MediaWiki development (which is at least partially paid for by the Foundation) seems to lurch about almost at random, with development being driven as much by what the devs feel like doing as what Wikimedia needs. Now, I realize that MediaWiki has customers other than Wikimedia. However, Wikimedia is the only customer that is paying them; they should get a much larger say in what gets developed.

On top of that, there is relatively poor communication between the communities (who have the desire for technical changes) and the developers (who are in a position to implement changes). Community-driven changes only take place when someone in the community manages to find a developer and convince that developer that the change is a good idea. This forces developers to be judges of consensus in communities that they are likely not even members of. There is no established mechanism for communities to come to a consensus regarding a change which requires either technical assistance (change of a configuration setting for the software) or software development, and, having come to that consensus, then request that the developers make that change. This isn't to say that such changes don't happen, it's just that there is no established mechanism. Whether or not a community can get a change executed comes down to whether or not they can successfully convince a developer that it's worth doing, which is a battle entirely independent to winning the consensus of the community for the change.

Another problem is the fact that many developers do double duty as system administrators. As a former developer turned system administrator, I will testify that this is one of the worst possible ways to run a development operation. There are endless reasons for this; I am not going to get into all of them, nor do I suggest that Wikimedia's operations team is guilty of all of them. However, I am a strong believer in a clear separation of responsibility between developers and administrators, especially on production hardware. (To be fair, Brion has done a decently good job of managing this, although there have certainly been some very egregious exceptions.)

On top of that, Brion is currently managed by Brad. Brad, while more technical than most CEOs I've run across, is neither sufficiently technical to direct Brion effectively, nor does he have the time to do it on top of all the other stuff he has to do. Nor is there any guarantee that the permanent ED will be even as technical as Brad. Brion is not sufficiently skilled (or, more pertiently, experienced) a CTO to effectively keep Brad appropriately informed on technical matters or, from what I've seen, to manage Wikimedia's technical assets and contracts in a fiscally prudent manner. As as result, the Foundation wastes money on poorly-considered purchases (and, especially, on its strategy of "throwing hardware at the problem" of its grossly inefficient software) and contracts, and doesn't get a whole lot of value out of funding MediaWiki development. It's clear to me that Wikimedia needs to shake up its technical side somewhat.

My recommendation is threefold. First, appoint a true CTO: someone who has the technical skills to manage both developers and operations, without actually having to be either a developer or a system administrator, and also the managerial skills to interface effectively with the nontechnical people in the ED's office, the CFO's office, and the Board. The role of the CTO (a title which Brion currently holds, but he does not really perform the duties of the office) is to direct operations, infrastructure investment, and development to ensure that the goals of the Foundation are being met by those activities, to keep the leadership of the Foundation informed on technical developments in a manner that is comprehensible to them, and to ensure that the directives that are set by the leadership are met in a timely manner by the technical staff and volunteers who report to the CTO.

Second, appoint a Technical Committee (or TechCom). The purpose of the TechCom, which would be a committee operating under the auspices of the Board, is to determine the technical needs of the Foundation and of the communities and convert those into directives to be given to the CTO for implementation. They would do so in consultation with the Board, with representatives of the communities, and with the CTO and other technical personnel. The CTO would probably be ex officio a member of the TechCom. The TechCom would be the entity to establish the mechanism by which a project requests a technical change; once the TechCom has evaluated the request and prioritized it, the CTO then decides how to make the request happen and assigns it to the appropriate teams for implementation.

Third, separate the technical staff into development and operations teams. The development team, led by a Senior Developer (a role for which Brion is probably most appropriate), would develop MediaWiki and other software required to meet the objectives and directives determined by the TechCom and the Board, and would report up to the CTO. The operations team, led by the Director of Technical Operations, would be responsible for maintaining the servers, hosting arrangements, and other such things as are required to maintain the day to day technical operation of one of the Internet's more complicated sites.

The Senior Developer will likely have to do a lot of volunteer coordination, since most of the MediaWiki developers are volunteers. However, it would likely make sense to allocate some budget to either contracting out development of code and/or hiring programmers, especially where such development could increase the efficiency of the systems used by Wikimedia. Current management strategy gives the developers no real incentive to improve efficiency because they have control over both operations and the hardware acquisition budget; therefore, they can simply solve performance problems by throwing more hardware at them. This has resulted in the Foundation being significantly overinvested in server hardware. On top of that, having Brion doing so many different tasks prevents him from doing any of them as well as he could. Divesting him of his operational and hardware acquisition responsibilities would free him to actually develop the code as well as give him time to recruit and manage volunteers for the project, which hopefully would lead to a better completion rate on outstanding projects. We've been promised SUL for what, over a year now, and stable versions on dewiki is now overdue as well. I cannot help but imagine that this is in part due to Brion being stretched too thin; but I also suspect it is due to inadequate supervision of Brion and the other technical resources as well.