In the last few days, with very little discussion among developers, an implementation of brainstorm was setup on the pidgin.im site. I would like to emphasize that taking this site down is not because we are indifferent to user feedback, nor actively ignoring it. Here, particularly, nothing could be further from the truth.

Our concern is two part. Firstly, the brainstorm site is built on top of drupal, which requires significant resources on the part of the server. We are concerned that under the load we have seen in the past, our servers will be unable to run drupal, and its existence, rather than furthering communication between our users and our developers, will in fact impede it, by bringing our website down entirely.

Secondly, our concern is that, like the forums we used to have on SourceForge, the brainstorm site will become a backwater of sorts, where users submit feedback yes, but also a site where that feedback goes largely unheard.

The brainstorm site, to be useful, will require a very active developer presence. Partly because proposing and voting on features does no good if developers do not see the votes, but for more mundane and fundamental reasons as well.

Already, in the couple days the site is been up, we see some problems. Duplicates here, as in the Trac tickets, are quite common, and quite frequently going unnoticed by the average user. Over time, duplicates like this will mean that the same idea will either have its votes split across so many different items that it will appear, falsely, relatively unpopular, or will be voted up so many times by the same user (once on each instance), that when it is noticed and merged, it will, equally falsely, appear disproportionately popular.

A separate concern of my own is that users seeing the brainstorm site, with its high emphasis on voting, will fail to take head of the note that it does not and will not significantly affect the direction development takes.

Open source developers work best, work at all in many cases, when they are given free reign to work on projects and subprojects that interest them,. If I, or anyone else, tried to tell our developers that they must drop everything and work on the 5 most popular ideas, most of them would quickly find that their free time could be more enjoyably spent on other projects. In an ideal world the ideas that most interest developers would closely mirror the desires of the larger community, but when users fail to step up, develop a trust relationship with others, and join in the development process, reality deviates from the ideal. In such cases, things of great importance to the larger community will languish.

The idea behind this voting system, as I see it, is to answer the question “What can I do to help?”. When an idea is highly popular, but not yet being worked on, we as a community, if there is to be a community, and not just a clamoring mob of users, need to realize that help is needed to achieve that goal. A highly voted for idea should thus be seen as an open invitation for a patch, or more realistically, a series of patches, from someone who has never before worked with us, or who, having worked with us some in the past, is looking around for what to tackle next.

For this reason, along with my concerns about resources, I feel it to be very important that a voting system be integrated in with the normal development process. While this might somewhat increase the burden of submitting a new idea, most if not all of the ideas submitted so far reflect tickets already submitted in Trac. However, once an idea is submitted, having the voting integrated in with the development process will ensure that it is, in fact, feedback; that it is seen by those working with the Pidgin, Finch and libpurple code base.

Mr. Mike Jazayeri announced that you can now sign into AIM from Gmail’s chat interface.¹ This integration is cool, but I would love to see it go a step further. I would like to be able to add AIM buddies to my Google talk buddy list directly. That will truly be a newsworthy day.

A few days ago several of us (Pidgin developers) were approached by Ms. Dahna McConnachie asking for an interview for the Australian version of PC World. Apparently I was one of the only developers to respond to her request, and as a result, I have now appeared in print.¹ I only spent an hour or so answering her questions, and I did not take the time to read it over as I should have. On the other hand, it appears she did not spend much if any time editing it either. For example, she clearly googled to see what “oscar” is, but came up with a totally wrong acronym. Still, I think it is cool in a way to be interviewed.

(The expansion of “oscar” was removed about 14:43 EDT)

In the days immediately preceding the 2.2.0 release, I called for some time to be spent looking at the growing backlog of bugs and enhancement requests that has accumulated in trac since we went public with pidgin and finch. Sean in particular responded beyond my hopes, closing more than 100 tickets in just a few days!!

In related news, Ethan, with the help of one of the trac developers, managed to nail the source of our continual problems with trac. By eliminating the drop down that listed the people to whom a ticket could be assigned, he radically reduced the performance hit we take by running trac on our server. The load average has dropped from 10+ to 3-5, and the CPU utilization is down in the 20-40% range instead of the 100-110% range (possible only because we are a vmware guest I suspect, how else could you use 110% of your processor?).

Overall, we are responding faster to more issues since we left SF’s trackers for trac, while still maintaining a relatively fast development cycle. In the months to come, hopefully we will nail an even greater chunk of the incoming tickets, and come up with a better way for translators to interact with the project.

Despite this though, we are continually being told that we are unresponsive to our users, and insulted for our decisions. Only this morning, for example, we were told that 90% of users think that every change we have made since the first 2.0.0 beta is a step backwards. When the user failed to come up with anything other than vague insults and obvious red-herrings in more than 10 minutes, we removed him from the channel. This is regrettable, not that we removed him, but that it was necessary. We try very hard to listen to everyone who approaches us with comments and criticism of our work. While we cannot take all the advice we receive, it is physically impossible, and in fact do not take much of it, we have reversed unpopular decisions and/or worked with various segments of the user base to come up with some compromise.

Still, I honestly think that (almost) no one really cares about the exact graphic used for the available state. For years I have seen people asking about replacing this graphic or that graphic, or even all of them. Such people have been genuinely concerned that the specific graphic used for this or that was poorly chosen. The complaints about the green circle do not (with a few rare exceptions) match that pattern. Rather, people complain about the graphic being “huge” even though it is the same size as the protocol icon it replaced. Alternatively, they complain about it being ugly, but want to replace it not with a different icon representing available, but with an option to use it as an emblem over the protocol icon instead. If the icon itself, and not the removal of the protocol icons, was the problem, then returning the protocol icons and returning status to an emblem would not be the solution. Similarly, the icon is clearly only “huge” in relation to the very small emblems that previously denoted state.

That being said, it will be interesting to see if the complaints change in light of the new option to see protocol icons merged in for 2.2.0. I suspect that they will not, though hopefully the (relatively) few users who tried to remain rational in the firestorm debates that have come up since 2.0.0, will be pleased.

Something rather amusing happened today. I realized that one of my less computer inclined friends was never aware of our name change. He discovered Pidgin from a list of clients on the LiveJournal website, downloaded it, and was very confused as to why installing it caused Gaim to disappear. Yay for people for whom this mess is of so little importance!! ☺

Last night someone saw fit to spam our ticketing system with a comment appended to one ticket, and a new ticket submission with text identical to the preceding comment. These posts are fairly typical of the requests we see for the return of protocol icons, demonstrating the childishness of the requesters, and the typical unwillingness to consider anything beyond their own use case.

That being said, the post itself was not entirely out of line. I dislike someone advertising for a fork in our trackers, and so fully support rekkanoryo in promptly closing the ticket. The trackers, to be at all useful, must have a steady turn-around time. For this reason, Sean agreed with me to attempt a pair of bug closing releases to follow the current cycle. Leaving requests open for an indefinite period of time because someone might want to work on that request someday simply does not work. We tried that in the Source Forge feature request tracker, and watched as that tracker ballooned to a size that defied any attempt at organization or management. This means that a fork, by its very nature something we are not going to work on, must be removed from the tracker.

Still, forks and forking play an very significant role in open source. Though they can lead to confusion and fragmentation of the user base and developer pool, they can also allow for groups of developers who have real and significantly differing ideas on where a project should go to both achieve their desired ends. This is good. The primary freedom of open source is not the freedom from cost, but the freedom to shape software to do what you want. This freedom is never exercised without cost, but is available at all only by accepting the very different costs associated with open source, costs not in money, but in time and effort.

It is because my co-developers and I pay that time and effort that Pidgin is available to anyone, and it is because we have paid for it that we get the deciding say in where it will go. We are part of a community, but in any community rights and responsibilities are split, and in any successful community, rights are balanced by responsibilities. We owe the community regular releases, our best effort at fixing bugs, and an up-front openness about our goals and our limitations. The community owes us respect for the time and effort we put into the project. If we saw that respect, if people accepted that we might just know a little bit more about our own user base than they do, they would stand a much better chance at convincing us of the need for a given change. I am not asking that users grovel, just that they stop calling us idiots, and stop assuming that their own use case is necessarily the most common one.

At the end of the day, we will not, cannot, please all of our users. The changes we have made, controversial though they are, are not made in a vacuum. Many of them have been requested for years. In a few cases, they are changes that we were only gradually brought to believe were the correct course after years of requests and days if not weeks or in some cases months of debate and experimentation. It is in recognition of this reality that I return to my original topic, that forks play an important role in open source.

This idea is not a new one to this project. Several times in its history different groups have felt such strong disagreement with a decision that we (or our predecessors) have made that they have forked Pidgin (then Gaim) to meet their own needs. Though to date none of these efforts have survived for long, the possibility always remains that we have made a significant mistake, and some future fork will prove a success. This is the nature of open source, and I wish anyone attempting to take on the work of a successful open source project the best of luck. That previous attempts have failed I attribute to the overall correctness of our own vision of the needs of the community so far. Despite high initial energy levels, so far forks such as gaim-claws or opengaim have failed to create any momentum of their own, failed to prove worth the effort involved. In short, they have proved that there was in fact no need for the changes they proposed.

Today, a very small, but incredibly vocal group of users are attempting to insist that protocol icons in the buddy list are crucial. They do so for a variety of reasons, and their logic, where it even exists, often breaks down. The use of protocol icons to distinguish buddies across accounts is a very flawed system, necessarily subject to a relatively high degree of error, and substantially limited. One person inadvertently disproved their own attempted justification by attempting to say that you could use them to distinguish between buddies on the same protocol! Others are concerned that our current file transfer support differs in success rates across the various protocols, or that things like Direct IM are available only on one protocol. This argument too breaks down, with the realization that we are actively attempting to make this transparent for the user, with improved file transfer support, and improved buddy selection support.

Others have failed to fully take advantage of Pidgin’s capabilities, and once they realize the power of the “contacts” pidgin supports, realize that most if not all of their desire for protocol icons is removed.

That being said, if this is in fact the first truly wrong call we have made, then it stands to reason that a fork would not only be appropriate, but would succeed. This is particularly true because with 2.0.0, and continuing with each release since, the core/ui split is complete, and libpurple is a reality.

Though many users have criticized us for spending time on this architectural change that has no immediate benefit to the user base, it was precisely our users who we had in mind as we pushed to complete it. Earlier I stated that one of the responsibilities of the developers of a given project is to openly admit their own limitations. We are limited. We are limited in the amount of time we can devote to the project, and we are limited in our ability to design an interface that is equally ideal for all subsets of our highly diverse user base. Though I believe this later limitation to be an unavoidable one, it is no less real. Given that Pidgin will never be the best interface for all users, it becomes critical that the vast amount of time and effort spent in figuring out and implementing support for AIM, Yahoo, MSN and other closed protocols not be wasted. Libpurple achieves that goal.

Users claim that they want the protocol icons. I call on them to put their money where their mouth is, by spending the time and effort necessary to become an open source programmer, and to create a libpurple based client that will blow Pidgin and Finch away, and raise the bar for a high quality instant messaging client ever higher. Many of us now involved in Pidgin and Finch started out with little or no programming experience, learning as we went from the existing code, from online tutorials, from text books we have purchased, and from our own mistakes. Nothing stops anyone else from doing the same thing, except, perhaps, the fact that just maybe there really is not any real demand or need for the changes so vocally being demanded.

As the release of pidgin & finch (and libPurple) 2.0.2 draws near, I cannot be but proud to work with such an amazing group of people. Though it appears that 65 tickets will slip, will be pushed from this release to the next, my co-developers have closed an amazing 89 tickets for this release, not counting the 25 tickets closed towards our next milestone, 2.1.0.

The growth of bugs in our new bug tracker has been nothing short of phenomenal. We have done our best to keep up with it, but all of us, myself particularly included, have a limited number of hours each week we can donate to the project. In light of that, the 150 ticket informal goal I set for this release was rather unrealistic. It is inevitable, with such a high count, that some tickets would slip. With over 100 open tickets already for the 2.1.0 release, this slippage will necessarily mean that the next release will also face severe challenges closing all the issues.

That is why I must ask for patience from our users regularly. We value your input, we want to solve your issues. There is simply so very much that needs to happen, and so little time in which to do it. We will continue to do the best we can, with the resources we have, and help (in the form of patches) is always appreciated.

We released beta7 last night. I think Sean scared people off by saying that this is the first of the betas to actually be of “beta” quality. Still, there have been about 15 thousand downloads of it so far, and very few bug reports. My colleagues deserve the credit for this, working very long and very hard to get as many of the bugs squashed as they could in the last week or so.

One interesting thing that I note afresh today, as I have in the past, is the “interesting” phenomena in which a user will think that multiple repeated requests will change the answer. One user responded to a series of questions in his ticket (#414) with simple repeated assertions that he dislikes the change to use only status icons. When I closed his ticket, he responded by opening five or ten new, entirely duplicate tickets. Another user, this time in #pidgin on irc.freenode.net, tried asking the same question in three or four different ways across an hour or two, hoping to find a different answer.

Still, overall it has been very quiet, which makes me happy. I hope, but do not expect, that the final release (hopefully later this week) will go as smoothly.

It has come to my attention that when I changed the SF Bug, RFE, Support, gtk-bugs, and rejected patches trackers to be visible to project members only, SF stopped sending emails out to people who had commented on those items, but are not project members.

I closed these trackers down shortly after our public release because it quickly became apparent that their existence was hindering the adoption of trac, and would continue to do so as long as they existed. In some cases, when requested to submit at trac instead of SF, some people would refuse, siting the fact that it means registering again, or that trac is in some way confusing. We (the project members) had decided that as good a home as SF has been to us, that it was time for us to grow beyond it. This combination of intransigence and our decision left me with very few options.

1. I could leave the SF trackers open, knowing that people would submit there, and knowing I intended to close every item there. This would result in some items only days or hours old being closed with a canned message, just like those months and years old. This would be confusing and frustrating for all involved.

2. I could close the trackers, and close the items in those trackers afterwards. I thought that this would continue to send an email to the submitter and any who had commented on that item. I have since been informed that this is not the case; only submitters and project members get emails. If it had worked as I thought, it would have been an ideal way to clean out the tickets and yet give users the opportunity to reopen any they were still concerned about.

3. I could attempt to import several thousand open items into trac. I rejected this because one of the flaws of the SF trackers is that the several thousand ticket size was too large for me to manage, and oppressively large for many other developers. Many tickets went unclosed, not because the issue was unfixed, but because no developer had read that ticket in a very long time. I decided that a clean sweep was the only way to get our ticketing back under control. Hopefully with the new development processes that we are implementing in our new space, we will be able to prevent the new trac ticketing from reaching that state of bloat at all.

I am sorry that the flaws in SF’s ticketing have caused this move to be a time of greater confusion than is strictly necessary. I will continue to try to provide the best support to our users that my available time allows, and I hope that you all will forgive, or at least understand, the decisions I have made.

UPDATE: I have learned that my information was bad; SF has in fact been sending emails to commenters as well as submitters and project members. This is a good thing.

Today in the logs I notice that Verizon is denying all mail from pidgin.im. While less than pleased with this, I am also reluctant to jump through hoops for every ISP out there. We run very standard mailing list software (mailman), and with the sole exception of having migrated users over from the gaim-*@lists.sf.net mailing lists, you are only getting mail from us if you asked for it in some form.

Coming now, instead of last week, I suspect that this is not due to the vast amount of mail caused by importing stuff from subversion into monotone. Were it coming then, I would sympathize with the ISP. Even sane admins would balk at several thousand emails for a given user suddenly appearing without warning. Our mailing list traffic this week has been rather normal though, so I can only conclude that Verizon is reacting in a somewhat draconian manner.

If you are a Verizon user, and are not getting mail from us, I suggest you contact Verizon and take this issue up with them.