Human-Computer Interaction by Antti Oulasvirta

I've been always curious to see a ranking of HCI conferences based on expected citations per paper. Is CHI the King as most believe? How do the thematic conferences like CSCW and MobileHCI do? And what about regional editions of CHI? Unfortunately, the usual suspects in bibliometric ranking, such as ISI, do a poor job in covering our diverse conferences. There are many good uses for a ranking (and even more bad ones, I know). For instance, PhD students and others who are unfamiliar with the ecology of conferences in HCI, would benefit from knowing which conferences give most "bang for the buck." 

Although there are many bibliometric analyses of HCI, I was not able to unearth a ranking of conferences (please email me if you know one and I'll link it here!). However, thanks to Bernt Schiele's tip, I found out that Microsoft Academic Search provides citation and publication numbers for HCI conferences, and one can constrain the view to the last 10 years. So, I took that list and removed conferences that have very low citations, as well as those that had published less than 100 papers. I think pruning is justified, because these conferences are either newborns or dead men walking. I then, using my lousy Excel skills, calculated the index citations/paper, and ranked the conferences accordingly. Voilá! 

This simplistic measure has some credibility. First, Microsoft Research has done good job expanding the coverage of HCI venues and the numbers start to speak for themselves. Nevertheless, some important conferences, such as Pervasive, cannot be found from the list. Second, while we can guess the limits of this index - which must be analogous to the limits of using GDP per capita for ranking countries - it is (deceptively?) easy to understand, and it has value as a predictor of expected citations for an accepted paper. But common sense must be applied when drawing conclusions from it. For instance, I would not trust anything else than very clear differences. Conferences that have proportionately more posters and short papers are handicapped against those that favor full papers. (It'd make sense to redo the calculation with data from full papers only.) And, of course, newer conferences, like Persuasive, are handicapped when calculating a raw ten-year average.  

Some of the results were surprising to me and they got me to reconsider my publication goals. My first observations:

1. CHI is not the King. Actually, it's not even in the Top 5. The common misperception of CHI being the best is based on the fact that it has the largest total impact on the field thanks to its massive annual volume of publications. But it's not the best bet for any single submission. 
2. UIST is the King. Long live the new King! Actually, ECSCW and UbiComp are so close behind UIST that we should throne the three together. I'm not surprised to see UbiComp here. However, take a note that ECSCW has a way lower total volume than UbiComp.
3. Then there's a group of dozen or so conferences that compare to CHI. It was surprising to me that this group is so big; I thought the distribution of rich/poor conferences would be more polarized. 
4. Some positive surprises within that dozen. For instance, I'm glad to see that DIS and IDC are so high, I know they have worked hard for high quality and it's paying off. One thing makes me wonder, though: Why is Australian User Interface Conference AUIC so high up on the list - it's a regional conference, and it's the first time I hear about it?  
5. One negative surprise is INTERACT IFIP, which is very low on the list. What happened? I heard that in the 1990s, it was "the other CHI". But I'm not very surprised to see that MobileHCI is far behind the top lot, at 4.6 citations/paper and thereby on par with NordiCHI. I'm in the paper committee of MobileHCI'12, and I hope we can discuss ways to shape up a bit... 
6. Then there's a big group of conferences that publish a lot, but almost for nothing. I know that some of these confences deliberately serve other purposes than citations, and that's fine. But then there are some that are just utterly hopeless. In my view, they should either do something radical or reconsider their reason for existence.

Update: I'm happy to see that the post gained a lot of attention in Twitter. Researchers are passionate about conference rankings, and I fully understand why: Careers may depend on where you publish, and there are many strong preconceptions about the quality of conferences. I have received a number of useful suggestions on how to improve this ranking. Trying to do a better ranking would be useful for our community, because it would force us to reflect the nature and goals of our field. Unfortunately, I don't have time to do it myself. But if somebody is serious about doing it, I can forward the ideas that I received.

Table: Ranking of HCI conferences based on average
citations per paper during the last 10 years.