Q3: Small Conferences

From Health of Conferences Committee

Question 3: PROGRAM COMMITTEES

Does your community practice:

• double blind submissions
• program committee submission restrictions
• rebuttals (author responses)
• large program committees
• program subcommittees
• others?

Do these practices seem to help or hurt promoting your field?

SIGITE

Given the size of our conference, we have essentially used a single person program committee, although we have used a fairly sizeble community of reviewers. As our conference grows, we will need to address this issue.

SIGCSE

double blind submissions – yes

program committee submission restrictions - ???

rebuttals (author responses) - no

large program committees - we encourage reasonably large program committees to include many members in meaningful ways. We do not use this approach to replace or duplicate the regular reviewers.

program subcommittees - program committees have members focused on various aspects of the event (e.g, papers, panels, workshops, local arrangements, ...). We do not use subcommittees to subdivide the reviewing process.

Having many reviewers and utilizing many people on a program committee has been a great help in giving the community a better sense of identify and connection. It also seems to have had a significant effect in encouraging increased conference attendance.

SIGAda

We do not use any of these practices. Our reviews are blind in that the reviewer does not know who wrote the paper. But the Program Chair does know that information when assigning papers to reviewers.

SIGDA

double blind submissions -Yes - all events (except some workshops).

program committee submission restrictions - Yes in DAC (max 4) and ISLPED (max 3). Unfortunately, it is not widely practiced - there have been cases where 10% of the papers in the conference came from the same author. This is (in my opinion) unacceptable, as most of these were incremental ideas.

rebuttals (author responses) - Yes, only in DAC. I would like to see all SIGDA conferences/symposia do it.

large program committees – Yes

program subcommittees - Yes, in all (I think) conferences/symposia we have subcomms.

Others - ICCAD and DAC have rotation rules of TPC members (3 yrs. in, 3yrs. out), except for subcomm chairs who can stay an addl. year. ISLPED doesn't have it, but it is done at the discretion of the TPC chairs. I think such rules are good.

There were times in the past where people had spent 10 years or more on the same TPC. This should be unacceptable.

SIGAPP

double blind submissions - We use a single blind submission.

program committee submission restrictions - Our conference is organized as a set of tracks, each track having a chair. The chair should submit at most one paper to his/her own track and even that is discouraged. There are no restrictions on track PC members.

rebuttals (author responses) - No.

large program committees - Yes.

program subcommittees - Yes, that is essentially the track model.

Others - No.

These practices help.

SIGUCCS

double blind submissions - Had to google this - and we don't!

program committee submission restrictions - Where committee members submit, someone else within the program track will review their submission.

rebuttals (author responses) Reviewers and, if necessary, Program Chair deals with these.

large program committees Yes

program subcommittees Yes, according to track/topic

Do these practices seem to help or hurt promoting your field? -They help!

SIGACCESS

Our PC has not used double blind submissions. I don't recall this ever being raised as an issue. Nor have we had the restriction about PC submissions -- we certainly expect that many of the PC members would submit to the conference as they are leaders in the field. We also have not tried author rebuttals.

The PC does not meet in person, but does all reviewing through an online system. For '05, a three-step process was used that seemed to work well. That is, each paper got three reviews. When this cycle was completed, all papers were opened up to the entire committee for online voting on which to accept and which to reject. Reviewers could read manuscripts they hadn't previously read and also had available the reviews of the other committee members. This seemed to get consensus. Finally, after acceptances were decided, there was voting for the Best Paper awards.

SIGMIS

Double blind submissions definitely help the field to remove perceived bias toward large research institutions. This approach also negates the necessity to restrict program committee submissions. Author responses are a good mechanism to clarify misunderstandings on the part of reviewers and are therefore perceived as helpful to the field.

SIGMICRO

double blind submissions - yes

program committee submission restrictions – No

rebuttals (author responses)- Yes

large program committees - Relatively large, yes

program subcommittees - No

All of the above practices help. Some do take exception to double blind review. We tend to let the authors elect to have their paper be double blind or single-blind.

SIGSAC

double blind submissions - Yes.

program committee submission restrictions - Yes.

rebuttals (author responses) - No.

large program committees - Yes.

program subcommittees - No.

others - Use of external reviewers.

Generally we believe these practices have helped our field.

SIGecom

We don't have program committee submission restrictions. We encourage PC members to submit their own work. The PC co-chairs were not allowed to submit our own work.

We believe having a larger program committee helps in promoting the field.

SIGBED

We don't use blind submissions, and papers by PC members are allowed. There are no rebuttals. PC usually consists of 30-35 members and this seems adequate. Having a large PC consisting of many people who are natural and strong Contributors increases the fraction of PC papers in th eprogram, I think this is not a good thing, and we may switch to a model of small PC with PC submissions forbidden.

SIGMETRICS

We practice double blind submissions and large program committees, and for workshops, we have workshop co-located at conferences.

The conference practices a double blind review process which turns out to be a bit fake in my opinion: most of the times reviewers can guess the identity of the authors.

Sigmetrics never had a restriction on the number of submissions for PC members. Indeed, looking at this year's program there are some papers co-authored by the same PC member. Several PC members had more than 1 submission. We do not practice rebuttals (never had), not program subcommittees. In a conference like Sigmetrics I do not think that subcommittees would work.

SIGSIM

Pretty much all of these. For the most part these work fine.

SIGWEB

HT and JCDL use program committee submission restrictions HT uses program subcommittees

I've seen no evidence that this helps or hurts promotion.

The submission restrictions are informal, and usually only apply to the committee chairs and track chairs.

The subcommittees were originally designed to boost submissions from outside the core field at HT, but this has not yet really been in place long enough to say with certainty that this is either working or not. In the two years we've tried this, submission have been slightly up, but it's not clear this is due to subcommittees.