Q2: Small Conferences

From Health of Conferences Committee

Question 2: NON-INCREMENTAL Has your community recently adopted new practices to promote non-incremental new ideas?

• big ideas sessions • more papers • shorter papers • deemphasizing detailed evaluation • others?

For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community.

SIGITE Given our relative youth, we have not yet have to face the problem of incremental development of ideas. If anything, our primary problem is the "non-incrmentality" of papers in the sense that many papers do not situate the work sufficiently in previous work.

SIGCSE SIGCSE takes the following approach for reviewing:

• reviewing for both the SIGCSE Symposium and ITiCSE Conferences are double blind • we maintain a database including about 779 reviewers for the Symposium and 584 reviewers for the ITiCSE conference. • each paper is sent to about 6 reviewers, and each reviewer receives 2-4 papers. • a minimum of 4 reviews is required for each paper, and the Program • Co-Chairs assign last-minute reviewers if the minimum number of reviews has not been met. • Most papers have 5 or 6 reviews. • SIGCSE members typically view their involvement in reviewing as a contribution to the community, and they are enthusiastic in this work. • After the reviewing period and after decisions are made, authors may view the reviews of their papers (although the reviews are anonymous), and each reviewer may view the other reviews of the papers they considered (again anonymously).

I believe that we always have looked for a balance of papers on a variety of subjects. This is not done formally, although each committee looks for balance and interesting new thoughts. Also, we have added several keynotes to our conferences to solicit new or different perspectives.

SIGAda We have not adopted any new practices to promote non-incremental new ideas. Our conference is of a small enough size that any such ideas can be championed by a single Program Committee Member and put on the program.

SIGDA big ideas sessions -Sort of - it's more like Special Sessions/ Hot Topics Sessions - but they're all invited and many times are in already established fields. I don't think it works too well.

more papers - The opposite - acceptance rates have gone way down! Doesn't necessarily reflect better quality, it's just that there's more inflation in the number of submissions (because there are more people in academic positions and industry people sometimes need to justify attendance to the conference by presenting a paper)

shorter papers - Not really. ICCAD (e.g.) does not allow short papers. All other conferences and symposia have short papers and/or posters, but the ratio didn't change that much over the years.

deemphasizing detailed evaluation - No - it's more of the opposite. More detailed evaluation is required since there's more papers on the same topic. I would personally like less detailed evaluation, esp. for emerging fields, but it's not happening.

others? - I think there should be guidelines about how much *new* material is required for a paper. There are guidelines for journals (e.g., 15-30% new material compared to the conference version), but not for conferences. Many times there's a minor change in the algorithm, new results, but everything is the same. This doesn't bring anything new, esp. when the author has 5-10 papers in the same conference! Many are going far too often for quantity vs. quality, and unfortunately this feeds the inflation problem we have. Incremental papers are many times liked by industry people because they provide a better solution to the problem they have *now*, but I'd prefer if priority would be given to papers/ideas that look at problems that are relevant 10-20 years from now.

SIGAPP SAC is already a conference for such types of papers so we haven't needed any adjustments in procedures to accommodate such papers. If an idea is not sufficiently well developed but appears to be a good idea, we can accommodate it in a poster session.

Our approach works well.

SIGUCCS big ideas sessions - Not sure quite how to answer this -- we're continually keeping up with changes in emphasis on technologies and services

more papers - Number of submissions is probably increasing. However, the location of the conference where the paper would be presented probably has some influence. For our Fall 2005 conference, the total number of accepted papers was 102, I believe, which is slightly more than we used to have. This led to 5 concurrent track sessions, compared to 4 in the past.

shorter papers - Paper length is expected to be between 4 and 8 pages.

deemphasizing detailed evaluation - Program chairs are expected to recruit competent reviewers from those who volunteer, although I doubt that we formally check their quality/credentials.

For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community. - It's working but, as with everything, could probably be improved. I'm not sure that we have a strict paper review process that is adhered to from year to year, e.g. at least two reviewers per paper, review of abstract, review of paper, re-review if necessary prior to final submission.

SIGACCESS We have not specifically instituted a 'big ideas' promotion, although we have discussed with the PC Chair that such papers would be welcomed. Consistent with other conferences, we have sought ways to include more of the interesting submissions, however. The last few conferences have included poster sessions for student and other papers that seem interesting, but not quite ready for a full technical paper. We have also tried to create other opportunities for students to participate and get mentoring (especially since many students are at universities where there is not a lot of other work in our area). To this end, we've had a Doctoral Consortium since the '02 conference. This year we are looking into the conference participating in ACM's Student Research Competition.

SIGMIS Panel sessions have been used as a "big ideas session." Panelists get the discussion underway. Shorter papers promote more ideas, but without necessary depth of discussion. More papers is a good way to serve potentially varying interests of attendees.

SIGMICRO This year, MICRO is encouraging "non-traditional" papers. We have just started this in our call for papers and haven't yet seen if we will receive many, nor if we will receive many of significant quality.

SIGDOC deemphasizing detailed evaluation – definitely not: we’re strengthening the evaluation process.

This year we ran into issues related to early international submissions (given tighter travel request processes abroad) and we’ve adopted a policy for discussion at the next board meeting.

SIGSAC big ideas sessions - Yes, we have used panels with invited speakers on big-idea topics. more papers - No, we have not increased substantially the number of accepted papers.

	workshops are considering this idea. I'd like to point out that our major conferences, e.g., ACM CCS, Mobicom do have a number of workshops associated with it. Many of the big-idea, work in progress papers are encouranged here.

deemphasizing detailed evaluation - No, as we think this is not a good idea. Others - Not yet.

Our workshops associated with CCS have worked well in promoting new ideas in areas that might not be in the main stream yet. In fact these workshops have worked so well that some will be merged to form new conferences. In particular, we plan to merge the ACM SASN and ACM Mobicom WiSe workshops (possibly togehter with a European workshop) and form a new ACM Conference on Ad-hoc, Sensor and Wireless Networks Security.

SIGecom We are still in the review process, so we will see how everything turns out. But our original intention is to accept more papers into the conference (quality permitting) by possibly reducing the length of oral presentations during the conference. We will have to wait and see how well this is working in the community.

SIGMETRICS Recently, we have been able to accept more papers by changing the talk length from 25 minutes talk plus 5 minutes of questions to 20 minutes talk plus 5 minutes of questions. While I prefer the original (longer) length, this does allow us to accept more papers, and therefore increase the number of conference attendees. I think the MAMA workshop, the JSSPP workshop, and the many tutorials held in conjunction with Sigmetrics every year also increase the number of attendees.

SIGBED SIGBED is a new SIG, and our main meetings are EMSOFT and IPSN. The identity and style of these meetings is still in development, and we are quite open to new ideas. We don't have problems on this front (yet!).

SIGSIM Panels, work in progress sessions, case-oriented papers.

These are all working fine. Need careful selection of reviewers and enthusiastic organizers.