Q2: IEEE Conferences

From Health of Conferences Committee

Question 2: NON-INCREMENTAL

Has your community recently adopted new practices to promote non-incremental new ideas?

• big ideas sessions
• more papers
• shorter papers
• deemphasizing detailed evaluation
• others?

For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community.

ACM/IEEE

Panels of discussion, that only publishes summaries, are a useful practice.

- K.Z.

Not really, that's what the workshops are good for. Short papers will only undermine the quality of submissions and make the review process more arbitrary.

- I.M.

Rejecting papers with good ideas based on one negative review hurt the community. Also rejecting papers from Asia because they are not well written (English problem) is also hurting the community. I have seen so many good idea papers that are not well written. The benefit of presenting the idea to the community is lost by rejecting the paper on presentation ground or by rejecting the paper based on one negative. This is something I would lie to see change.

-S.H.

We are not doing any of these things. We introduce new topics to the Conference through a panel to increase interest, and then, possibly, a special session. We do have a lecture and application series, where the reviewing is less formal, to provide specific knowledge to our attendees. However, we've never had an issue refusing "big ideas" - many best papers in the past have been in this category.

- S.D.

We use the practice of Special Tracks. Each year we have a Call for Special Tracks - so that experts representing new emerging areas could organize special sessions on some particular topics besides the usual regular sessions. Besides having a separate programme and review committee, Special Tracks have no difference from the regular sessions (the presentation time is the same, and the publication length in the proceedings is the same). I believe Special Tracks significantly help to promote new emerging sub-areas and ideas at the conference.

- A.T.

Some workshops/conferences have a 6 page limit; but shorter than that most people will not want to come to the conference to give a short paper.

- C.B.

In addition to technical papers, the conference include about 40-50 tutorials, 4 invited speakers for Plenary sessions, one Keynote, and 12 invited "Masterworks" presentations. The masterworks sessions and invited speakers are intended to provided targeted "big issues" and also bring to the conference highly regarded people who are leaders of the different communities. This works very well.

- B.K.

Have a special session dedicated to new ideas that are untested and futuristic.

- V.C.

We are consistently trying to adopt this practice to make the conference(s) much more meaningful to attendees. Sessions of only submitted papers are not enough any more - this way we can ensure the input from experienced people who might not otherwise have submitted papers. We can add very controversial topics that no-one else would have considered adopting. Already the conferences seem to be too big. Let's maintain standards, but also go back to considering poster sessions.

- A.A.

The security community has had the NSPW (new security paradigms workshop, this used to be an ACM conference and is now sponsored by ACSA) for more than a decade, and there has been a proliferation of papers, often also scattered across several conferences, which blunted the impact. CRA recently hosted a `Grand Challenges' workshop in the field, but that turned out to be more of a consensus group. We (IEEE TFIA) try to encourage this type of work by organizing smaller max. 60 participants) single-track workshops which are to provide a richer and more focused forum for interaction and debate among participants, but on balance I'm not convinced that the present approach is truly enjoying encouraging success in supporting or generating truly disruptive ideas.

- S.W.

We adopted accepting short papers (4 pages or less) in 2005. It seems to be working well. We still accept full papers (limited to 12 pages single-spaced). About 20% of the submissions have been short papers, and, while we have not set any quotas for acceptance, about 20% of the accepted papers for our 2005 conference were short papers.

- V.M.

We are successfully using practice of having approximately equal(about 30%) groups of papers/tutorials from scientists from East (EU,USA, …), from scientist from West (Ukraine, Russia, Armenia, Belorus,…), and from students (PhD’s are in this group too).

- V.H.

Co-locating; no fixed partners. Yet to see the long-term effectiveness.

- C.K.K.

Short papers and posters are becoming more popular. Those are no more considered "devaluated" publications. I’m aware of just one big idea session, held along with VTS and called Elevator Talks. Too early to say if this is really a good idea. Seems to be, but I’m not sure that the number of submissions is high enough for this sort of session.

- M.L.

In very few conferences there are "new ideas" sessions. I don't think they have worked as expected though.

- D.S.

The big idea sessions have just been introduced in CFPs this year. We will have to wait and see if the published papers actually contain big ideas or simply poorly evaluated small ideas. Some conferences have accepted increased number of papers. However, this is a modest increase (e.g., 32 instead of 28). While this is a good idea, this will have only small benefit.

- R.G.

OOPSLA

OOPSLA has a "big idea" session. It's called Onward! This has a subcommittee (I was on it once) to review the submissions. The accepted papers get a one hour presentation slot. Many software engineering (SE) venues have promoted workshops that have newer ideas - things less mature or not completely evaluated. Also most SE venues have short papers.

- J.M.

SC 05

In SC 05 we went to 5 parallel tracks for the technical papers. This was done not to increase the technical papers but to expand other aspects of the conference, including an additional award sessions, special sessions associated with the themes of the conference, etc. The chart shows that actually the number of technical papers declined slightly and then stayed about the same. Several years ago, we tried the concept of "extended abstracts" for selection and the having the full papers done for only the accepted papers. This was done mostly to help authors - but it was found that it causes several problems. First, there was not enough detail to fully evaluate some papers. Second, we felt getting the final papers in on time is more challenging. In SC05 we had full papers submitted and it seemed to work well. This appears to work well.

- B.K.

IPDPS, SC, HiPC, ICPP, SPAA

I have not seen this practice yet in the parallel and distributed computing community.

- D.B.