Q2: Large Conferences
From Health of Conferences Committee
(Difference between revisions)
| Revision as of 21:33, 22 February 2006 MarkDHill (Talk | contribs) ← Previous diff |
Current revision MarkDHill (Talk | contribs) |
||
| Line 48: | Line 48: | ||
| :For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community. | :For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community. | ||
| - | This is not done formally, although each committee looks for balance and interesting new thoughts. Also, we have added several keynotes to our conferences to solicit new or different perspectives. | + | :This is not done formally, although each committee looks for balance and interesting new thoughts. Also, we have added several keynotes to our conferences to solicit new or different perspectives. |
| - | '''ICSE''' | + | '''DAC''' |
| - | :Past instances of the Int'l Symposium on the Foundations of Software Engineering (FSE) had an informal evening 'fun flames' session over beer and snacks, giving interested people a low-stress way of getting feedback on very early and immature ideas. This was quite an informal gathering that was intended more for *airing* new ideas rather than *promoting* them. This year the Int'l Symposium on Software Testing and Analysis (ISSTA) are soliciting submission for a more formal new ideas track, but it is too early to tell how successful this will be. | + | :We do have short papers, and we do try to keep the submission rate above 20%. However, we still require rigorous review of all papers. |
| - | :large program committees | + | :Do these practices seem to help or hurt promoting your field? |
| - | :Yes (as many as 45 on past PCs of ICSE). | + | :We practice all of the above, and feel that they are all necessary. They seem to help. |
| - | :program subcommittees | ||
| - | :Not presently (although this was used many years ago in ICSE). | ||
| - | :Do these practices seem to help or hurt promoting your field? | + | '''Super Computing''' |
| - | :Past experiences with a subcommittee structure for the ICSE PC were reportedly disastrous. In addition, there is a feeling among some that scientific and scholarly standards in software engineering are not sufficiently uniform to justify author rebuttals, which because of the lack of uniformity would end up being broad advocacy statements that all authors would submit rather than narrow rebuttals of misunderstood points. | + | :In addition to technical papers, the conference include about 40-50 tutorials, 4 invited speakers for Plenary sessions, one Keynote, and 12 invited “Masterworks” presentations. The masterworks sessions and invited speakers are intended to provided targeted “big issues” and also bring to the conference highly regarded people who are leaders of the different communities. This works very well. |
| - | :From ICSE 2005 PC co-Chair: A note on managing large submissions and subcommittees. At ICSE 2005 (I was co-PC chair of the Research Track), the General Chair decided to split out the Education Track and Experience Track, which in previous years had been reviewed by the same PC as the research track. Not only had the single PC model failed to scale, but the PC members had troubles keeping the unique acceptance criteria of each track in focus (e.g., lots of Edu and Experience submissions rejected for lack of novelty). And perhaps research-PC types weren't best suited for judging those categories anyway. | + | |
| - | :The result was that although the research PC was at least 10% smaller than the previous year, we didn't need a 2-stage review process and the number of reviews per PC member averaged 22.5. That was lower than the 28 reviews from 2004, which had a two-stage process. The two-stage process is problematic because of the higher level of management and interaction, and the need to compress the first reviewing phase to make time for the second phase. | + | |
| - | :I'll note that this fragmentation is like establishing subcommittees, but it is not like dividing a discipline into subjects. Each track is really different. A problem with topical subcommittees in SE is that many papers are multi-topic: "An AI approach to IDE automation of refactoring to support software evolution - an empirical evaluation". | + | |
| - | :I think the number of submissions has leveled off for the time-being (for us), so further efforts are not really necessary at this time. | + | |
| + | :more papers | ||
| + | :In SC 05 we went to 5 parallel tracks for the technical papers. This was done not to increase the technical papers but to expand other aspects of the conference, including an additional award sessions, special sessions associated with the themes of the conference, etc. The chart shows that actually the number of technical papers declined slightly and then stayed about the same. | ||
| - | '''DAC''' | + | :shorter papers |
| - | :We do have short papers, and we do try to keep the submission rate above 20%. However, we still require rigorous review of all papers. | + | :No – the papers are all 30 minutes. |
| - | :Do these practices seem to help or hurt promoting your field? | + | :deemphasizing detailed evaluation |
| - | :We practice all of the above, and feel that they are all necessary. They seem to help. | + | :Several years ago, we tried the concept of “extended abstracts” for selection and the having the full papers done for only the accepted papers. This was done mostly to help authors – but it was found that it causes several problems. First, there was not enough detail to fully evaluate some papers. Second, we felt getting the final papers in on time is more challenging. In SC05 we had full papers submitted and it seemed to work well. This appears to work well. |
| + | |||
| + | :others? | ||
| + | :For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community. | ||
Current revision
[edit]
Question 2: NON-INCREMENTAL
Has your community recently adopted new practices to promote non-incremental new ideas?
- big ideas sessions
- more papers
- shorter papers
- deemphasizing detailed evaluation
- others?
For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community.
OOPSLA
- If I understand it correctly, then the answer is yes. OOPSLA has been very active in introducing new venues for a long time (Educators Symposium, Doctoral Symposium, Practitioners Reports, Demos, of course in additon to the traditional other venues such as workshops, and tutorials. I apologize if I didn't understand what is meant by non-incremental.
- big ideas sessions
- Yes, it has been a tradition for OOPSLA to host specific tracks devoted to specific issue with related tutorials, workshops and panels, and some times with its own invited speaker (e.g., THe OnWard! track)
- more papers
- Introduced recently different categories to the papers track (e.g., Essays, selected papers from the OnWard! track that pass the same technical papers review criteria)
- shorter papers
- Yes, we introduced Practioner Reports which doesn't have the same acceptance criteria as the technical papers but they are not citable either. They server the practioners community rather than academia though we see more and more interst from Academia in what the practioners are doing.
- deemphasizing detailed evaluation
- Only for non technical papers
- others?
- For OOPSLA Practitioner Reports and Onward! proved very valuable to our community.
SIGGRAPH
- SIGGRAPH already has a special sessions program where broader ideas/interest areas are presented each year. It is a very successful program that draws anywhere from several hundred to a couple of thousand attendees. There is also the Sketches program that may not focus on "big ideas" but on new and innovative ideas that are on the horizon of interest amongst attendees.
- more papers
- While SIGGRAPH is concerned about the growing number of submissions, there is also concern for bringing in appropriate content with the changes in research areas. There is increasing pursuit of these new and less represented areas so that the scope of content provided at the conference is comprehensive.
- shorter papers
- I am unclear about the papers committee's concern about papers length. However, the Sketches program seems to be filling a need for shorter papers presentation.
- deemphasizing detailed evaluation
- Again, I think that this depends on the program chair and the committee process and it changes from year to year.
SIGCSE
- others
- I believe that we always have looked for a balance of papers on a variety of subjects.
- For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community.
- This is not done formally, although each committee looks for balance and interesting new thoughts. Also, we have added several keynotes to our conferences to solicit new or different perspectives.
DAC
- We do have short papers, and we do try to keep the submission rate above 20%. However, we still require rigorous review of all papers.
- Do these practices seem to help or hurt promoting your field?
- We practice all of the above, and feel that they are all necessary. They seem to help.
Super Computing
- In addition to technical papers, the conference include about 40-50 tutorials, 4 invited speakers for Plenary sessions, one Keynote, and 12 invited “Masterworks” presentations. The masterworks sessions and invited speakers are intended to provided targeted “big issues” and also bring to the conference highly regarded people who are leaders of the different communities. This works very well.
- more papers
- In SC 05 we went to 5 parallel tracks for the technical papers. This was done not to increase the technical papers but to expand other aspects of the conference, including an additional award sessions, special sessions associated with the themes of the conference, etc. The chart shows that actually the number of technical papers declined slightly and then stayed about the same.
- shorter papers
- No – the papers are all 30 minutes.
- deemphasizing detailed evaluation
- Several years ago, we tried the concept of “extended abstracts” for selection and the having the full papers done for only the accepted papers. This was done mostly to help authors – but it was found that it causes several problems. First, there was not enough detail to fully evaluate some papers. Second, we felt getting the final papers in on time is more challenging. In SC05 we had full papers submitted and it seemed to work well. This appears to work well.
- others?
- For each practice you are using, what is your view of how well it is working within your community? Please comment on the merit of the other strategies as applies to your community.
