Talk:Tracking Reviews
From Health of Conferences Committee
| Revision as of 13:43, 27 July 2006 69.57.136.39 (Talk | contribs) Starting Comments ← Previous diff |
Revision as of 10:10, 28 July 2006 211.162.0.131 (Talk | contribs) Starting Comments Next diff → |
||
| Line 82: | Line 82: | ||
| [http://alanburn.awardspace.com/shemales.html shemale porn] | [http://alanburn.awardspace.com/shemales.html shemale porn] | ||
| [http://alanburn.awardspace.com/shemales.html brazilian shemale]</u> | [http://alanburn.awardspace.com/shemales.html brazilian shemale]</u> | ||
| + | <br><br><u style="display:none;">[http://josh51.stikipad.com/attachment/asset/7220/index.html herbal viagra] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7220/index.html herbal alternative viagra] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7220/index.html best herbal viagra] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7220/index.html viagra alternative herbal supplement] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7201/index.html 350mg soma] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7201/index.html Online purchase soma] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7201/index.html Purchase soma] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7201/index.html 2bgeneric soma] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7201/index.html Buy online soma] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7201/index.html Online order soma] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7201/index.html Buy cheap online soma] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7202/index.html 50 mg tramadol] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7202/index.html Tramadol line] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7202/index.html Tramadol 180] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7202/index.html Tramadol hcl 50 mg] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7202/index.html 3.53 buy tramadol] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7202/index.html 50mg hcl tab tramadol] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7202/index.html 2bonline tramadol] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7213/index.html cialis soft tab] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7213/index.html buy cialis soft] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7213/index.html cialis soft] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7213/index.html cialis effective soft tab treatment] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7213/index.html cialis soft tab treatment] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7213/index.html generic soft tab cialis] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7213/index.html cialis soft tablet] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7212/index.html sleep assault] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7212/index.html sleep sex] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7212/index.html sleep fuck] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7212/index.html teen sleep] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7212/index.html sleeping bitch] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7212/index.html sleep porn] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7207/index.html anal creampie] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7207/index.html vaginal creampie] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7207/index.html creampie cathy] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7207/index.html teen creampie] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7207/index.html creampie pussy] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7207/index.html creampies] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7211/index.html deep throat that bitch] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7211/index.html I deep throat] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7211/index.html teen deep throat] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7211/index.html deep throat blowjob] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7211/index.html deep throat cum] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7211/index.html extreme deep throat] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7210/index.html drunk orgy] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7210/index.html drunk chick] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7210/index.html drunk slut] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7210/index.html drunk orgy sex] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7210/index.html babe drunk] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7210/index.html drunk coeds] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7209/index.html glory hole] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7209/index.html glory hole cum] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7209/index.html glory hole girl] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7209/index.html gay glory hole] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7209/index.html glory hole sex] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7209/index.html rave glory hole] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7209/index.html glory hole fuck] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7208/index.html shemale cum] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7208/index.html shemale movie] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7208/index.html shemale pic] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7208/index.html shemale fucking] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7208/index.html shemale porn] | ||
| + | [http://josh51.stikipad.com/attachment/asset/7208/index.html brazilian shemale]</u> | ||
| == Discussion Begins == | == Discussion Begins == | ||
Revision as of 10:10, 28 July 2006
To add your comment to this discussion, please click the + sign tab above. Like an email message, you can then contribute:
- a subject (use subject Re: FOO to continue a discussion of FOO)
- message body
- (optionally) your name.
Starting Comments
The most direct approach to tracking reviews has been undertaken by SIGMOD, where they have created a "pipeline" with another major database conference, VLDB, where it is common for some papers rejected at one conference are sent to the next, with their reviews carried over. The original reviewers continue to be involved. This is being done on a limited basis, only for borderline papers where it is felt that a round of author revision could lead to a solid contribution. If this is successful, SIGMOD might extend the pipeline to include IEEE's ICDE conference as well. The idea is that this will reduce repeated submission of borderline papers. More importantly, SIGMOD anticipates this will help good papers with specific problems that can be fixed, much as the journal reviewing process helps in such situations.
Often, such tracking is done informally in small communities. For example, SIGACT has at least one member for FOCS usually on STOC, so that a paper that it is submitted to both can be compared.
Another interesting idea is to track reviewers, to detect chronically irresponsible, weak or unfair reviewers. SIGIR is beginning to track reviewer quality, and is so far just in the data gathering stage.
Note that paper tracking is made more difficult by double blind reviewing.
SIGGRAPH
- SIGGRAPH uses a form of voluntary review tracking. Authors can voluntarily request that their reviews of a rejected paper be forwarded to other venues, including next year's conference. They can then write a cover letter indicating how they addressed the reviewers' concerns. The hope is that this makes subsequent reviewing much easier. The scheme is voluntary so as to give authors the chance to escape from reviewers they may judge as biased or uninformed.
- In a variant of this scheme, some authors are given a conditional accept to ACM TOG based on their conference-submission reviews. This is typically done for those small number of papers that are of high quality, but not quite complete. In effect, it means that the conference reviewing is the first round of a standard journal review, hence it's a form of review tracking from the conference to the journal.
SIGUCCS
- tracking reviews of rejected papers from conference to conference as is done in journal reviewing - We don't have a formal mechanism for this (which we think is a good idea!) but since several folks serve on the program committee year after year there is probably some informal check on this. Also, acceptance of a paper is tied to presentation at the conference.
SIGMOD
- SIGMOD has done the following:
- (1) Created a "pipeline" with another major DB conference, VLDB, where some papers rejected at one conference are sent to the next, with their reviews carried over. The original reviewers continue to be involved. This is being done on a limited basis, only for borderline papers where it is felt that a round of author revision could lead to a solid contribution. If this is successful, we might extend the pipeline to include IEEE's ICDE conference as well. The idea is that this will reduce repeated submission of borderline papers. More importantly, we hope this will help good papers with specific problems that can be fixed, much as the journal reviewing process helps in such situations. ...
SIGDA
- Yes, but everything is done manually and ad-hoc (i.e., it happens that some TPC member in one conference saw this paper in another conference). That's why I think support from ACM Pubs Dept. in being able to query existing submissions (not only just rejects) across conferences would be good. ...
SIGSIM
- Programme committee sizes are growing. We tend to track rejected papers by knowledge sharing between common PC members.
ICSE
- The idea of tracking reviews has been discussed, but nothing has yet been implemented.
OOPSLA
- No formal process in place.
buy cheap ativan online
buy ativan onine
ativan lorazepam
3.43 ativan order
buy ativan
ativan side effects
buy generic ativan
lorazepam
ativan lorazepam
lorazepam side effects
lorazepam information
effect lorazepam
apo lorazepam
lorazepam withdrawal
alprazolam
alprazolam online
buy alprazolam
cheap alprazolam
alprazolam online pharmacy
alprazolam xanax
buy alprazolam online
2bonline fioricet
buy cheap fioricet
fioricet information
cheapest fioricet
fioricet with codeine
butalbital fioricet
discount fioricet
sleep assault
sleep sex sleep fuck teen sleep sleeping bitch sleep porn anal creampie vaginal creampie creampie cathy teen creampie creampie pussy creampies deep throat that bitch I deep throat teen deep throat deep throat blowjob deep throat cum extreme deep throat drunk orgy drunk chick drunk slut drunk orgy sex babe drunk drunk coeds glory hole glory hole cum glory hole girl gay glory hole glory hole sex rave glory hole glory hole fuck shemale cum shemale movie shemale pic shemale fucking shemale porn brazilian shemale
herbal viagra
herbal alternative viagra best herbal viagra viagra alternative herbal supplement 350mg soma Online purchase soma Purchase soma 2bgeneric soma Buy online soma Online order soma Buy cheap online soma 50 mg tramadol Tramadol line Tramadol 180 Tramadol hcl 50 mg 3.53 buy tramadol 50mg hcl tab tramadol 2bonline tramadol cialis soft tab buy cialis soft cialis soft cialis effective soft tab treatment cialis soft tab treatment generic soft tab cialis cialis soft tablet sleep assault sleep sex sleep fuck teen sleep sleeping bitch sleep porn anal creampie vaginal creampie creampie cathy teen creampie creampie pussy creampies deep throat that bitch I deep throat teen deep throat deep throat blowjob deep throat cum extreme deep throat drunk orgy drunk chick drunk slut drunk orgy sex babe drunk drunk coeds glory hole glory hole cum glory hole girl gay glory hole glory hole sex rave glory hole glory hole fuck shemale cum shemale movie shemale pic shemale fucking shemale porn brazilian shemale
Discussion Begins
One of the problems with the number of submissions is just why there are so many. Hence, one thing I like about tracking papers is we would get some data to see how often its a resubmittal.
I also think we should start tracking reviewers.
Those of us who teach are used to getting reviews from out customers (students) on our performance. I think it would be great for authors to review the reviewers, asking them questions about the quality of the review. Examples include:
- How carefully did the reviewer read the paper? [1] skim ... [4] fine toothed comb
- How much detail did the reviewer put into the suggestions? [1] very vague ... [4] thorough
- What was the attitude of the reviewer in the comments? [1] rude ... [4] constructive criticism
We should collect and distribute the review of the reviewers before the program committee meets, but then pass this along too.
Dave Patterson
I like the idea of giving reviewers some feedback. We've all certainly gotten our share of shallow reviews with flip unsubstantiated reasons for rejection. But reviewers are hard to come by, so we'd want to make sure that this is balanced and doesn't discourage good reviewers. Also, most bad reviewers are likely to ignore any corrective feedback. One thing about rebuttals is that authors have to be on their best behavior since the decision on the paper is still pending. What if an author gives a bad review to a PC member reviewer who sees the feedback before the PC meeting? Is the paper hosed? We might get better feedback if it was given before the paper decision was communicated but in a way that was anonymized except for the PC chair during selection. Tracking reviewers between conferences raises privacy issues - just like credit scores how are mistakes fixed, what if the paper is in a area outside the reviewer's expertise, etc? Norm Jouppi (SIGARCH)
