Call for Papers

Overview

Following the success of the inaugural International Workshop on Comparative Evaluation in Requirements Engineering (CERE) which was held at RE'03, we are pleased to announce the second International Workshop, CERE'04, to build on the progress made so far.

The need for an assessment of the progress made in RE research is now commonly felt across the RE community. A number of requirements and specification exemplars have been used informally over the years, for instance, the meeting scheduler, the London ambulance computer aided dispatch system, and the light control system. These exemplars have been useful for illustrating new RE tools, techniques and methods, and for identifying potential lines of research. Unfortunately, the commonly used exemplars in RE all lack standardized descriptions and well-defined and universally accepted evaluation criteria, so different approaches cannot be compared with each other directly. This important issue serves as a starting point for current research into techniques for comparative evaluation. There are now many signs that research in RE is becoming mature enough that the RE community should make detailed comparative evaluations of alternative techniques.

At CERE'04, there will be presentations of results from current research and discussion of next steps. To this end, we are seeking position papers and technical papers along the themes of the workshop.

Themes

• Research method and research validation in RE:
  • How do we choose our research goals?
  • How do we evaluate success?
  • How do we measure the impact/importance of a research program?
  • Should we be more explicit about our research methods?
• The role of comparative evaluation in RE:
  • Establishing the necessary consensus on how to compare research results
  • Strengths and weaknesses of various comparative evaluation approaches
  • Experience of these evaluation approaches in other fields
• Determining which sub-areas in RE are ready for comparative evaluation.
  • Identifying task samples and evaluation criteria
  • Proposing potential benchmarks for specific RE activities.
• Reporting on the results of empirical studies and comparative evaluation of RE techniques, methods and tools.

Format

CERE will be a discussion-oriented workshop to promote interaction and exchange of ideas among participants. In the first half of the workshop, there will be presentations and discussion of submitted papers. During the afternoon, we will synthesize the ideas and examine the way forward for comparative evaluation. Papers will be reviewed by an international program committee and together with reports on the afternoon sessions will be published in printed proceedings.

Submissions

We will be accepting two types of submissions, short position papers and full technical papers. Both types of papers should tackle one of the topics or questions from the Themes section above. Submissions will be accepted through the CERE'04 web site (http://www.di.unipi.it/CERE04).

• Position papers (3-5 pages). Short papers, stating the position of the author(s) on any of the topics within the scope of the workshop. For example, positions papers could describe experience with a particular research evaluation method, or could propose an area of RE that is ripe for benchmarking. Position papers will be evaluated based on their potential for generating discussion, and on the originality of the positions expressed.
• Technical papers (8-10 pages) Full papers either describing experience of comparative evaluation, or report on the results of such evaluation. For example, a full paper might describe how a comparative evaluation of RE techniques was performed in practice, either by research labs or in industrial settings; or it may present the results of the actual performance of RE tools, methods or processes, in lab-based experiments or in field trials.

Important dates

Organisers