CERE'03 logo The Third International Workshop on Comparative Evaluation in Requirements Engineering
Paris, France
August 29, 2005

Held in conjunction with the
13th IEEE International Conference on Requirements Engineering (RE'05).
Home
Overview
Call for papers
Workshop format
Instructions for authors
Important dates
Program & Schedule
Program Committee
Venue
Registration

Resources

Latest news



Contacts
Organizers
Webmaster

Workshop Schedule

Monday, August 29, 2005

8:30-9:00Registration at IAE
9:00-9:15Welcome and Introduction
9:15-10:30Paper Session I: Research Frameworks & Evaluation
A Requirements Engineering Reference Model by Ricardo Gacitua & Pete Sawyer
A Classification of RE Papers: Are We Researching or Designing RE Techniques? by Roel Wieringa, Neil Maiden, Nancy Mead and Colette Rolland
Discussant reply
Open discussion
10:30-11:00Coffee break
11:00-12:30 Papers Session II: Quantitative and Theoretical Research
Comparing Ordinal and Ratio Scale Data in Requirements Prioritisation by Lena Karlsson and Bjorn Regnell
Common Metamodel of Goal-Oriented Requirements Engineering Methodologies by Y. Tahara, S. Nagano, N. Yoshioka & S. Honiden
Discussant reply
Open discussion
12:30-14:00Lunch
14:00-15:15 Papers Session III: Qualitative Research
Evaluation Framework for Requirements Engineering Method Adoption: The BaRE Method Case by Uolevi Nikula and Jorma Sajanie
Ethnography in Requirements Engineering Research - Experiences and Confessions of an Auto-Ethnographer by Raymond Barnes
Discussant reply
Open discussion
15:15-15:30Organization of break-out groups
15:30-16:00Coffee break
16:00-17:00Break-out group meetings
17:00-17:30Group reports & wrap-up
Conclusion


Workshop Program

The following papers have been selected for presentation at the workshop:
  • A Requirements Engineering Reference Model
    Ricardo Gacitua and Pete Sawyer
    This paper describes a reference model to identify research opportunities in Requirements Engineering (RE). The reference model represents our attempt to rationalise a number of recent papers proposing research agendas in RE. It has been developed to provide guidance for the identification of research problems in RE but also to help researchers new to the field develop an understanding of the RE research landscape. The model articulates these by grouping related aspects of RE in a way that allows them to be compared, extended, and exploited, as well as providing a framework to represent the body of knowledge about RE.
  • Position Paper: A Classification of RE Papers: Are We Researching or Designing RE Techniques?
    Roel Wieringa, Neil Maiden, Nancy Mead and Colette Rolland
    Discussion of a paper in RE program committees is often complicated by lack of agreement about evaluation criteria to be applied to the paper. For some years now, successive program chairs have attempted to increase clarity by including a paper classification in their CFP, and making the evaluation criteria per paper class explicit. This short note presents a paper classification based on this experience.
  • Comparing Ordinal and Ratio Scale Data in Requirements Prioritisation
    Lena Karlsson and Bjorn Regnell
    The importance of prioritising requirements is widely recognised. A number of different techniques for prioritising requirements have been proposed, some based on an ordinal scale, others on a ratio scale. Different scales provide different levels of information, e.g. the ratio scale is richer than the ordinal scale. This paper aims to investigate the difference in information and aim to find ways to measure and compare the different levels of information. This is important since techniques using a richer scale tend to be more time-consuming and complex to use. Thus, there is a trade-off between simple techniques yielding less information and complex techniques yielding more information. The paper suggests three approaches to compare the level of information inherent in the two scales, and one way to use the cost/value diagram when the priorities are based on ranks. An empirical data set was used to verify the suggested approaches.
  • Common Metamodel of Goal-Oriented Requirements Engineering Methodologies
    Yasuyuki Tahara, Shinichi Nagano, Nobukazu Yoshioka and Shinichi Honiden
    Goal-orientation is one of the most important approaches to requirements engineering. Systematization of goal-oriented RE is needed to make effective the activities of engineers who want to adopt goal-orientation. However, existing efforts are not sufficient as bases for the systematization, because they lack the common terminology needed for the systematization. As a result, they cannot clarify the features intrinsic to goal-oriented RE in general and specific to individual methodologies. In this paper, we provide a common metamodel, specifying a common terminology, which has the potential to enable the systematization. We build metamodels of the KAOS and i*/Tropos methodologies on the basis of a common metamodel. The metamodel consists of individual elements used in the requirements models of the methodologies such as goals and agents, and relations between the elements such as responsibility and operationalization. The metamodel also specifies new concepts called requisites and composite tasks, as different elements from the goals and operations, respectively. We show the validity of our common metamodel by examining if KAOS and i*/Tropos metamodels can be correctly built on the basis of our metamodel.
  • Evaluation Framework for Requirements Engineering Method Adoption: The BaRE Method Case
    Uolevi Nikula and Jorma Sajanie
    A fundamental question related to method development is whether the method has the desired effect or not. It is evident that the characteristics of a new method will play a role in chancing the adopting organization, but the role of environment and organization is not always as clear. However, to be able to differentiate the affecting factors it is necessary to pay attention to these contextual issues in order to identify changes and to understand their causes. This paper describes an evaluation framework that is based on findings in studies on adoption of technological innovations. The framework lists and classifies a general set of contextual factors, supports the use of method specific factors, and guides evaluation efforts to detect causalities. The practical applicability of the framework is demonstrated by using it to evaluate a new domain specific method designed to ease the adoption of basic requirements engineering practices in small organizations.
  • Position Paper: Ethnography in Requirements Engineering Research - Experiences and Confessions of an Auto-Ethnographer
    Raymond Barnes
    Social science-based inquiry approaches such as ethnography are well recognized as valuable and relevant in information systems (IS) as well as requirements engineering (RE) research communities. However, the highly subjective and interpretive nature of ethnographic research may pose significant methodological problems and challenges, regarding how and when it can be effectively employed, and whether the results can be evaluated as relevant and even acceptable scholarly work.
    In this position paper I describe my experiences in developing a research design uniting two case study research approaches; namely, a traditional case study combined with an unconventional auto-ethnographic case study. I propose that despite the challenges, autoethnography offers unique opportunities for insight and discovery by both IS and RE researchers studying deeply entrenched and polemical issues, in a richly detailed yet scholarly manner.