Application-oriented evaluation of the SDM reference model: Framework, instantiation and initial findings

Reference models have become fundamental tools in Information Systems research owing to the associated claim of reusing existing knowledge and getting faster and better solutions by adapting reference models instead of modeling and developing them individually. While the advantages are widely stated they are currently not reflected in documented acceptance and usage of reference models in practice. This calls for empirical substantiation of the claims of reference models. This paper summarizes a framework for the application-oriented evaluation of reference models and instantiates the framework for a reference model for Service Data Management (SDM). We argue that the claim of a reference model should be evaluated in three distinct stages of a reference model supply chain: reference modeling, solution design, and business transformation. Based on these stages, we propose a detailed evaluation plan using the Goal-Question-Metric approach and present first evaluation results, which have already led to improvements of the reference model. The paper is intended as stimulus for discussing viable approaches to empirically support research on reference modeling.