Function in engineering: Benchmarking representations and models

Joshua D. Summers, Claudia Eckert, Ashok K. Goel

Research output: Contribution to journalArticlepeer-review

22 Scopus citations


This paper presents the requirements and needs for establishing a benchmarking protocol that considers representation characteristics, supported cognitive criteria, and enabled reasoning activities for the systematic comparison of function modeling representations. Problem types are defined as reverse engineering, familiar products, novel products, and single-component systems. As different modeling approaches share elements, a comparison of modeling approaches on multiple levels was also undertaken. It is recommended that researchers and developers of function modeling representations collaborate to define a canonically acceptable set of benchmark tests and evaluations so that clear benefits and weaknesses for the disparate collection of approaches can be compared. This paper is written as a call to action for the research community to begin establishing a benchmarking standard protocol for function modeling comparison purposes. This protocol should be refined with input from developers of the competing approaches in an academically open environment. At the same time, the benchmarking criteria identified should also serve as a guide for validating a modeling approach or analyzing its failure.

Original languageEnglish
Pages (from-to)401-412
Number of pages12
JournalArtificial Intelligence for Engineering Design, Analysis and Manufacturing: AIEDAM
Issue number4
StatePublished - 1 Nov 2017
Externally publishedYes


  • Benchmarking
  • Engineering
  • Function
  • Models
  • Representation Characteristics


Dive into the research topics of 'Function in engineering: Benchmarking representations and models'. Together they form a unique fingerprint.

Cite this