TY - GEN
T1 - Bringing Systems Engineering Models to Large Language Models
T2 - 12th International Conference on Model-Based Software and Systems Engineering, MODELSWARD 2024
AU - Alarcia, Ramón María García
AU - Russo, Pietro
AU - Renga, Alfredo
AU - Golkar, Alessandro
N1 - Publisher Copyright:
© 2024 by SCITEPRESS – Science and Technology Publications, Lda.
PY - 2024
Y1 - 2024
N2 - Although showing remarkable zero-shot and few-shot capabilities across a wide variety of tasks, Large Language Models (LLMs) are still not mature enough for off-the-shelf use in engineering design tasks. Organizations implementing model-based systems engineering practices into their product development processes can leverage on ontologies, models, and procedures to enhance LLMs applied to engineering design tasks. We present a methodology to integrate an Object-Process Methodology model of a space system into an LLMbased spacecraft design assistant and show a performance improvement, as compared to a conventional LLM. The benchmark is evaluated through subjective expert-assessed and an objective cosine-similarity-based criteria. The results motivate additional efforts in integrating Model-Based Systems Engineering practice into LLMs as means to improve their performance and reduce shortcomings such as hallucinations and black-box, untraceable behavior.
AB - Although showing remarkable zero-shot and few-shot capabilities across a wide variety of tasks, Large Language Models (LLMs) are still not mature enough for off-the-shelf use in engineering design tasks. Organizations implementing model-based systems engineering practices into their product development processes can leverage on ontologies, models, and procedures to enhance LLMs applied to engineering design tasks. We present a methodology to integrate an Object-Process Methodology model of a space system into an LLMbased spacecraft design assistant and show a performance improvement, as compared to a conventional LLM. The benchmark is evaluated through subjective expert-assessed and an objective cosine-similarity-based criteria. The results motivate additional efforts in integrating Model-Based Systems Engineering practice into LLMs as means to improve their performance and reduce shortcomings such as hallucinations and black-box, untraceable behavior.
KW - Design Assistant
KW - Engineering Design
KW - Large Language Models
KW - Model-Based Systems Engineering
KW - Object-Process Methodology
KW - Systems Engineering
UR - http://www.scopus.com/inward/record.url?scp=85190371259&partnerID=8YFLogxK
U2 - 10.5220/0012621900003645
DO - 10.5220/0012621900003645
M3 - Conference contribution
AN - SCOPUS:85190371259
SN - 9789897586828
T3 - International Conference on Model-Driven Engineering and Software Development
SP - 334
EP - 345
BT - Proceedings of the 12th International Conference on Model-Based Software and Systems Engineering
A2 - Domínguez Mayo, Francisco José
A2 - Pires, Luís Ferreira
A2 - Seidewitz, Edwin
PB - Science and Technology Publications, Lda
Y2 - 21 February 2024 through 23 February 2024
ER -