The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering

DANIEL Braun; OLEKSANDRA Klymenko; T. I.M. Schopf; YUSUF Kaan Akan; FLORIAN Matthes

doi:10.1145/3460824.3460826

The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering

DANIEL Braun, OLEKSANDRA Klymenko, T. I.M. Schopf, YUSUF Kaan Akan, FLORIAN Matthes

Informatics 19 - Chair of Software Engineering for Business Information Systems

Technical University of Munich

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

Since the introduction of Word2Vec in 2013, so-called word embeddings, dense vector representation of words that are supposed to capture their semantic meaning, have become a universally applied technique in a wide range of Natural Language Processing (NLP) tasks and domains. The vector representations they provide are learned on huge corpora of unlabeled text data. Due to the large amount of data and computing power that is necessary to train such embedding models, very often, pre-trained models are applied which have been trained on domain unspecific data like newspaper articles or Wikipedia entries. In this paper, we present a domain-specific embedding model that is trained exclusively on texts from the domain of engineering. We will show that such a domain-specific embeddings model performs better in different NLP tasks and can therefore help to improve NLP-based AI in the domain of Engineering.

Original language	English
Title of host publication	2021 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021
Publisher	Association for Computing Machinery
Pages	8-12
Number of pages	5
ISBN (Electronic)	9781450388887
DOIs	https://doi.org/10.1145/3460824.3460826
State	Published - 2 Apr 2021
Event	3rd International Conference on Management Science and Industrial Engineering, MSIE 2021 - Virtual, Online, Japan Duration: 2 Apr 2021 → 4 Apr 2021

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	3rd International Conference on Management Science and Industrial Engineering, MSIE 2021
Country/Territory	Japan
City	Virtual, Online
Period	2/04/21 → 4/04/21

Keywords

Engineering
Word Embeddings

Access to Document

10.1145/3460824.3460826

Cite this

Braun, DANIEL., Klymenko, OLEKSANDRA., Schopf, T. I. M., Kaan Akan, YUSUF., & Matthes, FLORIAN. (2021). The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering. In 2021 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021 (pp. 8-12). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3460824.3460826

@inproceedings{a342e6a2ff2d4ac8858876c84059947c,

title = "The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering",

abstract = "Since the introduction of Word2Vec in 2013, so-called word embeddings, dense vector representation of words that are supposed to capture their semantic meaning, have become a universally applied technique in a wide range of Natural Language Processing (NLP) tasks and domains. The vector representations they provide are learned on huge corpora of unlabeled text data. Due to the large amount of data and computing power that is necessary to train such embedding models, very often, pre-trained models are applied which have been trained on domain unspecific data like newspaper articles or Wikipedia entries. In this paper, we present a domain-specific embedding model that is trained exclusively on texts from the domain of engineering. We will show that such a domain-specific embeddings model performs better in different NLP tasks and can therefore help to improve NLP-based AI in the domain of Engineering.",

keywords = "Engineering, Word Embeddings",

author = "DANIEL Braun and OLEKSANDRA Klymenko and Schopf, {T. I.M.} and {Kaan Akan}, YUSUF and FLORIAN Matthes",

note = "Publisher Copyright: {\textcopyright} 2021 ACM.; 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021 ; Conference date: 02-04-2021 Through 04-04-2021",

year = "2021",

month = apr,

day = "2",

doi = "10.1145/3460824.3460826",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "8--12",

booktitle = "2021 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021",

}

Braun, DANIEL, Klymenko, OLEKSANDRA , Schopf, TIM, Kaan Akan, YUSUF & Matthes, FLORIAN 2021, The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering. in 2021 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 8-12, 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021, Virtual, Online, Japan, 2/04/21. https://doi.org/10.1145/3460824.3460826

The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering. / Braun, DANIEL; Klymenko, OLEKSANDRA ; Schopf, T. I.M. et al.
2021 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021. Association for Computing Machinery, 2021. p. 8-12 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - The Language of Engineering

T2 - 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021

AU - Braun, DANIEL

AU - Klymenko, OLEKSANDRA

AU - Schopf, T. I.M.

AU - Kaan Akan, YUSUF

AU - Matthes, FLORIAN

PY - 2021/4/2

Y1 - 2021/4/2

N2 - Since the introduction of Word2Vec in 2013, so-called word embeddings, dense vector representation of words that are supposed to capture their semantic meaning, have become a universally applied technique in a wide range of Natural Language Processing (NLP) tasks and domains. The vector representations they provide are learned on huge corpora of unlabeled text data. Due to the large amount of data and computing power that is necessary to train such embedding models, very often, pre-trained models are applied which have been trained on domain unspecific data like newspaper articles or Wikipedia entries. In this paper, we present a domain-specific embedding model that is trained exclusively on texts from the domain of engineering. We will show that such a domain-specific embeddings model performs better in different NLP tasks and can therefore help to improve NLP-based AI in the domain of Engineering.

AB - Since the introduction of Word2Vec in 2013, so-called word embeddings, dense vector representation of words that are supposed to capture their semantic meaning, have become a universally applied technique in a wide range of Natural Language Processing (NLP) tasks and domains. The vector representations they provide are learned on huge corpora of unlabeled text data. Due to the large amount of data and computing power that is necessary to train such embedding models, very often, pre-trained models are applied which have been trained on domain unspecific data like newspaper articles or Wikipedia entries. In this paper, we present a domain-specific embedding model that is trained exclusively on texts from the domain of engineering. We will show that such a domain-specific embeddings model performs better in different NLP tasks and can therefore help to improve NLP-based AI in the domain of Engineering.

KW - Engineering

KW - Word Embeddings

UR - http://www.scopus.com/inward/record.url?scp=85122640265&partnerID=8YFLogxK

U2 - 10.1145/3460824.3460826

DO - 10.1145/3460824.3460826

M3 - Conference contribution

AN - SCOPUS:85122640265

T3 - ACM International Conference Proceeding Series

SP - 8

EP - 12

BT - 2021 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021

PB - Association for Computing Machinery

Y2 - 2 April 2021 through 4 April 2021

ER -

Braun DANIEL, Klymenko OLEKSANDRA , Schopf TIM, Kaan Akan YUSUF, Matthes FLORIAN. The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering. In 2021 3rd International Conference on Management Science and Industrial Engineering, MSIE 2021. Association for Computing Machinery. 2021. p. 8-12. (ACM International Conference Proceeding Series). doi: 10.1145/3460824.3460826

The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

TSaaS: Technology Scouting as a Service - KI gestütztes Matching von Technologien mit Problemstellungen aus dem Maschinen- und Anlagenbau

Cite this

The Language of Engineering: Training a Domain-Specific Word Embedding Model for Engineering

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

TSaaS: Technology Scouting as a Service - KI gestütztes Matching von Technologien mit Problemstellungen aus dem Maschinen- und Anlagenbau

Cite this