BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring

Zhenrui Yue; Camilo Requena Witzig; Daniel Jorde; Hans Arno Jacobsen

doi:10.1145/3427771.3429390

BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring

Zhenrui Yue
, Camilo Requena Witzig
, Daniel Jorde
, Hans Arno Jacobsen

Informatics 13 - Chair of Decentralized Information Systems and Data Management

Technical University of Munich

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

153 Scopus citations

Abstract

Non-intrusive load monitoring (NILM) based energy disaggregation is the decomposition of a system's energy into the consumption of its individual appliances. Previous work on deep learning NILMalgorithms has shown great potential in the field of energy management and smart grids. In this paper, we propose BERT4NILM, an architecture based on bidirectional encoder representations from transformers (BERT) and an improved objective function designed specifically for NILM learning. We adaptthe bidirectional transformer architecture to the field of energy disaggregation and follow the pattern of sequence-to-sequence learning. With the improved loss function and masked training, BERT4NILM outperforms state-of-the-art models across various metrics on the two publicly available datasets UK-DALE and REDD.

Original language	English
Title of host publication	NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring
Publisher	Association for Computing Machinery, Inc
Pages	89-93
Number of pages	5
ISBN (Electronic)	9781450381918
DOIs	https://doi.org/10.1145/3427771.3429390
State	Published - 18 Nov 2020
Event	5th International Workshop on Non-Intrusive Load Monitoring, NILM 2020, co-located with ACM BuildSys 2020 - Virtual, Online, Japan Duration: 18 Nov 2020 → …

Publication series

Name	NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring

Conference

Conference	5th International Workshop on Non-Intrusive Load Monitoring, NILM 2020, co-located with ACM BuildSys 2020
Country/Territory	Japan
City	Virtual, Online
Period	18/11/20 → …

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 7 Affordable and Clean Energy

Keywords

Deep Learning
Energy Disaggregation
NILM
Neural Network
Non-Intrusive Load Monitoring
Transformer

Access to Document

10.1145/3427771.3429390

Cite this

Yue, Z., Witzig, C. R., Jorde, D., & Jacobsen, H. A. (2020). BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring. In NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring (pp. 89-93). (NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring). Association for Computing Machinery, Inc. https://doi.org/10.1145/3427771.3429390

Yue, Zhenrui ; Witzig, Camilo Requena ; Jorde, Daniel et al. / BERT4NILM : A Bidirectional Transformer Model for Non-Intrusive Load Monitoring. NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring. Association for Computing Machinery, Inc, 2020. pp. 89-93 (NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring).

@inproceedings{a9edb6e818c34495985b93208eab5dcb,

title = "BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring",

abstract = "Non-intrusive load monitoring (NILM) based energy disaggregation is the decomposition of a system's energy into the consumption of its individual appliances. Previous work on deep learning NILMalgorithms has shown great potential in the field of energy management and smart grids. In this paper, we propose BERT4NILM, an architecture based on bidirectional encoder representations from transformers (BERT) and an improved objective function designed specifically for NILM learning. We adaptthe bidirectional transformer architecture to the field of energy disaggregation and follow the pattern of sequence-to-sequence learning. With the improved loss function and masked training, BERT4NILM outperforms state-of-the-art models across various metrics on the two publicly available datasets UK-DALE and REDD.",

keywords = "Deep Learning, Energy Disaggregation, NILM, Neural Network, Non-Intrusive Load Monitoring, Transformer",

author = "Zhenrui Yue and Witzig, \{Camilo Requena\} and Daniel Jorde and Jacobsen, \{Hans Arno\}",

note = "Publisher Copyright: {\textcopyright} 2020 ACM.; 5th International Workshop on Non-Intrusive Load Monitoring, NILM 2020, co-located with ACM BuildSys 2020 ; Conference date: 18-11-2020",

year = "2020",

month = nov,

day = "18",

doi = "10.1145/3427771.3429390",

language = "English",

series = "NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring",

publisher = "Association for Computing Machinery, Inc",

pages = "89--93",

booktitle = "NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring",

}

Yue, Z, Witzig, CR, Jorde, D & Jacobsen, HA 2020, BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring. in NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring. NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring, Association for Computing Machinery, Inc, pp. 89-93, 5th International Workshop on Non-Intrusive Load Monitoring, NILM 2020, co-located with ACM BuildSys 2020, Virtual, Online, Japan, 18/11/20. https://doi.org/10.1145/3427771.3429390

BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring. / Yue, Zhenrui; Witzig, Camilo Requena; Jorde, Daniel et al.
NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring. Association for Computing Machinery, Inc, 2020. p. 89-93 (NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - BERT4NILM

T2 - 5th International Workshop on Non-Intrusive Load Monitoring, NILM 2020, co-located with ACM BuildSys 2020

AU - Yue, Zhenrui

AU - Witzig, Camilo Requena

AU - Jorde, Daniel

AU - Jacobsen, Hans Arno

PY - 2020/11/18

Y1 - 2020/11/18

N2 - Non-intrusive load monitoring (NILM) based energy disaggregation is the decomposition of a system's energy into the consumption of its individual appliances. Previous work on deep learning NILMalgorithms has shown great potential in the field of energy management and smart grids. In this paper, we propose BERT4NILM, an architecture based on bidirectional encoder representations from transformers (BERT) and an improved objective function designed specifically for NILM learning. We adaptthe bidirectional transformer architecture to the field of energy disaggregation and follow the pattern of sequence-to-sequence learning. With the improved loss function and masked training, BERT4NILM outperforms state-of-the-art models across various metrics on the two publicly available datasets UK-DALE and REDD.

AB - Non-intrusive load monitoring (NILM) based energy disaggregation is the decomposition of a system's energy into the consumption of its individual appliances. Previous work on deep learning NILMalgorithms has shown great potential in the field of energy management and smart grids. In this paper, we propose BERT4NILM, an architecture based on bidirectional encoder representations from transformers (BERT) and an improved objective function designed specifically for NILM learning. We adaptthe bidirectional transformer architecture to the field of energy disaggregation and follow the pattern of sequence-to-sequence learning. With the improved loss function and masked training, BERT4NILM outperforms state-of-the-art models across various metrics on the two publicly available datasets UK-DALE and REDD.

KW - Deep Learning

KW - Energy Disaggregation

KW - NILM

KW - Neural Network

KW - Non-Intrusive Load Monitoring

KW - Transformer

UR - https://www.scopus.com/pages/publications/85097599766

U2 - 10.1145/3427771.3429390

DO - 10.1145/3427771.3429390

M3 - Conference contribution

AN - SCOPUS:85097599766

T3 - NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring

SP - 89

EP - 93

BT - NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring

PB - Association for Computing Machinery, Inc

Y2 - 18 November 2020

ER -

Yue Z, Witzig CR, Jorde D, Jacobsen HA. BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring. In NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring. Association for Computing Machinery, Inc. 2020. p. 89-93. (NILM 2020 - Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring). doi: 10.1145/3427771.3429390

BERT4NILM: A Bidirectional Transformer Model for Non-Intrusive Load Monitoring

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this