Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization

Mozhi Zhang; Keyulu Xu; Ken Ichi Kawarabayashi; Stefanie Jegelka; Jordan Boyd-Graber

Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization

Mozhi Zhang, Keyulu Xu, Ken Ichi Kawarabayashi, Stefanie Jegelka, Jordan Boyd-Graber

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

24 Scopus citations

Abstract

Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings. However, orthogonal mapping only works on language pairs whose embeddings are naturally isomorphic. For non-isomorphic pairs, our method (Iterative Normalization) transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language's average vector is zero. Iterative Normalization consistently improves word translation accuracy of three CLWE methods, with the largest improvement observed on English-Japanese (from 2% to 44% test accuracy).

Original language	English
Title of host publication	ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference
Publisher	Association for Computational Linguistics (ACL)
Pages	3180-3189
Number of pages	10
ISBN (Electronic)	9781950737482
State	Published - 2020
Externally published	Yes
Event	57th Annual Meeting of the Association for Computational Linguistics, ACL 2019 - Florence, Italy Duration: 28 Jul 2019 → 2 Aug 2019

Publication series

Name	ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

Conference

Conference	57th Annual Meeting of the Association for Computational Linguistics, ACL 2019
Country/Territory	Italy
City	Florence
Period	28/07/19 → 2/08/19

Cite this

Zhang, M., Xu, K., Kawarabayashi, K. I., Jegelka, S., & Boyd-Graber, J. (2020). Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization. In ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (pp. 3180-3189). (ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference). Association for Computational Linguistics (ACL).

Zhang, Mozhi ; Xu, Keyulu ; Kawarabayashi, Ken Ichi et al. / Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization. ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Association for Computational Linguistics (ACL), 2020. pp. 3180-3189 (ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference).

@inproceedings{9b7f8a4925664e619fbeef30931be581,

title = "Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization",

abstract = "Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings. However, orthogonal mapping only works on language pairs whose embeddings are naturally isomorphic. For non-isomorphic pairs, our method (Iterative Normalization) transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language's average vector is zero. Iterative Normalization consistently improves word translation accuracy of three CLWE methods, with the largest improvement observed on English-Japanese (from 2% to 44% test accuracy).",

author = "Mozhi Zhang and Keyulu Xu and Kawarabayashi, {Ken Ichi} and Stefanie Jegelka and Jordan Boyd-Graber",

note = "Publisher Copyright: {\textcopyright} 2019 Association for Computational Linguistics; 57th Annual Meeting of the Association for Computational Linguistics, ACL 2019 ; Conference date: 28-07-2019 Through 02-08-2019",

year = "2020",

language = "English",

series = "ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference",

publisher = "Association for Computational Linguistics (ACL)",

pages = "3180--3189",

booktitle = "ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference",

}

Zhang, M, Xu, K, Kawarabayashi, KI, Jegelka, S & Boyd-Graber, J 2020, Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization. in ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Association for Computational Linguistics (ACL), pp. 3180-3189, 57th Annual Meeting of the Association for Computational Linguistics, ACL 2019, Florence, Italy, 28/07/19.

Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization. / Zhang, Mozhi; Xu, Keyulu; Kawarabayashi, Ken Ichi et al.
ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Association for Computational Linguistics (ACL), 2020. p. 3180-3189 (ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization

AU - Zhang, Mozhi

AU - Xu, Keyulu

AU - Kawarabayashi, Ken Ichi

AU - Jegelka, Stefanie

AU - Boyd-Graber, Jordan

PY - 2020

Y1 - 2020

N2 - Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings. However, orthogonal mapping only works on language pairs whose embeddings are naturally isomorphic. For non-isomorphic pairs, our method (Iterative Normalization) transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language's average vector is zero. Iterative Normalization consistently improves word translation accuracy of three CLWE methods, with the largest improvement observed on English-Japanese (from 2% to 44% test accuracy).

AB - Cross-lingual word embeddings (CLWE) underlie many multilingual natural language processing systems, often through orthogonal transformations of pre-trained monolingual embeddings. However, orthogonal mapping only works on language pairs whose embeddings are naturally isomorphic. For non-isomorphic pairs, our method (Iterative Normalization) transforms monolingual embeddings to make orthogonal alignment easier by simultaneously enforcing that (1) individual word vectors are unit length, and (2) each language's average vector is zero. Iterative Normalization consistently improves word translation accuracy of three CLWE methods, with the largest improvement observed on English-Japanese (from 2% to 44% test accuracy).

UR - http://www.scopus.com/inward/record.url?scp=85084069366&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85084069366

T3 - ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

SP - 3180

EP - 3189

BT - ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

PB - Association for Computational Linguistics (ACL)

T2 - 57th Annual Meeting of the Association for Computational Linguistics, ACL 2019

Y2 - 28 July 2019 through 2 August 2019

ER -

Zhang M, Xu K, Kawarabayashi KI, Jegelka S, Boyd-Graber J. Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization. In ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Association for Computational Linguistics (ACL). 2020. p. 3180-3189. (ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference).

Are girls neko or shojo? Cross-lingual alignment of non-isomorphic embeddings with iterative normalization

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this