Integrating multimodal cues using grammar based models

Manuel Giuliani; Alois Knoll

doi:10.1007/978-3-540-73281-5_94

Integrating multimodal cues using grammar based models

Manuel Giuliani, Alois Knoll

nformatics 6 - Chair of IRobotics, Artificial Intelligence and Real-time Systems

Technical University of Munich

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Scopus citations

Abstract

Multimodal systems must process several input streams efficiently and represent the input in a way that allows the establishment of connections between modalities. This paper describes a multimodal system that uses Combinatory Categorial Grammars to parse several input streams and translate them into logical formulas. These logical formulas are expressed in Hybrid Logic, which is very suitable for multimodal integration because it can represent temporal relationships between modes in an abstract way. This level of abstraction makes it possible to define rules for multimodal processing in a straightforward way.

Original language	English
Title of host publication	Universal Access in Human-Computer Interaction
Subtitle of host publication	Ambient Interaction - 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007. Held as Part of HCI Int. 2007 Proc.
Publisher	Springer Verlag
Pages	858-867
Number of pages	10
Edition	PART 2
ISBN (Print)	9783540732808
DOIs	https://doi.org/10.1007/978-3-540-73281-5_94
State	Published - 2007
Event	4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007 - Beijing, China Duration: 22 Jul 2007 → 27 Jul 2007

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	PART 2
Volume	4555 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007
Country/Territory	China
City	Beijing
Period	22/07/07 → 27/07/07

Access to Document

10.1007/978-3-540-73281-5_94

Cite this

Giuliani, M., & Knoll, A. (2007). Integrating multimodal cues using grammar based models. In Universal Access in Human-Computer Interaction: Ambient Interaction - 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007. Held as Part of HCI Int. 2007 Proc. (PART 2 ed., pp. 858-867). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4555 LNCS, No. PART 2). Springer Verlag. https://doi.org/10.1007/978-3-540-73281-5_94

Giuliani, Manuel ; Knoll, Alois. / Integrating multimodal cues using grammar based models. Universal Access in Human-Computer Interaction: Ambient Interaction - 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007. Held as Part of HCI Int. 2007 Proc.. PART 2. ed. Springer Verlag, 2007. pp. 858-867 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2).

@inproceedings{cf9f7a79340f4964bd5caa525203f345,

title = "Integrating multimodal cues using grammar based models",

abstract = "Multimodal systems must process several input streams efficiently and represent the input in a way that allows the establishment of connections between modalities. This paper describes a multimodal system that uses Combinatory Categorial Grammars to parse several input streams and translate them into logical formulas. These logical formulas are expressed in Hybrid Logic, which is very suitable for multimodal integration because it can represent temporal relationships between modes in an abstract way. This level of abstraction makes it possible to define rules for multimodal processing in a straightforward way.",

author = "Manuel Giuliani and Alois Knoll",

year = "2007",

doi = "10.1007/978-3-540-73281-5_94",

language = "English",

isbn = "9783540732808",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

number = "PART 2",

pages = "858--867",

booktitle = "Universal Access in Human-Computer Interaction",

edition = "PART 2",

note = "4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007 ; Conference date: 22-07-2007 Through 27-07-2007",

}

Giuliani, M & Knoll, A 2007, Integrating multimodal cues using grammar based models. in Universal Access in Human-Computer Interaction: Ambient Interaction - 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007. Held as Part of HCI Int. 2007 Proc.. PART 2 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 2, vol. 4555 LNCS, Springer Verlag, pp. 858-867, 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007, Beijing, China, 22/07/07. https://doi.org/10.1007/978-3-540-73281-5_94

Integrating multimodal cues using grammar based models. / Giuliani, Manuel; Knoll, Alois.
Universal Access in Human-Computer Interaction: Ambient Interaction - 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007. Held as Part of HCI Int. 2007 Proc.. PART 2. ed. Springer Verlag, 2007. p. 858-867 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4555 LNCS, No. PART 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Integrating multimodal cues using grammar based models

AU - Giuliani, Manuel

AU - Knoll, Alois

PY - 2007

Y1 - 2007

N2 - Multimodal systems must process several input streams efficiently and represent the input in a way that allows the establishment of connections between modalities. This paper describes a multimodal system that uses Combinatory Categorial Grammars to parse several input streams and translate them into logical formulas. These logical formulas are expressed in Hybrid Logic, which is very suitable for multimodal integration because it can represent temporal relationships between modes in an abstract way. This level of abstraction makes it possible to define rules for multimodal processing in a straightforward way.

AB - Multimodal systems must process several input streams efficiently and represent the input in a way that allows the establishment of connections between modalities. This paper describes a multimodal system that uses Combinatory Categorial Grammars to parse several input streams and translate them into logical formulas. These logical formulas are expressed in Hybrid Logic, which is very suitable for multimodal integration because it can represent temporal relationships between modes in an abstract way. This level of abstraction makes it possible to define rules for multimodal processing in a straightforward way.

UR - http://www.scopus.com/inward/record.url?scp=38149053105&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-73281-5_94

DO - 10.1007/978-3-540-73281-5_94

M3 - Conference contribution

AN - SCOPUS:38149053105

SN - 9783540732808

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 858

EP - 867

BT - Universal Access in Human-Computer Interaction

PB - Springer Verlag

T2 - 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007

Y2 - 22 July 2007 through 27 July 2007

ER -

Giuliani M, Knoll A. Integrating multimodal cues using grammar based models. In Universal Access in Human-Computer Interaction: Ambient Interaction - 4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007. Held as Part of HCI Int. 2007 Proc.. PART 2 ed. Springer Verlag. 2007. p. 858-867. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2). doi: 10.1007/978-3-540-73281-5_94

Integrating multimodal cues using grammar based models

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this