Big data generation

Tilmann Rabl; Hans Arno Jacobsen

doi:10.1007/978-3-642-53974-9_3

Big data generation

Tilmann Rabl, Hans Arno Jacobsen

University of Toronto

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

15 Scopus citations

Abstract

Big data challenges are end-to-end problems. When handling big data it usually has to be preprocessed, moved, loaded, processed, and stored many times. This has led to the creation of big data pipelines. Current benchmarks related to big data only focus on isolated aspects of this pipeline, usually the processing, storage and loading aspects. To this date, there has not been any benchmark presented covering the end-to-end aspect for big data systems. In this paper, we discuss the necessity of ETL like tasks in big data benchmarking and propose the Parallel Data Generation Framework (PDGF) for its data generation. PDGF is a generic data generator that was implemented at the University of Passau and is currently adopted in TPC benchmarks.

Original language	English
Title of host publication	Specifying Big Data Benchmarks - First Workshop, WBDB 2012, and Second Workshop, WBDB 2012, Revised Selected Papers
Publisher	Springer Verlag
Pages	20-27
Number of pages	8
ISBN (Print)	9783642539732
DOIs	https://doi.org/10.1007/978-3-642-53974-9_3
State	Published - 2014
Externally published	Yes
Event	2nd Workshop on Specifying Big Data Benchmarks, WBDB 2012 - Pune, India Duration: 17 Dec 2012 → 18 Dec 2012

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	8163 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	2nd Workshop on Specifying Big Data Benchmarks, WBDB 2012
Country/Territory	India
City	Pune
Period	17/12/12 → 18/12/12

Access to Document

10.1007/978-3-642-53974-9_3

Cite this

Rabl, T., & Jacobsen, H. A. (2014). Big data generation. In Specifying Big Data Benchmarks - First Workshop, WBDB 2012, and Second Workshop, WBDB 2012, Revised Selected Papers (pp. 20-27). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8163 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-642-53974-9_3

@inproceedings{db6a9140d8ab44868ed244960a4a890e,

title = "Big data generation",

abstract = "Big data challenges are end-to-end problems. When handling big data it usually has to be preprocessed, moved, loaded, processed, and stored many times. This has led to the creation of big data pipelines. Current benchmarks related to big data only focus on isolated aspects of this pipeline, usually the processing, storage and loading aspects. To this date, there has not been any benchmark presented covering the end-to-end aspect for big data systems. In this paper, we discuss the necessity of ETL like tasks in big data benchmarking and propose the Parallel Data Generation Framework (PDGF) for its data generation. PDGF is a generic data generator that was implemented at the University of Passau and is currently adopted in TPC benchmarks.",

author = "Tilmann Rabl and Jacobsen, {Hans Arno}",

year = "2014",

doi = "10.1007/978-3-642-53974-9_3",

language = "English",

isbn = "9783642539732",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "20--27",

booktitle = "Specifying Big Data Benchmarks - First Workshop, WBDB 2012, and Second Workshop, WBDB 2012, Revised Selected Papers",

note = "2nd Workshop on Specifying Big Data Benchmarks, WBDB 2012 ; Conference date: 17-12-2012 Through 18-12-2012",

}

Rabl, T & Jacobsen, HA 2014, Big data generation. in Specifying Big Data Benchmarks - First Workshop, WBDB 2012, and Second Workshop, WBDB 2012, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8163 LNCS, Springer Verlag, pp. 20-27, 2nd Workshop on Specifying Big Data Benchmarks, WBDB 2012, Pune, India, 17/12/12. https://doi.org/10.1007/978-3-642-53974-9_3

Big data generation. / Rabl, Tilmann; Jacobsen, Hans Arno.
Specifying Big Data Benchmarks - First Workshop, WBDB 2012, and Second Workshop, WBDB 2012, Revised Selected Papers. Springer Verlag, 2014. p. 20-27 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8163 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Big data generation

AU - Rabl, Tilmann

AU - Jacobsen, Hans Arno

PY - 2014

Y1 - 2014

N2 - Big data challenges are end-to-end problems. When handling big data it usually has to be preprocessed, moved, loaded, processed, and stored many times. This has led to the creation of big data pipelines. Current benchmarks related to big data only focus on isolated aspects of this pipeline, usually the processing, storage and loading aspects. To this date, there has not been any benchmark presented covering the end-to-end aspect for big data systems. In this paper, we discuss the necessity of ETL like tasks in big data benchmarking and propose the Parallel Data Generation Framework (PDGF) for its data generation. PDGF is a generic data generator that was implemented at the University of Passau and is currently adopted in TPC benchmarks.

AB - Big data challenges are end-to-end problems. When handling big data it usually has to be preprocessed, moved, loaded, processed, and stored many times. This has led to the creation of big data pipelines. Current benchmarks related to big data only focus on isolated aspects of this pipeline, usually the processing, storage and loading aspects. To this date, there has not been any benchmark presented covering the end-to-end aspect for big data systems. In this paper, we discuss the necessity of ETL like tasks in big data benchmarking and propose the Parallel Data Generation Framework (PDGF) for its data generation. PDGF is a generic data generator that was implemented at the University of Passau and is currently adopted in TPC benchmarks.

UR - http://www.scopus.com/inward/record.url?scp=84958546415&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-53974-9_3

DO - 10.1007/978-3-642-53974-9_3

M3 - Conference contribution

AN - SCOPUS:84958546415

SN - 9783642539732

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 20

EP - 27

BT - Specifying Big Data Benchmarks - First Workshop, WBDB 2012, and Second Workshop, WBDB 2012, Revised Selected Papers

PB - Springer Verlag

T2 - 2nd Workshop on Specifying Big Data Benchmarks, WBDB 2012

Y2 - 17 December 2012 through 18 December 2012

ER -

Big data generation

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this