Improving hadoop hive query response times through efficient virtual resource allocation

Tansel Dokeroglu; Muhammet Serkan Cınar; Seyyit Alper Sert; Ahmet Cosar; Adnan Yazıcı

doi:10.1007/978-3-319-26154-6_17

Improving hadoop hive query response times through efficient virtual resource allocation

Tansel Dokeroglu, Muhammet Serkan Cınar, Seyyit Alper Sert, Ahmet Cosar, Adnan Yazıcı

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

The performance of theMapReduce-based Cloud datawarehouses mainly depends on the virtual hardware resources allocated. Most of the time, the resources are values selected/given by the Cloud service providers. However, setting the right virtual resources in accordance with the workload demands of a query, such as the number of CPUs, the size of RAM, and the network bandwidth, will improve the response time when querying large data on an optimized system. In this study, we carried out a set of experiments with a well-known Mapreduce SQL-translator, Hadoop Hive, on benchmark decision support the TPC benchmark (TPC-H) database in order to analyze the performance sensitivity of the queries under different virtual resource settings. Our results provide valuable hints for the decision makers who design efficient MapReduce-based data warehouses on the Cloud.

Original language	English
Title of host publication	Flexible Query Answering Systems 2015 - Proceedings of the 11th International Conference, FQAS 2015
Editors	Olivier Pivert, Adnan Yazici, Henrik Larsen, Maria Amparo Vila, Troels Andreasen, Henning Christiansen, Janusz Kacprzyk, Slawomir Zadrozny, Guy De Tre, Gabriella Pasi
Publisher	Springer Verlag
Pages	215-225
Number of pages	11
ISBN (Print)	9783319261539
DOIs	https://doi.org/10.1007/978-3-319-26154-6_17
Publication status	Published - 2016
Externally published	Yes
Event	11th International Conference on Flexible Query Answering Systems, FQAS 2015 - Cracow, Poland Duration: Oct 26 2015 → Oct 28 2015

Publication series

Name	Advances in Intelligent Systems and Computing
Volume	400
ISSN (Print)	2194-5357

Conference

Conference	11th International Conference on Flexible Query Answering Systems, FQAS 2015
Country/Territory	Poland
City	Cracow
Period	10/26/15 → 10/28/15

Keywords

Hadoop
Hive
Multi-objective query Optimization
Virtual resource allocation

ASJC Scopus subject areas

Control and Systems Engineering
General Computer Science

Access to Document

10.1007/978-3-319-26154-6_17

Cite this

Dokeroglu, T., Cınar, M. S., Sert, S. A., Cosar, A., & Yazıcı, A. (2016). Improving hadoop hive query response times through efficient virtual resource allocation. In O. Pivert, A. Yazici, H. Larsen, M. A. Vila, T. Andreasen, H. Christiansen, J. Kacprzyk, S. Zadrozny, G. De Tre, & G. Pasi (Eds.), Flexible Query Answering Systems 2015 - Proceedings of the 11th International Conference, FQAS 2015 (pp. 215-225). (Advances in Intelligent Systems and Computing; Vol. 400). Springer Verlag. https://doi.org/10.1007/978-3-319-26154-6_17

Improving hadoop hive query response times through efficient virtual resource allocation. / Dokeroglu, Tansel; Cınar, Muhammet Serkan; Sert, Seyyit Alper et al.
Flexible Query Answering Systems 2015 - Proceedings of the 11th International Conference, FQAS 2015. ed. / Olivier Pivert; Adnan Yazici; Henrik Larsen; Maria Amparo Vila; Troels Andreasen; Henning Christiansen; Janusz Kacprzyk; Slawomir Zadrozny; Guy De Tre; Gabriella Pasi. Springer Verlag, 2016. p. 215-225 (Advances in Intelligent Systems and Computing; Vol. 400).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Dokeroglu, T, Cınar, MS, Sert, SA, Cosar, A & Yazıcı, A 2016, Improving hadoop hive query response times through efficient virtual resource allocation. in O Pivert, A Yazici, H Larsen, MA Vila, T Andreasen, H Christiansen, J Kacprzyk, S Zadrozny, G De Tre & G Pasi (eds), Flexible Query Answering Systems 2015 - Proceedings of the 11th International Conference, FQAS 2015. Advances in Intelligent Systems and Computing, vol. 400, Springer Verlag, pp. 215-225, 11th International Conference on Flexible Query Answering Systems, FQAS 2015, Cracow, Poland, 10/26/15. https://doi.org/10.1007/978-3-319-26154-6_17

Dokeroglu T, Cınar MS, Sert SA, Cosar A, Yazıcı A. Improving hadoop hive query response times through efficient virtual resource allocation. In Pivert O, Yazici A, Larsen H, Vila MA, Andreasen T, Christiansen H, Kacprzyk J, Zadrozny S, De Tre G, Pasi G, editors, Flexible Query Answering Systems 2015 - Proceedings of the 11th International Conference, FQAS 2015. Springer Verlag. 2016. p. 215-225. (Advances in Intelligent Systems and Computing). doi: 10.1007/978-3-319-26154-6_17

Dokeroglu, Tansel ; Cınar, Muhammet Serkan ; Sert, Seyyit Alper et al. / Improving hadoop hive query response times through efficient virtual resource allocation. Flexible Query Answering Systems 2015 - Proceedings of the 11th International Conference, FQAS 2015. editor / Olivier Pivert ; Adnan Yazici ; Henrik Larsen ; Maria Amparo Vila ; Troels Andreasen ; Henning Christiansen ; Janusz Kacprzyk ; Slawomir Zadrozny ; Guy De Tre ; Gabriella Pasi. Springer Verlag, 2016. pp. 215-225 (Advances in Intelligent Systems and Computing).

@inproceedings{3692b3d6419041efb7008f46d7a509ce,

title = "Improving hadoop hive query response times through efficient virtual resource allocation",

abstract = "The performance of theMapReduce-based Cloud datawarehouses mainly depends on the virtual hardware resources allocated. Most of the time, the resources are values selected/given by the Cloud service providers. However, setting the right virtual resources in accordance with the workload demands of a query, such as the number of CPUs, the size of RAM, and the network bandwidth, will improve the response time when querying large data on an optimized system. In this study, we carried out a set of experiments with a well-known Mapreduce SQL-translator, Hadoop Hive, on benchmark decision support the TPC benchmark (TPC-H) database in order to analyze the performance sensitivity of the queries under different virtual resource settings. Our results provide valuable hints for the decision makers who design efficient MapReduce-based data warehouses on the Cloud.",

keywords = "Hadoop, Hive, Multi-objective query Optimization, Virtual resource allocation",

author = "Tansel Dokeroglu and Cınar, {Muhammet Serkan} and Sert, {Seyyit Alper} and Ahmet Cosar and Adnan Yazıcı",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2016.; 11th International Conference on Flexible Query Answering Systems, FQAS 2015 ; Conference date: 26-10-2015 Through 28-10-2015",

year = "2016",

doi = "10.1007/978-3-319-26154-6_17",

language = "English",

isbn = "9783319261539",

series = "Advances in Intelligent Systems and Computing",

publisher = "Springer Verlag",

pages = "215--225",

editor = "Olivier Pivert and Adnan Yazici and Henrik Larsen and Vila, {Maria Amparo} and Troels Andreasen and Henning Christiansen and Janusz Kacprzyk and Slawomir Zadrozny and {De Tre}, Guy and Gabriella Pasi",

booktitle = "Flexible Query Answering Systems 2015 - Proceedings of the 11th International Conference, FQAS 2015",

address = "Germany",

}

TY - GEN

T1 - Improving hadoop hive query response times through efficient virtual resource allocation

AU - Dokeroglu, Tansel

AU - Cınar, Muhammet Serkan

AU - Sert, Seyyit Alper

AU - Cosar, Ahmet

AU - Yazıcı, Adnan

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2016.

PY - 2016

Y1 - 2016

N2 - The performance of theMapReduce-based Cloud datawarehouses mainly depends on the virtual hardware resources allocated. Most of the time, the resources are values selected/given by the Cloud service providers. However, setting the right virtual resources in accordance with the workload demands of a query, such as the number of CPUs, the size of RAM, and the network bandwidth, will improve the response time when querying large data on an optimized system. In this study, we carried out a set of experiments with a well-known Mapreduce SQL-translator, Hadoop Hive, on benchmark decision support the TPC benchmark (TPC-H) database in order to analyze the performance sensitivity of the queries under different virtual resource settings. Our results provide valuable hints for the decision makers who design efficient MapReduce-based data warehouses on the Cloud.

AB - The performance of theMapReduce-based Cloud datawarehouses mainly depends on the virtual hardware resources allocated. Most of the time, the resources are values selected/given by the Cloud service providers. However, setting the right virtual resources in accordance with the workload demands of a query, such as the number of CPUs, the size of RAM, and the network bandwidth, will improve the response time when querying large data on an optimized system. In this study, we carried out a set of experiments with a well-known Mapreduce SQL-translator, Hadoop Hive, on benchmark decision support the TPC benchmark (TPC-H) database in order to analyze the performance sensitivity of the queries under different virtual resource settings. Our results provide valuable hints for the decision makers who design efficient MapReduce-based data warehouses on the Cloud.

KW - Hadoop

KW - Hive

KW - Multi-objective query Optimization

KW - Virtual resource allocation

UR - http://www.scopus.com/inward/record.url?scp=84983200622&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84983200622&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-26154-6_17

DO - 10.1007/978-3-319-26154-6_17

M3 - Conference contribution

AN - SCOPUS:84983200622

SN - 9783319261539

T3 - Advances in Intelligent Systems and Computing

SP - 215

EP - 225

BT - Flexible Query Answering Systems 2015 - Proceedings of the 11th International Conference, FQAS 2015

A2 - Pivert, Olivier

A2 - Yazici, Adnan

A2 - Larsen, Henrik

A2 - Vila, Maria Amparo

A2 - Andreasen, Troels

A2 - Christiansen, Henning

A2 - Kacprzyk, Janusz

A2 - Zadrozny, Slawomir

A2 - De Tre, Guy

A2 - Pasi, Gabriella

PB - Springer Verlag

T2 - 11th International Conference on Flexible Query Answering Systems, FQAS 2015

Y2 - 26 October 2015 through 28 October 2015

ER -

Improving hadoop hive query response times through efficient virtual resource allocation

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this