The wide spread of Big Data applications and services raised the need of implementing dedicated frameworks for the efficient management of data storage and access. Complex data center architectures have been defined to support such applications, where the network connections are often the bottleneck for data access and retrieval. In this paper we analyze a data center architecture based on a Fat-Tree topology running Hadoop as the framework for data management. The network is based on the software defined networking paradigm allowing the routing protocol to be switched between the spanning tree, the shortest path bridging and the equal cost multipath. The results show that performance of Hadoop are strongly influenced by the choice of the network protocols in dependence of the traffic load. The performance evaluation is based on a dedicated novel simulation framework named MaxHadoop.

Hadoop Performance Evaluation in Software Defined Data Center Networks

Carmenini A.;Marotta A.;Cassioli D.
2019-01-01

Abstract

The wide spread of Big Data applications and services raised the need of implementing dedicated frameworks for the efficient management of data storage and access. Complex data center architectures have been defined to support such applications, where the network connections are often the bottleneck for data access and retrieval. In this paper we analyze a data center architecture based on a Fat-Tree topology running Hadoop as the framework for data management. The network is based on the software defined networking paradigm allowing the routing protocol to be switched between the spanning tree, the shortest path bridging and the equal cost multipath. The results show that performance of Hadoop are strongly influenced by the choice of the network protocols in dependence of the traffic load. The performance evaluation is based on a dedicated novel simulation framework named MaxHadoop.
978-1-5386-9223-3
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11697/141766
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 3
social impact