Asad, Zakia, Asad Rehman Chaudhry, Mohammad and Malone, David (2015) CodHoop: A system for optimizing big data processing. In: 2015 Annual IEEE Systems Conference (SysCon) Proceedings, 13-16 April 2015, Vancouver, BC.
Preview
DM-CodHoop-2015.pdf
Download (262kB) | Preview
Abstract
The rise of the cloud and distributed data-intensive (“Big Data”) applications puts pressure on data center networks due to the movement of massive volumes of data. This paper proposes CodHoop a system employing network coding techniques, specifically index coding, as a means of dynamically-controlled reduction in volume of communication. Using Hadoop as a representative of this class of applications, a motivating use-case is presented. The proof-of-concept implementation results exhibit an average advantage of 31% compared to vanilla Hadoop implementation which depending on use-case translates to 31% less energy utilization of the equipment, 31% more jobs that run simultaneously, or to a 31% decrease in job completion time.
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Keywords: | CodHoop; Big Data processing; cloud applications; distributed data-intensive applications; data center networks; Hadoop; proof-of-concept implementation; |
Academic Unit: | Faculty of Science and Engineering > Mathematics and Statistics Faculty of Science and Engineering > Research Institutes > Hamilton Institute |
Item ID: | 10066 |
Depositing User: | Dr. David Malone |
Date Deposited: | 05 Oct 2018 13:34 |
Refereed: | Yes |
Funders: | Science Foundation Ireland (SFI) |
URI: | https://mu.eprints-hosting.org/id/eprint/10066 |
Use Licence: | This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available here |
Repository Staff Only (login required)
Downloads
Downloads per month over past year