Jump to ContentJump to Main Navigation
Show Summary Details
More options …

it - Information Technology

Methods and Applications of Informatics and Information Technology

Editor-in-Chief: Conrad, Stefan

Online
ISSN
2196-7032
See all formats and pricing
More options …
Volume 58, Issue 4

Issues

Apache Flink in current research

Tilmann Rabl / Jonas Traub / Asterios Katsifodimos / Volker Markl
Published Online: 2016-06-24 | DOI: https://doi.org/10.1515/itit-2016-0005

Abstract

Recent trends in data collection and the decreasing prices of storage result in constantly growing amounts of analyzable data. These masses of data cannot easily be processed by traditional database systems as these do not allow for a sufficient degree of scalability. Programs especially designed for parallel data analysis on large-scale distributed systems are required. Developing such programs on clusters of commodity hardware is a complex challenge for even the most experienced system developers. Frameworks such as Apache Hadoop are scalable, but – when compared to SQL – extremely hard to program. The open-source platform Apache Flink is a link between conventional database systems and big data analysis frameworks. Flink is based on a fault tolerant runtime for data stream processing, which manages the distribution of data as well as communications within the cluster. A high diversity of use cases can be supported through various interfaces that allow for the implementation of data analysis processes. In this paper, we present an overview of Apache Flink as well as some current research activities on top of the Apache Flink ecosystem.

Keywords: Apache Flink; BBDC

ACM CCS: Information systems →Database management systems →Database management system engines →Parallel and distributed DBMSs →MapReduce-based systems

About the article

Tilmann Rabl

Tilmann Rabl is a senior researcher at the Database Systems and Information Management (DIMA) group at TU Berlin and at the German Research Center for Artificial Intelligence (DFKI). At DIMA he is research director and technical coordinator of the Berlin Big Data Center (BBDC). Tilmann Rabl is also cofounder of the startup bankmark.

Technische Universität Berlin, FG DIMA, Einsteinufer 17, 10587 Berlin, Germany

Jonas Traub

Jonas Traub is a research associate and PhD student at the Database Systems and Information Management group at TU Berlin. In his research he focuses on real-time processing of sensor data in the Internet of Things. Prior to that, Jonas Traub worked several years at IBM and as an independent consultant.

Technische Universität Berlin, FG DIMA, Einsteinufer 17, 10587 Berlin, Germany

Asterios Katsifodimos

Asterios Katsifodimos is a senior researcher at the Database Systems and Information Management (DIMA) group at TU Berlin and at the German Research Center for Artificial Intelligence (DFKI). He received his PhD from INRIA Saclay and Universite Paris-Sud.

Technische Universität Berlin, FG DIMA, Einsteinufer 17, 10587 Berlin, Germany

Volker Markl

Volker Markl is a Full Professor and Chair of the DIMA Group at TU Berlin and an Adjunct Full Professor at the University of Toronto. He is Director of the Intelligent Analytics for Massive Data Research Group at DFKI and Director of the Berlin Big Data Center.

Technische Universität Berlin, FG DIMA, Einsteinufer 17, 10587 Berlin, Germany


Accepted: 2016-05-07

Received: 2016-01-25

Published Online: 2016-06-24

Published in Print: 2016-08-28


Citation Information: it - Information Technology, Volume 58, Issue 4, Pages 157–165, ISSN (Online) 2196-7032, ISSN (Print) 1611-2776, DOI: https://doi.org/10.1515/itit-2016-0005.

Export Citation

©2016 Walter de Gruyter Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in