This document details the catalogue of extensions developed for Cygnus on top of Apache Flume.
The Flume extensions catalogue is a basic piece of documentation for all those FIWARE users using Cygnus. It describes the available extra components added to the Flume technology in order to deal with Twitter-like data.
Software developers may also be interested in this catalogue since it may guide the creation of new components (specially, sinks) for Cygnus/Flume.
Structure of the document
This document describes the Twitter Source and Twitter HDFS sink.
TwitterSource is a source designed to collect data from Twitter. This document contains an explanation about
TwitterSource configuration and functionality.
TwitterHDFSSink sink is currently the only one supported by the cygnus-twitter agent. This document contains an explanation about
TwitterHDFSSink functionality (including how the information within a Flume event is mapped into the storage data structures), configuration, uses cases and implementation details are given.