Fury Kubernetes Logging

Fury's logging stack is based on Elasticsearch, a popular open source search engine, and other tools in its ecosystem, like Kibana, an open source analytics and visualization platform for Elasticsearch. All logging components are deployed on top of Kubernetes, giving your logging stack the reliability and resilience Kubernetes can offer.

With Fury Kubernetes Logging, logs for all infrastructure components get collected out of the box. You can add your containerized application logs to the colletction by writing your application logs to stdout for output and stderr for errors.

Supported platforms

The Fury Kubernetes Logging module can be deployed on the following platforms

  • AWS
  • on-premises or unmanaged cloud clusters
  • Google Kubernetes Engine (GKE)
  • Azure Kubernetes Service (AKS)

Architecture

Architecture

The Fury Kubernetes Logging module comes with two architectural options, depending on the logging and traffic volume of your application: you can have a single Elasticsearch instance (if you're running a simple API with not a lot of traffic) or you can opt for the HA (High Availability) solution and run a cluster with three Elasticsearch instances (if you're running something like a SaaS platform on top of Kubernetes).

Cluster level log collection in Kubernetes works by running a node level logging agent, fluentbit, as a DaemonSet (i.e. a pod in every node) that collects container logs, parsing and enriching them with Kubernetes metadata and finally pushing them to Elasticsearch via fluentd.

You can then query logs using Kibana - see its tutorials for more information.

Logging Stack

The following packages are included in the Fury Kubernetes Logging stack. All the resources listed below are going to be deployed in the logging namespace in your Kubernetes cluster.

Package Description
cerebro Cerebro is an open source web admin tool for the Elasticsearch API that helps you manage your Elasticsearch cluster via a graphical user interface.
curator Curator helps you manage your Elasticsearch indices and snapshots. With Curator you can manage the retention policy for your infrastructure logs.
elasticsearch-single Single node Elasticsearch deployment.
elasticsearch-triple Three node Elasticsearch cluster deployment.
fluentd Fluentd is an open source data collector for unified logging that can store collected data in Elasticsearch.
kibana Kibana is an open source analytics and visualization platform for Elasticsearch. Kibana lets you perform advanced data analysis and visualize data in a variety of charts, tables and maps. You can use it to search, view and interact with data stored in Elasticsearch indices.

Deploy

Deploy your logging stack