Kubernetes Fury Ingress

Overview

Kubernetes Fury Ingress uses CNCF recommended, Cloud Native projects, such as Ingress NGINX an ingress controller using the well-known NGINX server as a URL path-based routing reverse proxy and load balancer, and cert-manager to automate the issuing and renewal of TLS certificates from various issuing sources.

The module also includes additional tools like Forecastle, a web-based global directory of all the services offered by your cluster.

Module's repository: https://github.com/sighupio/fury-kubernetes-ingress

Architecture

The reference architecture used to deploy the Fury Kubernetes Ingress Module is shown in the following figure:

• The traffic from end users arrives first at a Load Balancer that distributes the traffic between the available Ingress Controllers (usually, one for each availability zone).
• Once the traffic reaches the Ingress Controller, the Ingress proxies the traffic to the Kubernetes service based on the URL path of the request.
• The service is a Kubernetes abstraction that makes the traffic arrive at the pods where the actual application is running, usually using iptables rules.

For more information, please refer to Kubernetes Ingress official documentation.

Packages

Kubernetes Fury Ingress provides the following packages:

Package	Description
nginx	The NGINX Ingress Controller for Kubernetes provides delivery services for Kubernetes applications.
cert-manager	cert-manager is a Kubernetes add-on to automate the management and issuance of TLS certificates from various issuing sources.
external-dns	external-dns allows you to manage DNS records natively from Kubernetes in EKS clusters.
forecastle	Forecastle gives you access to a control panel where you can see your ingresses and access them on Kubernetes.

info

All the components are deployed in the ingress-nginx namespace in the cluster.

Compatibility

Kubernetes Version	Compatibility	Notes
1.28.x	✅	No known issues
1.29.x	✅	No known issues
1.30.x	✅	No known issues
1.31.x	✅	No known issues

Check the compatibility matrix for additional information on previous releases of the module.

Introduction: Ingress in Kubernetes

In Kubernetes, the Ingress resource provides a way to manage external access to services running within the cluster. Instead of exposing each service individually with LoadBalancers or NodePorts, an Ingress acts as a central entry point, routing external HTTP and HTTPS traffic to the appropriate backend services based on defined rules. Ingress resources enable functionalities like URL-based routing, SSL termination, and custom headers, streamlining how applications interact with external clients.

Using Ingress in Kubernetes simplifies managing traffic for large-scale applications by reducing the complexity of service exposure. It enables:

• Centralized Traffic Management: Control all ingress traffic at a single layer with customizable routing rules.
• SSL/TLS Termination: Simplify certificate management by handling encryption and decryption at the ingress layer.
• Scalability and Flexibility: Dynamically update routing configurations as workloads change without restarting services.

To use Ingress, a controller is required. The controller watches for Ingress resources in the cluster and implements the routing logic, translating it into configurations for an underlying proxy like NGINX.

KFD: Ingress module

The Ingress module provides KFD cluster with an NGINX Ingress Controller, an Ingress resource for each of the KFD-managed services (i.e. Grafana, Alertmanager...), cert-manager to deploy self-signed certificates for all internal Ingresses and Forecastle, a nice portal to group all Ingress links in one place.

You can choose three different configurations:

• None: NGINX Ingress Controller and Forecastle will not be installed. Note that cert-manager will still be included, as it's used also for Kubernetes certificates.
• single: a single NGINX Ingress Controller instance will be installed, for which the class name will be nginx.
• dual: KFD will deploy two separate NGINX Ingress Controller, one "internal" to be used for management purposes and one "external" to expose applications to end users.

If you are deploying KFD using the EKSCluster provider, KFD will also install external-dns to automatically manage Route53 DNS rules.

The NGINX Ingress Controller is deployed as a DaemonSet, meaning that there will be one instance per node (limited by default to the "infra" nodes), with the following default configuration:

• Maximum allowed size of the client request body: 10m
• HTTP status code used in redirects: 301
• Metrics are scraped by Prometheus every 10s
• Validating Admission webhook that validates an ingress definition does not break NGINX configuration.

You can see all available configuration options in the Provider's documentation

tip

If you want to also add your own Ingress resources to Forecastle, you can do that by annotating them:

kubectl annotate ingress <YOUR_INGRESS> "forecastle.stakater.com/expose=true" --overwrite

NGINX Ingress Controller

The NGINX Ingress Controller is one of the most popular and widely used ingress controllers for Kubernetes. It leverages the robust and proven NGINX web server to manage and route HTTP(S) traffic. Key features include:

• Advanced Traffic Management: Supports URL-based routing, host-based routing, and complex rewrites.
• High Performance: Optimized for high traffic loads and low latency.
• TLS/SSL Support: Handles HTTPS traffic efficiently, supporting multiple certificates.

In KFD, you can choose between a single instance or a double instance of NGINX Ingress Controller. With the single instance, all Ingresses will be exposed through the same controller. If you choose to have the dual configuration, KFD will create two classes: internal, which is intended to be used to expose traffic for Ops teams, and external, which is intended to be used by end users.

The single instance will expose the TCP/31080 and TCP/31443 ports on nodes where it will be installed.

The double instance will expose the following ports on nodes where they will be installed:

• TCP/31080 and TCP/31443 ports for the internal class
• TCP/32080 and TCP/32443 ports for the external class

cert-manager

cert-manager, a CNCF open-source software, provides resources to manage X.509 certificates natively and automatically in Kubernetes. It can be configured to issue certificates from both well-known, public Issuers as well as private ones. It also manages the renewal of the issued certificates.

NGINX Ingress Controller can automatically provision certificates for its Ingress resources using one of the Issuers provided by cert-manager in a Kubernetes cluster. Using these two technologies together helps exposing applications to be accessed from outside a Kubernetes cluster in a more efficient way, managing the certificates for each route independently and automatically.

KFD will install cert-manager regardless of the choice you make about the Ingress Controller, because it uses it to also create certificates used by Kubernetes.

You can also specify your own configuration for the ClusterIssuer resources, which will be responsible for the certificates used by all Ingresses.

external-dns

external-dns automatically manages DNS records from Kubernetes. Paired with NGINX Ingress Controller, it provides a powerful combination that can automate the lifecycle of DNS records for your Ingresses.

Currently, KFD will use external-dns only when installed using the EKSCluster provider to automate the creation and deletion of Route53 records.

Forecastle

Forecastle is a control panel which dynamically discovers and provides a launchpad to access applications deployed on Kubernetes.

It will provide a simple frontend with a list of Ingresses that are available in a cluster, either auto-discovered or injected through annotations.

KFD annotates every Ingress it creates to make Forecastle show them out of the box.

Monitoring

The following Prometheus Rules are already defined for this package to enable alerting:

Parameter	Description	Severity	Interval
NginxIngressDown	This alert fires if Prometheus target discovery was not able to reach ingress-nginx-metrics in the last 15 minutes.	critical	15m
NginxIngressFailureRate	This alert fires if the failure rate (the rate of 5xx responses) measured on a time window of 2 minutes was higher than 10% in the last 10 minutes.	critical	10m
NginxIngressFailedReload	This alert fires if the ingress' configuration reload failed in the last 10 minutes.	warning	10m
NginxIngressLatencyTooHigh	This alert fires if the ingress 99th percentile latency was more than 5 seconds in the last 10 minutes.	warning	10m
NginxIngressLatencyTooHigh	This alert fires if the ingress 99th percentile latency was more than 10 seconds in the last 10 minutes.	critical	10m
NginxIngressCertificateExpiration	This alert fires if the certificate for a given host is expiring in less than 7 days.	warning
NginxIngressCertificateExpiration	This alert fires if the certificate for a given host is expiring in less than 1 day.	critical

Read more

You can find more info about these topics at the following links:

• https://kubernetes.io/docs/concepts/services-networking/ingress/
• https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
• https://letsencrypt.org/
• https://cert-manager.io/