Recently a deadly combination of vulnerabilities emerged, posing a severe threat to Kubernetes clusters utilizing Ingress-Nginx. By exploiting three critical vulnerabilities:
- CVE-2022-4886: the path field sanitization can be bypassed and ingress-nginx credentials can be obtained
- CVE-2023-5044: Code injection via ‘permanent-redirect’ annotation
- CVE-2023-5043: Ingress nginx annotation injection causes arbitrary command execution
attackers can execute arbitrary code and escalate privileges, all with access to the Nginx Annotation object. These vulnerabilities have been confirmed in both NGINX and Kubernetes/Ingress-Nginx, as reported by Google and various GitHub issues.
Ingress-Nginx Controller: A Cornerstone of Kubernetes Networking
The Ingress-Nginx controller is a component in Kubernetes, leveraging NGINX as a reverse proxy and load balancer. This controller streamlines the management of incoming traffic, directing it to the right services in your cluster. Ingress-Nginx enables secure communication. With the power of NGINX, Ingress-Nginx offers a strong foundation for managing ingress traffic in your Kubernetes setup.
Annotations: Customizing Ingress-Nginx Behavior
Annotations in Kubernetes are digital markers that add metadata to objects. They enhance configuration management and aid in ensuring seamless communication between application components. Annotations in the Ingress-Nginx controller allow you to customize its behavior. These notes, embedded in your configuration, act as guidelines that help the controller manage incoming web traffic and set security rules. While annotations offer clear benefits, they can also be difficult to secure, with misconfigurations posing significant security risks.
Nginx CVE Review
- CVE-2023-5044 and CVE-2023-5043: These vulnerabilities allow attackers to steal sensitive credentials from the cluster by manipulating the Ingress object configuration. By default, these secrets include highly privileged credentials for the Kubernetes API server. The combination of these vulnerabilities enables attackers to inject arbitrary code into the Ingress controller process using the
permanent-redirect(CVE-2023-5044) annotation fields. This injected code grants access to the Ingress controller’s service account token, associated with a ClusterRole allowing access to all Kubernetes secrets within the cluster.
- CVE-2022-4886: This vulnerability exposes a critical flaw in the way Ingress-Nginx handles the
pathfield within Ingress routing definitions. When an attacker manipulates this vulnerable
pathfield, they can redirect requests to an internal file containing the service account token, a client credential used for authentication against the Kubernetes API server.
- By crafting a malicious
pathparameter, an attacker can orchestrate a route that points to the internal file housing the service account token.
- Once the attacker gains control of the Ingress object, they can exploit this misconfiguration to steal the Kubernetes API credentials from the Ingress controller. With these credentials in hand, the attacker gains unauthorized access to all secrets within the cluster.
- By crafting a malicious
Upwind’s research team has discovered numerous instances where both NGINX and Ingress-Nginx configurations possessed permissions to extract sensitive secrets, posing a substantial risk to Upwind customers. In response, our research team meticulously analyzed these vulnerabilities and proactively notified Upwind users, ensuring they were informed and equipped to safeguard their systems against potential exploits.
Nginx Attack Path Scenarios
- In multi-tenant clusters, attackers exploit Ingress object vulnerabilities, compromising the entire cluster through escalated privileges.
- Compromised supply chains and tainted configurations allow malicious actors to inject code, endangering users deploying these setups.
- Tainted online examples from untrusted sources create deceptive entry points, necessitating heightened security in Kubernetes environments. Using configuration samples from the web or ChatGPT can expose users to tainted setups, elevating the risk of exploitation.
- Malicious Helm charts in supply chains enable unauthorized access, especially in multi-tenant clusters, where attackers manipulate Ingress objects within their namespaces.
- Insiders with configuration change rights but no direct cluster access pose threats, emphasizing the importance of securing internal permissions effectively.
How To Fix Nginx CVEs
This deadly combination requires immediate action. You can remediate Nginx CVEs by doing the following:
- If you are using the out-of-the-box Kubernetes Ingress-Nginx controller, upgrade to version 1.9 or above to patch all vulnerabilities.
- If you are utilizing Helm charts, update to version 4.8.3 to ensure protection against all three vulnerabilities.
- For users employing pure NGINX, upgrading to version 1.19 or above is essential to safeguard against potential attacks.
Additionally, In the ingress-nginx controller deployment consider using the following flag in the Args section to mitigate vulnerabilities:
apiVersion: v1 kind: Pod metadata: labels: ... name: ingress-nginx-controller namespace: ingress-nginx spec: containers: - args: - /nginx-ingress-controller ... ### Append the flag inside the args section ### - -–enable-annotation-validation ### ...
If your rule includes
Prefix enable the
strict-validate-path-type option from nginx-ingress-controller version 1.18 and above.
Upwind’s Response to Nginx Vulnerabilities
Upon learning about the Nginx vulnerabilities, Upwind’s research team was able to rapidly research and identify where these toxic combinations exist in our customers’ environments and notify them for immediate remediation.
Going beyond the research team, the Upwind platform also identifies toxic combinations like Nginx using the following:
- Vulnerability funnel: prioritize vulnerabilities by using context such as if they are in-use, actively loaded in memory or exposed to the Internet
- Topology map: automatically see which resources are receiving Internet ingress or egress communication and identify any abnormal resource communication
- Inventory management: easily search for image names or versions to rapidly identify vulnerable images in your environment and perform needed upgrades
Find Nginx Vulnerabilities
For further information on Nginx vulnerabilities or for assistance identifying critical vulnerability exposures in your environment within minutes, please ping us at [email protected].