Arrikto has scanned all of the images that are part of Arrikto EKF using the Grype open source vulnerability scanner to identify issues wrt the recently disclosed log4j vulnerabilities (CVE-2021-45105, CVE-2021-45046, CVE-2021-44228). If you are using EKF version 1.3.1 or earlier, then you are not vulnerable to these bugs. If you are using the latest EKF version 1.4, you may be vulnerable.
Please read below to understand which components of EKF are vulnerable, how to mitigate this, and what Arrikto is doing for the next version of EKF. We identified two set of images that use the Apache log4j library.
The first one is S3Proxy, which uses log4j as a dependency of the Apache jclouds library, and the second one is the KFServing images contained in EKF 1.4, which use log4j as part of the Ray Python library.
The version of the log4j library used in the S3Proxy software is 1.2.17, and hence not vulnerable to these newly disclosed CVEs. Nevertheless, since this is a very old version of the log4j library, we will update it to the latest version available (2.17.0) with the next version of EKF. We advise everyone running EKF on Azure to update to the latest version of EKF once it's made public. For users of EKF on AWS or GCP, no further action is needed, since we don't deploy S3Proxy on these platforms.
All of the KFServing images (except for one, see below) use Python. This means that they are not vulnerable since the log4j library bundled with Ray is never being used. In the future, we will make sure to update these images with a newer version of Ray that bundles a fixed version of log4j so that security scanners don't report these false positive issues.
Finally, there is one KFServing image, 'kfserving/pmmlserver:v0.6.1', which contains the log4j library *and* uses Java. This image implements the PMML predictor and as far as we know, it's vulnerable to the newly discovered log4j issues. To find out if you are currently serving any models using this vulnerable image and delete it, please run the following commands:
1. Identify any InferenceServices that use the vulnerable image:root@rok-tools:~# kubectl get isvc -A -ojson \
> | jq -r '.items[]
> | select((.spec.predictor | has("pmml"))
> or .spec.predictor.containers[]?.image == "kfserving/pmmlserver:v0.6.1")
> | "\(.metadata.namespace) \(.metadata.name)"'
kubeflow-user test1
kubeflow-user test2
2. Remove the InferenceServices reported by step 1:
root@rok-tools:~# kubectl delete isvc -n kubeflow-user test1
inferenceservice.serving.kubeflow.org "test1" deleted
root@rok-tools:~# kubectl delete isvc -n kubeflow-user test2
inferenceservice.serving.kubeflow.org "test2" deleted
In the next version of EKF, we are going to disable this predictor so that users will not be able to spawn inference services using a vulnerable image, until we can rebuild the image to use a fixed version of the log4j library. If you need to use this predictor in your cluster, please open a new issue and ask that we provide you an updated image.
Comments
0 comments
Please sign in to leave a comment.