Ragas 0.4.4, released November 14, 2026, reports a fix for CVE-2026-6587 in its multimodal evaluation metrics. The maintainers say it covers both the collections and legacy APIs. The issue concerns evaluation inputs that can cause the evaluation machine to retrieve internal resources or local images and send the result to a model provider.
A new release, not a new disclosure
At this article's November 14 check, the GitHub advisory still listed no patched version. It was published April 20 and last updated April 22, 2026, with affected versions 0.2.3 through 0.4.3. We are reporting the maintainer's new fix claim, not presenting the advisory as updated or claiming independent validation.
The operational catch: a score may still appear
The current metric documentation says local image access is off by default and internal-address image URLs are refused. An image that is not loaded becomes text for the model, with a log message rather than an error. Explicit opt-ins exist, but the documentation warns to use them only with trusted contexts.
That distinction matters to anyone running a dashboard that judges AI answers against retrieved documents and images. Our interpretation: a completed evaluation can conceal a change in what evidence the evaluator actually received. A number is not enough to prove that the same test ran.
Review the evaluation pipeline, not just the package
- Identify the affected path. Ask which evaluations use images and where those references originate. Keep untrusted datasets separate from internal image collections.
- Compare inputs before comparing scores. Use a small, harmless reference set. Confirm which images were loaded and read the warnings before interpreting a score change as a model-quality change.
- Do not restore access just to recover old numbers. Review each exception with the owner of the data and the evaluation host. Prefer a dedicated, minimal image collection over broad filesystem access.
- Keep network controls independent. The release notes acknowledge a limitation behind HTTP proxies: the image check cannot see the final address for names resolved only by the proxy. Review that route separately.
We have not run exploit tests or independently verified the fix. Nothing here establishes exploitation of your environment. The practical next step is to assess the update alongside the actual dataset, image policy and outbound network path.