# GHSA-jm3v-qxmh-hxwv: Scrapy's redirects ignoring scheme-specific proxy settings

> Scrapy fails to switch proxy settings during HTTP/HTTPS redirects, potentially exposing traffic to unintended proxies if scheme-specific proxies are configured.

- **Severity:** Not scored
- **Exploited in the wild:** Not known
- **Login needed:** Not needed
- **Attackable:** Over the internet
- **Victim action needed:** Not needed
- **Fix available:** Yes
- **Published:** 14 May 2024

## What is GHSA-jm3v-qxmh-hxwv?

GHSA-jm3v-qxmh-hxwv is a vulnerability in Scrapy, published on 14 May 2024. An attacker can exploit it over the internet without logging in, with no action from the victim.

### Official description

### Impact

When using system proxy settings, which are scheme-specific (i.e. specific to `http://` or `https://` URLs), Scrapy was not accounting for scheme changes during redirects.

For example, an HTTP request would use the proxy configured for HTTP and, when redirected to an HTTPS URL, the new HTTPS request would still use the proxy configured for HTTP instead of switching to the proxy configured for HTTPS. Same the other way around.

If you have different proxy configurations for HTTP and HTTPS in your system for security reasons (e.g., maybe you don’t want one of your proxy providers to be aware of the URLs that you visit with the other one), this would be a security issue.

### Patches

Upgrade to Scrapy 2.11.2.

### Workarounds

Replace the built-in retry middlewares (`RedirectMiddleware` and `MetaRefreshMiddleware`) and the `HttpProxyMiddleware` middleware with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.

### References

This security issue was reported by @redapple at https://github.com/scrapy/scrapy/issues/767.

## Which versions are affected by GHSA-jm3v-qxmh-hxwv?

2 open source packages ship an affected version, including Scrapy and scrapy.


| Package | Ecosystem | Affected | Fixed in |
| --- | --- | --- | --- |
| `Scrapy` | PyPI | All versions | 2.11.2 |
| `scrapy` | PyPI | All versions | 2.11.2 |

## How do I fix GHSA-jm3v-qxmh-hxwv?

Upgrade Scrapy to 2.11.2 or later.

## Is GHSA-jm3v-qxmh-hxwv being exploited?

There is no known exploitation of GHSA-jm3v-qxmh-hxwv in the wild and no public exploit code.

## Fix prompt for AI coding agents

```text
Check this codebase for GHSA-jm3v-qxmh-hxwv and fix it if it is affected.

What it is: GHSA-jm3v-qxmh-hxwv is a vulnerability in Scrapy, published on 14 May 2024. An attacker can exploit it over the internet without logging in, with no action from the victim.
Exploitation: There is no known exploitation of GHSA-jm3v-qxmh-hxwv in the wild and no public exploit code.

Affected packages and fixed versions:
- Scrapy (PyPI): affected all versions; fixed in 2.11.2
- scrapy (PyPI): affected all versions; fixed in 2.11.2

Task:
1. Search the dependency manifests and lockfiles (for example package.json, package-lock.json, requirements.txt, poetry.lock, uv.lock, pom.xml, build.gradle, go.mod, Cargo.lock, Gemfile.lock) and any Dockerfiles for the affected packages, including transitive dependencies.
2. If an affected version is in use, upgrade it to the fixed version for its release line with the project's package manager, so the lockfile is regenerated. Prefer the smallest upgrade that fixes it.
3. If there is no fix, or the upgrade would break the build, do not guess: explain the risk and the safest workaround instead.
4. Run the tests and summarise what changed and why.
5. If nothing in the project is affected, say so and change nothing.

Reference: https://insights.sbomify.com/GHSA-jm3v-qxmh-hxwv/ (Markdown: https://insights.sbomify.com/GHSA-jm3v-qxmh-hxwv/index.md)
```

## Sources

- OSV: https://osv.dev/vulnerability/GHSA-jm3v-qxmh-hxwv

---
From sbomify Insights: https://insights.sbomify.com/GHSA-jm3v-qxmh-hxwv/
Summaries are generated and may contain mistakes; the official description above is the record.