VAITP Dataset

← Back to the dataset

CVE-2026-55520

Exponential regex backtracking in robots.txt parser leads to DoS.

  • CVSS 7.1
  • CWE-400
  • Resource Management
  • Remote

Protego is a pure-Python robots.txt parser with support for modern conventions. Prior to 0.6.2, protego._urlpattern._URLPattern._prepare_pattern_for_regex translates every asterisk in an Allow or Disallow directive into a lazy regular-expression wildcard, so a directive containing many asterisks creates exponential backtracking. After protego.Protego.parse processes a crafted robots.txt file, protego.Protego.can_fetch can spend an attacker-controlled period matching a near-miss URL and deny service to the crawler. The vulnerable path is src/protego/_urlpattern.py in the _URLPattern match logic. This issue is fixed in version 0.6.2.

CVSS base score
7.1
Published
2026-08-28
OWASP
A04 Insecure Design
Orthogonal defect classification
Timing/Serialization
Code defect classification
Timing Issues
Category
Resource Management
Subcategory
Resource Exhaustion
Accessibility scope
Remote
Impact
Denial of Service (DoS)
Affected component
Protego
Fixed by upgrading
Yes

Solution

Upgrade protego to version 0.6.2 or later.

Vulnerable code sample

import re

class _URLPattern:
    def __init__(self, pattern: str):
        self.pattern = pattern
        self.regex = re.compile(self._prepare_pattern_for_regex(pattern))

    # VULNERABLE: unbounded wildcard leads to exponential backtracking
    def _prepare_pattern_for_regex(self, pattern: str) -> str:
        # Replace every asterisk with a lazy wildcard
        return '^' + pattern.replace('*', '.*?') + '$'

    def matches(self, url: str) -> bool:
        return bool(self.regex.match(url))

Patched code sample

import re

class _URLPattern:
    def __init__(self, pattern: str):
        self.pattern = pattern
        self.regex = re.compile(self._prepare_pattern_for_regex(pattern))

    # FIX: escape literals and limit wildcard to non‑greedy safe form
    def _prepare_pattern_for_regex(self, pattern: str) -> str:
        # Escape all regex metacharacters, then safely restore wildcard handling
        escaped = re.escape(pattern)
        # Replace escaped asterisks with a non‑greedy wildcard that cannot cause backtracking
        safe = escaped.replace(r'\*', '.*?')
        return '^' + safe + '$'

    def matches(self, url: str) -> bool:
        return bool(self.regex.match(url))

Published fix

Public sources link this CVE to the changes below. The code samples above were generated to illustrate the defect; these links lead to the code the project actually changed.

  • Merge commit from fork (scrapy/protego)
    src/protego/_urlpattern.py, src/protego/_urlpattern.py · _URLPattern.match, src/protego/_urlpattern.py · _URLPattern.__init__

Cite this entry

@misc{vaitp:cve202655520,
  title        = {{Exponential regex backtracking in robots.txt parser leads to DoS.}},
  author       = {Bogaerts, Fr\'ed\'eric and Ivaki, Naghmeh and Fonseca, Jos\'e},
  year         = {2026},
  note         = {VAITP Python Vulnerability Dataset, entry CVE-2026-55520},
  howpublished = {\url{https://netpack.pt/vaitp/vulnerability/CVE-2026-55520/}}
}

Download this entry as JSON Download the whole dataset

Introducing the "VAITP dataset": a specialized repository of Python vulnerabilities and patches, meticulously compiled for the use of the security research community. As Python's prominence grows, understanding and addressing potential security vulnerabilities become crucial. Crafted by and for the cybersecurity community, this dataset offers a valuable resource for researchers, analysts, and developers to analyze and mitigate the security risks associated with Python. Through the comprehensive exploration of vulnerabilities and corresponding patches, the VAITP dataset fosters a safer and more resilient Python ecosystem, encouraging collaborative advancements in programming security.

The supreme art of war is to subdue the enemy without fighting.

Sun Tzu – “The Art of War”

:: Shaping the future through research and ingenuity ::

Before you use the VAITP AI models

Please read and accept the following disclaimer:

By choosing "I understand and accept", you acknowledge that you understand and agree to these terms.