Skip to content

Make Add-On scratch a multi-part upload #521

Open

Description

@duckduckgrayduck

Issue

Something broke during the DocumentCloud dependency update today and it's really only been working coincidentally in some cases. store_event_data which calls a PATCH to /api/addon_events/pk with a JSON blob previously was working without a limit.

This is why I started working on enforcing a limit via nginx in this PR:
#496

It turns out Django does have a native limit which is DATA_UPLOAD_MAX_MEMORY_SIZE. This is set to 2.5MB by default. This wasn't evaluated correctly in older versions of django-rest-framework for JSON blobs and was the subject of a CVE here:
https://fieldguide.lutrasecurity.com/CVE-2026-73228/

This was patched in our most recent update of django-rest-framework, which was a minor revision bump.

We do currently use scratch > 2.5MB on some Add-Ons and I initially thought up to 25MB would be ok, since it's just there for the occasional store and load call.

Resolution

Since scratch is already deferred and only loaded in the detail view, keeping it in Postgres is fine. To restore support for scratch over 2.5 MB, we can send it as a multipart file upload instead of a JSON body. The nginx 25MB will be the limit that caps this.

In app:

  • Add ScratchField to documentcloud/addons/serializers.py and declare scratch = ScratchField(required=False, help_text=_("Field to store data for add-on between events")) on AddOnEventSerializer.
  • Add tests for a multipart file PATCH over 2.5 MB and a normal JSON PATCH.
    No parser changes are needed, because the default parsers already include MultiPartParser.

In python-documentcloud:

  • Change AddOn.store_event_data() to send scratch as a real file part with a filename, for example files={"scratch": ("scratch.json", json.dumps(data), "application/json")}. A plain form field still counts against the limit.
  • Release a new version.

Activity

  1. changed the title [-]Make Add-On scratch a JSON blob created on S3[/-] [+]Make Add-On scratch a multi-part upload[/+] on Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions