Allow for arbitrarily long file selections in batch download #124

New issue

Open

opened 2026-06-11 10:10:01 +00:00 by aaperis · 0 comments

aaperis commented

2026-06-11 10:10:01 +00:00

(Migrated from github.com)

Please describe the feature

As a user I want to select as many files as I want to batch download as a TAR bundle so that I have a better UX and dont have to manually select batches of files.

Related to #116

Detailed description

The current GET /api/datasets/{datasetId}/download.tar?fileIds=...
endpoint caps selections at 200 files (MAX_TAR_SELECTION) because
common reverse proxies reject request lines over ~8 KB. The cap
is purely about URL length; TAR streaming itself scales fine.

One idea is to switch to a POST-then-GET selection token but there could be simpler ways to do this.

Do not passs fileIDs as URL parameters in the request
If using POST is unavoidable for this, use signed tokens or any security measure available.

## Please describe the feature As a user I want to select as many files as I want to batch download as a TAR bundle so that I have a better UX and dont have to manually select batches of files. Related to #116 ## Detailed description The current `GET /api/datasets/{datasetId}/download.tar?fileIds=...` endpoint caps selections at 200 files (`MAX_TAR_SELECTION`) because common reverse proxies reject request lines over ~8 KB. The cap is purely about URL length; TAR streaming itself scales fine. One idea is to switch to a POST-then-GET selection token but there could be simpler ways to do this. AC - [ ] Do not passs `fileID`s as URL parameters in the request - [ ] If using `POST` is unavoidable for this, use signed tokens or any security measure available.