Skip to content

Feat/s3 file browser v1 - #208

Open
dklOrdix wants to merge 323 commits into
mainfrom
feat/s3-file-browser-v1
Open

dklOrdix wants to merge 323 commits into
mainfrom
feat/s3-file-browser-v1

Conversation

@dklOrdix

@dklOrdix dklOrdix commented Jun 18, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #129

@dklOrdix
dklOrdix changed the base branch from main to feat/s3-file-browser June 18, 2026 11:08
F3l1x1vo and others added 4 commits June 23, 2026 08:16
* #153 allow upload of empty files

* test: add test for uploading empty files in S3 storage provider tests

---------

Co-authored-by: dklOrdix <dkl@ordix.de>
@F3l1x1vo

Copy link
Copy Markdown
Collaborator

@copilot resolve the merge conflicts in this pull request. Pull the changes from v0 (feat/s3-file-browser) into this v1 branch.

…s3-file-browser-v1

# Conflicts:
#	package-lock.json
#	src/lib/server/storage/s3-provider.ts

Copilot AI commented Jun 24, 2026

Copy link
Copy Markdown

@copilot resolve the merge conflicts in this pull request. Pull the changes from v0 (feat/s3-file-browser) into this v1 branch.

Done. Merged origin/feat/s3-file-browser into this branch (commit d8d9d68).

Two conflicts were resolved:

  • src/lib/server/storage/s3-provider.ts: Combined v1's empty-key guard + directory expansion with v0's 1000-key chunk batching in deleteObjects.
  • package-lock.json: Kept v1's icon packages (gravity-ui, lsicon, material-icon-theme) and regenerated with npm install.

Copilot AI requested a review from F3l1x1vo June 24, 2026 06:58
@F3l1x1vo

Copy link
Copy Markdown
Collaborator

@copilot Pull the new changes from 'feat/s3-file-browser' (the parent branch) and resolve any merge conflicts that appear. No substantial commits have been made to the v1 branch (this current branch) since the last merge, so prioritize the changes of the parent branch (feat/s3-file-browser).

…s3-file-browser-v1

# Conflicts:
#	dev/setup.sh

Copilot AI commented Jun 25, 2026

Copy link
Copy Markdown

@copilot Pull the new changes from 'feat/s3-file-browser' (the parent branch) and resolve any merge conflicts that appear. No substantial commits have been made to the v1 branch (this current branch) ...

Done. Merged origin/feat/s3-file-browser (9 new commits) into this branch (commit 1a66e94).

One conflict in dev/setup.sh: the parent branch added PUBLIC_STACKABLE_COCKPIT_UPLOAD_CONCURRENCY=3 while v1 had added DATABASE_* env vars at the same location in both heredocs. Both sets of vars are kept, with the parent's UPLOAD_CONCURRENCY line coming first.

F3l1x1vo and others added 3 commits June 29, 2026 10:38
* #153 allow upload of empty files

* test: add test for uploading empty files in S3 storage provider tests

* #173 add create button for empty directories and files

* #173 fix button height

* #207 make text file preview an editor instead

* #207 add syntax highlighting, add truncation security for the editor when saving

* #207 fix unit test, fix lint

* #207 improve loading of code editor

* fix build

* fix e2e test

* #207 prevent tab closing of unsaved changes, make discard button red, fix backdrop click bug

* #207 fix locale format on buttons, limit saving file size in backend, fix creating files and directories

* #207 add comments

---------

Co-authored-by: dklOrdix <dkl@ordix.de>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* add overflow fixes, add timeout on bucket connect, add tooltip fix

* move timeout value to flags

* add comment

* update comment

* fix svelte check, unit tests and e2e tests

* fix unit and e2e test

* fix unit test

* fix lint

---------

Co-authored-by: dklOrdix <dkl@ordix.de>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…HEAD

The transfer concurrency only applied within one item, so pasting many
single files ran fully serially. Items now run through the pool, and one
shared limiter caps in-flight copies across the whole request.

copyObject takes the source size when the caller knows it and skips the
HEAD for objects up to 5 GiB; multipart copies still read the source for
the ETag pin. A vanished source is reported by its own key.

The client tracks progress per running item, since item events now
interleave and arrive in completion order.
"Check new name" always reported the name as free: the dialog no longer
got a checker and fell back to "available". The server then refused the
item with a conflict after the dialog had closed.

The plan endpoint now takes optional new names per source key and plans
those items as renamed, under the same rules as the transfer itself, so
folders without a marker object count as taken. The dialog checks a name
with a one-item plan, and also rejects names that items without a
conflict in the same paste land on. A failed check shows an error and
lets the user retry. A result that arrives after the user changed the
choice or the name is ignored. The checker is now required.
@Bohreromir Bohreromir linked an issue Oct 16, 2026 that may be closed by this pull request
Workers in a browser project shared one storageState session, so parallel
storage tests disconnected each other and deleted each other's saved
connections. A worker-scoped fixture now logs each worker in as its own
mock OIDC user (Playwright's "one account per parallel worker" pattern),
replacing the setup-* projects.

With that, run fully parallel with 4 workers locally and 2 on CI, and
replace fixed sleeps in the storage specs with condition waits.
listAllKeysProgressively claimed to prefetch the next page while a batch
was processed, but it only sent the next request after the synchronous
callback returned, so nothing overlapped. Every caller's callback is
synchronous and cheap, so real prefetching would gain nothing either.

Replace the fake pipeline with a private listPages generator that both
listAllKeysProgressively and listAllKeys iterate, and pin the empty-page,
callback-error and abort-signal behaviour with tests.
…errors

deleteObjects listed every key under each directory into memory before
deleting anything, which does not scale to directories with millions of
objects. The listing also ran outside withS3Errors, so a 403 or 404
while listing surfaced as an unmapped 500.

Delete each full chunk of 1000 keys while the listing continues, so at
most about two chunks of keys are held at once, and map every S3 error
from the operation. A listing failure now leaves the chunks deleted
before it in place; deletes were never atomic across chunks anyway.
A delete that fails part way, for example on a later listing page, may
already have removed some of the selection. The error path only showed
a toast, so deleted objects stayed visible until a manual reload.
deleteObjects no longer uses it, so it had no production callers left,
and it loads every key under a prefix into memory. Callers that need a
recursive listing use listAllKeysProgressively. Its remaining test
cases now cover the progressive listing.
S3 connections with TLS verification could only trust the CAs bundled
with Node.js, so on-prem stores with an internal CA had to disable
verification. STACKABLE_COCKPIT_STORAGE_TLS_CA_CERT, set via the Helm
values storage.tls.caCert or storage.tls.secretClass, now names a CA
certificate that replaces the bundled CAs for every verified S3
connection, like the existing Trino and database CA settings.
…characters

The rename modal reused the conflict dialog's message key. Rename both
invalid-name keys into the storage_name_* family and mention control
characters, which the name policy already rejects.
An empty or missing folder showed "This bucket is empty". Only the bucket
root says that now; folders, including folders inside archives, say
"This folder is empty".
Garage's ListBuckets returns buckets in no particular order, so the
sidebar and the bucket overview changed order between visits. The
buckets API now sorts the list, and buckets added by hand are sorted
in.
Create → New directory / New text file opened the name popover with
focus left on the page, so typing, Enter and Escape did nothing until
the field was clicked. The field now gets focus with the proposed name
selected (without its extension for files), as in the rename dialog.
The titles "Files already exist" and "Cannot be transferred" were plain
paragraphs and the dialogs had no accessible name, so screen readers
announced them as unnamed dialogs. The refusal dialog also rendered an
empty paragraph. The title is now a heading that names the dialog, the
description describes it, and an empty description is left out.
Opening a browse link for another storage host (e.g. an old link from a
different connection) showed SvelteKit's bare "500 Internal Error": the
client-side redirect thrown during hydration broke the page. The load
now checks the host on the server and the client and answers 404 with
"You are not connected to the storage …", translated on the error page.
A link such as …/acme-analytics/sales%2F2026 showed an empty bucket with
no breadcrumb. The page compared the raw URL path with the path it built
from the route parameters (…/sales/2026), so it never treated the
listing as its own. Paths are now compared decoded, which also covers
characters that browsers leave unencoded, such as "@" and ":".
A folder deleted from one tab stayed open in other tabs as an empty
folder. Opening such a tab added the folder to Recent Locations again,
and "Restore tabs" brought it back. Deleting folders now closes the
tabs that show them or anything inside them, also in the saved tab
state. The active tab stays open and moves to the parent folder.
Deleting gave no feedback when it succeeded; renaming already shows
"Renamed to …". A successful delete now shows "Deleted "<name>"." for a
single item and "Deleted N items." for several.
Pressing Back while an upload ran left the page and stopped the upload
without telling the user. Navigating away during an upload now asks
first: "Stay" keeps uploading, "Stop upload and leave" aborts the
running requests and continues the navigation. Closing the tab or
reloading shows the browser's own confirmation.

A cancelled navigation no longer leaves the active tab waiting for it,
which kept the listing on its spinner after staying.
Bohreromir
Bohreromir previously approved these changes Oct 16, 2026
@Bohreromir Bohreromir closed this Oct 16, 2026
@Bohreromir Bohreromir reopened this Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallelize tests Filebrowser: Drop Upload on Explorer Epic: S3 File Browser v1

6 participants