Jump to content

Versity S3 Gateway on PMSS

From Pulsed Media Wiki


The Versity S3 Gateway (versitygw) is a lightweight S3 server that puts an S3 API in front of an ordinary POSIX filesystem — and it maps object keys straight onto file paths, with no proprietary layout. Store an object at mybucket/reports/jan.csv and it lands on disk at exactly .../mybucket/reports/jan.csv. On a Pulsed Media storage box or seedbox it runs from a single Go binary; in a test it held about 31 MB of RAM — the lightest of the self-hosted S3 options measured here. It is Apache-2.0 licensed and stateless. The keys are yours and the data stays in Pulsed Media's Finnish datacenters.

versitygw on a Pulsed Media storage box: bucket create, upload, list, and read-back over the S3 API — and the object key demobucket/hello.txt mapped straight to that path on disk. The server held about 31 MB of RAM.

When versitygw is the right tool

Reach for versitygw when the transparency of storage matters:

  • You already have files on disk (or another process writes them) and want to expose that same tree over S3 — because keys map 1:1 to paths, an S3 GET and a local cat read the same file.
  • You want no lock-in: stop the gateway and your data is still plain files in a plain directory, reachable over SFTP and SSH exactly as before.
  • You want the lightest full S3 gateway — in the test here it used about 31 MB, less than rclone serve s3 (about 51 MB) — while still supporting a fuller S3 feature set than rclone's experimental server.

For the simplest one-command endpoint, rclone serve s3 is the classic pick. For a console, versioning, and policies, use MinIO. For a store optimised for billions of small files, SeaweedFS. versitygw's niche is the direct, transparent POSIX mapping.

What one process needs

versitygw is a single Go binary — download it, run it, no root required. In a test on a Pulsed Media storage box (versitygw 1.7.0, Debian 12) the running gateway held about 31 MB of resident memory while serving a bucket. Because it is stateless and stores nothing of its own — the POSIX filesystem is the store — the disk cost is exactly your files, and there is no database to grow.

Verified behaviour

On a Pulsed Media storage box, versitygw was exercised end to end: download the binary, start it with the posix backend on a non-default port bound to localhost, then over the S3 API create a bucket, upload an object, list it, and read it back byte for byte. The uploaded object demobucket/hello.txt appeared on disk at ~/vgw-data/demobucket/hello.txt — the S3 key is the file path, exactly as designed. The server held about 31 MB of RAM. The screenshot shows that session.

Setting it up

Download the latest Linux_x86_64 release from the versitygw GitHub releases page and unpack the binary into ~/bin. Then start it with the POSIX backend. Pick your own port — not a default — and bind to localhost, a VPN, or a known client IP:

mkdir -p ~/vgw-data
ROOT_ACCESS_KEY=YOUR-KEY ROOT_SECRET_KEY=YOUR-SECRET \
  ~/bin/versitygw --port 127.0.0.1:PORT posix ~/vgw-data

Point any S3 client at it:

mc alias set vg http://127.0.0.1:PORT YOUR-KEY YOUR-SECRET
mc mb vg/mybucket
mc cp file.txt vg/mybucket/
# file.txt is now at ~/vgw-data/mybucket/file.txt

A stable hostname

Point S3 clients at your service permalink<your-service-label>.mcx.fi:PORT — so the endpoint keeps working when Pulsed Media migrates your service.

Security

  • Set a strong, random root access key and secret — versitygw requires them, so there is no anonymous-by-default footgun.
  • Do not use a default port. Pick your own, and bind to localhost, a VPN, or a known client IP unless you need public access.
  • Because keys map to paths, keep the backend directory private: chmod 700 ~/vgw-data.
  • versitygw moves and stores bytes; it does not encrypt them for you. Combine it with client-side encryption for sensitive data.

See also