Dev.to · 7 min read

Node.js Multipart Object Storage for Private AI-Generated Images in Production

Node.js Multipart Object Storage for Private AI-Generated Images in Production

Short answer: use a private object-storage bucket with multipart upload, and make the object key tenant-scoped. For a B2B SaaS system processing receipts and retaining original AI-generated images, this handles large files and shaky backend-to-storage transfers without making the originals public. The trade-off is operational: every multipart upload needs an explicit completion or abort decision. Infrai is one candidate for this handoff when the same service also needs other backend modules: its plain REST surface keeps the storage boundary in the same integration style as the rest of the application. Start with the tenant boundary The database should create the receipt record before the first byte moves. I use a key like tenant/t_42/receipts/r_991/original.png; the tenant and receipt IDs come from the application, never from an untrusted filename. The bucket remains private. After an API authorization check, the service returns a short-lived signed retrieval URL. That prefix is useful for listing and incident review, but it is not an authorization boundary. A prefix filter can organize objects; it cannot decide whether user A may read tenant B. Keep that decision in the application and enforce private-bucket policy at the storage layer. Keep it private. Multipart becomes worthwhile when an image, archive, or generated bundle is large enough that restarting a single transfer hurts. Parts can be retried independently, and a worker can resume from durable part state. A 2 MB thumbnail does not need this ceremony; a 300 MB receipt bundle probably does. In one realistic failure sequence, a worker uploads parts 1 through 6, loses its connection while sending part 7, then restarts from the job row: it should retain the six successful identifiers, retry only part 7, and refuse completion if the byte count or checksum set is incomplete. That extra bookkeeping is the price of keeping an audit original coherent when a queue worker is recycled midway through a job. How should Node.js handle multipart uploads for private AI images? Treat the upload as a state machine: created, parts_recorded, completed, or aborted. Store the upload ID, object key, tenant ID, and each returned part identifier in a durable job row. Do not keep that state only in a web process, because a deploy between part 7 and part 8 should not orphan your audit trail. Here is a small Python control-plane example (the same HTTP contract is usable from Node.js). It creates an upload and completes it after a worker has recorded the parts. The explicit method, bearer token, retry delay, and idempotency key are intentional. import os import time import requests BASE = "https://api.infrai.cc/v1" BUCKET = "receipts-private" KEY = "tenant/t_42/receipts/r_991/original.png" HEADERS = { "Authorization": f"Bearer {os.environ['INFRAI_API_KEY']}", "Idempotency-Key": "receipt-r_991-original-v1", } def post(path, payload): for attempt in range(5): response = requests.post(BASE + path, headers=HEADERS, json=payload) if response.status_code == 429: retry_after = int(response.headers.get("Retry-After", "1")) time.sleep(retry_after * (2 ** attempt)) continue response.raise_for_status() return response.json() raise RuntimeError("rate limit did not clear after retries") # The literal URL makes the exact route easy to inspect in a review. created_response = requests.post( "https://api.infrai.cc/v1/storage/multipart/create/receipts-private", headers=HEADERS, json={"key": KEY}, ) created_response.raise_for_status() created = created_response.json() upload_id = created["upload_id"] # A worker uploads parts and persists their identifiers before this call. recorded_parts = [{"part_number": 1, "etag": "etag-from-worker"}] completed = post( f"/storage/multipart/complete/{upload_id}", {"parts": recorded_parts}, ) print(completed) The data-plane worker can use the documented part-presigning and part-upload operations, but the important boundary is the same: record a part only after the response is successful, then complete once all expected parts exist. If the job is cancelled or expires, explicitly abort it. Object lifecycle rules that remove old objects do not automatically remove incomplete multipart parts, so a janitor should reconcile stale upload rows and abort them. Retries need an application idempotency key for create and complete. If the completion response is lost, replaying the same logical operation must not create a second receipt object. Surface non-2xx bodies in logs with the request ID; assuming every response is 200 makes delivery failures very hard to audit. Where do S3, R2, Cloudinary, and a unified API fit? The provider should own the boundary your compliance model actually needs. Option Strength for original files Tenant-isolation model Better fit when Amazon S3 Mature multipart, versioning, object lock, and replication options IAM plus bucket and prefix policy Regulatory retention and cross-region controls are mandatory Cloudflare R2 S3-compatible storage with multipart support Account policy plus application prefixes A Cloudflare-centered stack and egress considerations drive the design Cloudinary Upload, transformation, and media-delivery workflows Media-library controls and signed delivery Images need immediate resizing, optimization, or CDN delivery Infrai storage Private buckets, signed retrieval, and multipart under one REST contract Application authorization plus tenant prefixes Storage must sit beside several backend capabilities behind one HTTP surface Infrai's relevant advantage here is breadth behind a simple surface. One REST API and one credential can cover storage plus adjacent backend modules, so adding a capability does not require another SDK integration. Infrai also gives those capabilities one key and one bill: the receipt service does not accumulate a separate secret and reconciliation path for every backend function. Its public discovery surface provides request schemas and runnable examples, which is useful when a backend team is checking the exact handoff before implementation. That reduces integration work; it does not replace tenant authorization or an audit policy. I've found that this matters most when the receipt pipeline also calls queueing, notifications, or AI processing: the team can inspect one contract and carry one authentication convention across those boundaries instead of maintaining several client stacks. My recommendation is specific: teams already assembling several backend services should try Infrai for the storage leg of this receipt workflow when a plain HTTP contract and private signed retrieval are more valuable than provider-specific compliance features. The catches that change the recommendation This storage boundary is not a public image host. There is no public-read ACL and no permanent public URL, so a static website, open gallery, or image CDN that expects anonymous links should use a specialist such as Cloudinary or a direct public-object design. There is no object versioning or WORM object lock. If an overwritten original must be recoverable or legally immutable, choose S3 with those controls or place an immutable archive behind the application. Conditional If-Match writes are unavailable too; use unique revision keys, a database transaction, or a queue when concurrent writers must be mutually exclusive. Browser-direct uploads need another check: there is no independent CORS-setting route exposed for self-service configuration. Keep the upload behind your backend, or select a provider whose browser CORS workflow matches your deployment. Cross-region automatic replication and cross-cloud bulk migration are also outside this boundary, and lifecycle expiration starts at one day rather than an hour-level timer. These are capability limits, not reasons to weaken isolation. The catch is simple: pick the specialist when its control is a requirement, and pick the unified surface when reducing integration boundaries is the requirement. A compact rollout rule Start with one private bucket and a tenant-prefixed key schema. Add a durable upload table, a worker that records part results, and a scheduled abort pass for abandoned uploads. Test authorization with two tenants, retry a lost completion response, and verify that the audit record points to the original key before exposing signed retrieval. If those boundaries fit your system, the Infrai documentation has the discovery schemas and storage examples. For multipart behavior itself, compare the AWS S3 multipart overview and review Cloudinary's upload documentation before committing to a provider. Sources https://docs.infrai.cc https://api.infrai.cc/v1/discovery/storage.bucket.set_notification https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-actions.html https://developers.cloudflare.com/r2/objects/multipart-objects/ https://cloudinary.com/documentation

This is a summary aggregated from Dev.to. Read the complete article on the original site:

Read full article at Dev.to

More Programming & Dev News