conversations/docs/file-upload-modes.md

File Upload Modes

This document describes the different modes for handling file uploads in the Conversations application, and how to configure and use them.

Overview

The application supports two independent configuration points:

1. FILE_UPLOAD_MODE: how the frontend uploads files (frontend → storage/backend)
2. FILE_TO_LLM_MODE: how the backend provides files to the LLM (backend → LLM)

Each mode has different trade-offs in terms of security, performance, and LLM accessibility. The two settings can be combined based on your network constraints.

Configuration

Frontend upload mode (FILE_UPLOAD_MODE)

# Default: presigned URL upload (backward compatible)
FILE_UPLOAD_MODE=presigned_url

# Frontend uploads directly to backend
FILE_UPLOAD_MODE=backend_to_s3

Backend delivery mode (FILE_TO_LLM_MODE)

# Default: presigned URL mode (backward compatible)
FILE_TO_LLM_MODE=presigned_url

# Backend provides base64-encoded data URLs
FILE_TO_LLM_MODE=backend_base64

# Backend provides temporary URLs through the backend
FILE_TO_LLM_MODE=backend_temporary_url

Additional settings for backend temporary URL mode:

# Base URL to reach backend
FILE_BACKEND_URL="http://localhost:8071"

# Expiration time for temporary URLs (in seconds, default: 180 = 3 minutes)
FILE_BACKEND_TEMPORARY_URL_EXPIRATION=180

Mode Details

1. Presigned URL Mode (Default)

Frontend upload configuration: FILE_UPLOAD_MODE=presigned_url

Backend delivery configuration: FILE_TO_LLM_MODE=presigned_url

How it works:

• Frontend requests a presigned URL from the backend
• Frontend uploads the file directly to S3 using the presigned URL
• Frontend notifies the backend when upload is complete
• Backend initiates malware detection
• Backend returns presigned S3 URLs to the LLM

Advantages:

• Files don't pass through the backend server (lower bandwidth usage)
• Faster uploads for large files (direct to S3)
• S3 handles the upload, no backend load
• Backward compatible with existing frontend implementations

Disadvantages:

• S3 bucket must be accessible from the frontend
• Presigned URLs can be leaked if not handled carefully
• Frontend needs to handle S3 credentials/configuration

LLM Access:

• Images: Presigned S3 URLs with expiration (default: 3 minutes)
• Documents: Presigned S3 URLs with expiration (default: 3 minutes)

When to use:

• When frontend has direct access to S3
• When you want to minimize backend load
• When S3 is publicly accessible or accessible via VPN

2. Backend Base64 Mode

Frontend upload configuration: FILE_UPLOAD_MODE=backend_to_s3

Backend delivery configuration: FILE_TO_LLM_MODE=backend_base64

How it works:

• Frontend uploads the file directly to the backend
• Backend stores the file on S3
• Backend reads the file, encodes it as base64, and creates a data URL
• LLM receives the file as a base64-encoded data URL

Advantages:

• S3 can be private/internal (not accessible from frontend)
• Files always go through the backend for validation
• No presigned URLs to manage
• Better control over file access
• Data URLs work with all LLMs that support file content

Disadvantages:

• Backend memory usage increases (entire file loaded for base64 encoding)
• Slower for very large files (encoding overhead)
• Increased bandwidth on backend
• Data URLs can be very large in responses

LLM Access:

• Images: Base64-encoded data URLs (format: data:image/png;base64,...)
• Documents: Base64-encoded data URLs (format: data:application/pdf;base64,...)

When to use:

• When S3 is not accessible from the frontend
• When you want all file uploads to go through the backend
• When the LLM supports base64-encoded data URLs
• For smaller files (< 50MB)

3. Backend Temporary URL Mode

Frontend upload configuration: FILE_UPLOAD_MODE=backend_to_s3

Backend delivery configuration: FILE_TO_LLM_MODE=backend_temporary_url

How it works:

• Frontend uploads the file directly to the backend
• Backend stores the file on S3
• Backend generates a secure temporary access token stored in cache (TTL: 3 minutes by default)
• Backend returns a temporary URL pointing to the backend's file-stream endpoint
• LLM receives the temporary URL and accesses the file through the backend
• Backend validates the token and streams the file content from S3 to the LLM

Advantages:

• S3 can be private/internal (not accessible from frontend or LLM directly)
• Files always go through the backend for validation and access control
• LLM doesn't need direct access to S3
• Tokens expire quickly (better security than long-lived presigned URLs)
• No large data URL strings in memory or responses
• Lower backend memory usage than base64 mode
• Centralized file access control through the backend
• Good balance between security and performance

Disadvantages:

• LLM must be able to access the backend server
• File streaming goes through the backend (adds some latency)
• Time-limited access (token expires)

LLM Access:

• Images: Temporary backend URLs with format /api/v1.0/file-stream/{temporary_key}/ (token expiration: configurable, default: 3 minutes)
• Documents: Temporary backend URLs with format /api/v1.0/file-stream/{temporary_key}/ (token expiration: configurable, default: 3 minutes)

When to use:

• When S3 is not accessible from the frontend or LLM
• When you want backend control over uploads and file access
• When you want time-limited access to files with centralized control
• When you want the LLM to access files through the backend gateway
• For large files (backend streams directly from S3 without loading entirely into memory)