Verlex documentation

One flag controls your price-speed tradeoff:

# Performance mode, fastest start
result = verlex.cloud(my_function, api_key="gw_your_key", fast=True)

# Standard mode (default), lowest price
result = verlex.cloud(my_function, api_key="gw_your_key")

Performance mode (fast=True) requires the Performance plan ($10/mo). Standard mode (fast=False) is available on every plan. Both modes share one queue: Performance speeds up machine selection and launch, not queue position.

Two parts: a monthly plan, then a per-job cost. Verlex passes the provider's live hardware price through at cost and adds a fixed service fee per GPU-hour, prorated per second. There is no percentage markup on the hardware. The Standard plan is free forever ($0/month); the Performance plan ($10/month) unlocks fast mode and 300 GB of storage.

The service-fee tier is derived from the hardware's FP16 tensor performance (TFLOPS):

The service fee is charged per GPU-hour. There is no cold-start surcharge; a cold machine and a warm one cost the same.

• Per-second billing, 1 second minimum: you pay only for the time your job actually runs. Both the provider cost and the service fee are prorated to the second.
• Prepaid credits: buy credits in the dashboard (minimum top-up $10). Each job places a hold and settles the exact amount when it finishes.
• Auto top-up (opt-in): adds credits automatically with your saved card when the balance runs low (default: $50 added when it drops below $10).
• Funds-based execution: there is no runtime cap by default. A job runs until it finishes, your credits are exhausted, or it shows no CPU, GPU or output activity for 30 minutes. The optional timeout argument is the total time from submit to result, including queueing and provisioning; when it passes, the SDK cancels the job.

Every job is priced across 11 clouds (AWS, GCP, Azure, Verda, RunPod, Vast.ai, JarvisLabs, Hyperstack, TensorDock, Lyceum, Lambda Labs) plus serverless container lanes for small jobs, and routed to the cheapest machine that fits.

import verlex

# Default: priced across 11 clouds, routed to the cheapest machine that fits
result = verlex.cloud(train_model, gpu="A100", api_key="gw_your_key")

# Provider list: cost comparison and failover stay inside your subset
result = verlex.cloud(
    train_model,
    gpu="A100",
    provider=["runpod", "gcp", "verda"],
    api_key="gw_your_key",
)

# Hard pin: one provider, no cross-provider failover if it is out of stock
result = verlex.cloud(train_model, gpu="A100", provider="aws", api_key="gw_your_key")

Pinning: a single pinned provider disables cross-provider failover: if that cloud has no capacity, the job fails instead of moving. Prefer a provider list, which keeps cost comparison and failover inside your chosen subset.

Serverless: small jobs are routed to serverless container lanes (billed per second) when that is cheaper than a VM, or faster within budget on the Performance plan. This happens automatically; nothing to configure.

What happens between verlex.cloud(...) and the result. Numbers are the server's defaults.

Send local files up to the VM before your job runs and bring generated files back when it finishes. Pass a Workspace to any cloud call. Auto mode mirrors your project root (respecting .gitignore) and returns whatever your code creates or changes; or list explicit Upload and Output specs when you want full control over what goes up and what comes back.

import verlex
from verlex import Workspace

def train():
    from pathlib import Path
    Path("runs").mkdir(exist_ok=True)
    Path("runs/loss.json").write_text('{"loss": 0.42}')
    Path("model.pt").write_text("weights")
    return {"status": "done"}

# Auto mode mirrors your project root (respecting .gitignore) up to the VM,
# then returns new and changed files to the same paths locally
result = verlex.cloud(train, workspace=Workspace(), gpu="A100", api_key="gw_your_key")

# ./runs/loss.json and ./model.pt now exist locally

import verlex
from verlex import Workspace, Upload, Output

def finetune():
    import json
    from pathlib import Path
    cfg = json.load(open("config.json"))  # uploaded and renamed below
    Path("runs").mkdir(exist_ok=True)
    Path("runs/epoch1.ckpt").write_text("weights")  # trained on ./dataset
    Path("model.pt").write_text("final weights")
    return cfg["name"]

ws = Workspace(
    root=None,  # implied once uploads are listed; spelled out here for clarity
    uploads=[
        Upload("./configs/run.json", to="config.json"),  # rename into the workspace
        Upload("./dataset"),  # whole folder, hierarchy kept
    ],
    outputs=[
        Output(path="model.pt"),  # bring a single file back
        Output(pattern="runs/*.ckpt"),  # plus files matched by glob
    ],
)

result = verlex.cloud(finetune, workspace=ws, gpu="A100", api_key="gw_your_key")

Your function runs inside the synced workspace, so relative paths like open('data/train.csv') work unchanged. Listing explicit Upload specs replaces the auto mirror — you get exactly the files you named, and nothing else; pass root='auto' if you want the project mirror alongside them. Explicit Output specs never affect what goes up. There is no hard size limit by default: Verlex warns once the workspace passes 100 MB, so set a ceiling with Workspace(max_size_bytes=...) if you need one. Import Workspace, Upload, and Output from the top-level verlex package.

Verlex scans your function's source, pins the versions of imported packages installed locally, and bundles local .py modules automatically. When you need certainty, pass pip_packages=: those packages are authoritative, always installed, and a failed install fails the job loudly instead of continuing silently.

import verlex

def train():
    import numpy as np
    import torch
    from my_helpers import preprocess  # local .py modules are bundled automatically
    # ... your training code ...
    return {"loss": 0.03}

# Auto-detect: imported packages are pinned to your local versions
result = verlex.cloud(train, api_key="gw_your_key")

# Authoritative installs: always installed, a failed install fails the job
result = verlex.cloud(
    train,
    api_key="gw_your_key",
    pip_packages=["torch==2.3.1", "numpy==1.26.4"],
    python_version="3.11",  # 3.10, 3.11, 3.12, or 3.13
)

python_version controls the interpreter on the cloud side: supported values are 3.10, 3.11, 3.12, and 3.13. By default Verlex matches your local interpreter.

Kill the cold start before it happens. Call verlex.prewarm(your_function) and Verlex analyzes it, boots the right hardware, and hands you back a handle once the machine is ready (pass wait=False to get the handle immediately and let it boot in the background while your code runs). Calling the handle runs your function on that machine with no provisioning wait, and the machine stays attached to the handle between calls, so every call is warm. Release it with handle.release() or a with block when you are done; unreleased handles release themselves when your program exits, and a hard kill is covered by the server-side idle timeout.

import verlex

def train(batch):
    import torch
    ...

wf = verlex.prewarm(train, gpu="A100", api_key="gw_your_key")  # A100 starts booting NOW

data = load_and_clean()      # local preprocessing runs meanwhile

result = wf(data)            # runs train(data) on the warm machine
more = wf(other_data)        # still warm, the VM stays attached

# Define inputs/outputs with a Workspace, just like verlex.cloud —
# it's a per-call argument on the warm handle:
from verlex import Workspace, Upload, Output
out = wf(data, workspace=Workspace(
    uploads=[Upload("./configs/run.json", to="config.json")],  # sync inputs up
    outputs=[Output(pattern="checkpoints/*.ckpt")],            # bring results back
))

wf.release()                 # recommended, not required: frees the machine now.
                             # If you skip it, the handle auto-releases when your
                             # program exits (atexit). A hard kill that skips atexit
                             # is covered by the server-side idle timeout instead.

# Or auto-release with a context manager:
with verlex.prewarm(train, gpu="A100") as wf:
    result = wf(data)

# Scope mode: prewarm() with no function sizes hardware from THIS file's
# code and binds no function. You pass the function to run as the first
# argument on every call (workspace= stays a keyword, so it comes last):
wf = verlex.prewarm(gpu="A10")          # no function -> scope mode
result = wf(train, data)                # function is the first call arg
out = wf(train, data, workspace=Workspace())  # same, with a Workspace
wf.release()

There are two ways to call prewarm, and they change how you invoke the handle:

• workspace= works the same as on any other cloud call: it syncs inputs up before the function runs and brings generated outputs back. Because it is a keyword, its position never changes, and it is per-call, so different calls on the same warm VM can carry different Workspaces.
• Multi-GPU cluster jobs, spot jobs, and workloads that resolve to the serverless lane skip pre-warming; they are already routed optimally.
• If the warm session expires before you call, the handle still works and simply provisions normally.

Billing: you pay only for the time the warm machine actually exists at the real instance rate, never for analysis time, failed warm-ups, or skipped sessions. Idle time between calls is billed in segments; job runtime is billed as a normal job.

The machine is released by whichever of these comes first:

Don't want to manage when code runs in the cloud? Let Verlex decide automatically. When your system's CPU, memory, or GPU usage exceeds the threshold, heavy functions are transparently offloaded to the cheapest cloud provider.

import verlex

# Pass your API key directly
verlex.overflow(api_key="gw_your_key")

# Your code runs normally.
# When CPU, memory, or GPU exceeds 85%, functions go to the cloud.
data = load_data()
result = train_model(data)   # system overloaded? → cloud
evaluate(result)             # resources free → runs locally

Parameters

Defaults: With just an API key, overflow monitors CPU, memory, and GPU. If any of them exceeds 85% usage, functions are automatically offloaded to the cloud in standard mode.

Note: GPU monitoring uses pynvml (included with the base install) and needs an NVIDIA GPU. Without one, overflow gracefully falls back to CPU and memory monitoring only.

CPU and memory monitoring need psutil. Install with: pip install 'verlex[overflow]'

Use verlex.session() when a model or service should load once and handle several streaming turns on the same warm VM. The function starts serve_loop(), and each session.send() yields response chunks as they arrive.

import verlex
from verlex import serve_loop

def serve():
    model = load_model()  # loaded once per VM

    def reply(prompt, history):
        return model.stream(prompt)  # str or an iterable of chunks

    serve_loop(reply)

with verlex.session(
    serve,
    gpu="T4",
    idle_timeout=90,
    token_timeout=300,
) as session:
    for token in session.send("Explain gradient checkpointing"):
        print(token, end="", flush=True)

Use it as a context manager. The VM stops and billing ends when the block exits or idle_timeout passes without input. ready_timeout limits the initial boot and model-load wait; token_timeout is a stall timeout that resets whenever the handler produces output.

Interactive sessions are experimental. verlex chat provides the equivalent terminal workflow. For independent function calls on one warm machine, use prewarm() instead.

Connect Claude, Codex, Gemini CLI — or any MCP-compatible assistant — to your Verlex account and drive it in plain language: "run this fine-tune on an A100, show me the log as it goes, and kill it if the loss blows up".

Install

pip install "verlex[mcp]>=0.17.0"

# Storing your key once here means the configs below
# can carry no secret at all.
verlex login

Connect your assistant

Claude Code

claude mcp add verlex -- verlex-mcp

# View only: no submitting, cancelling or spending
claude mcp add verlex -- verlex-mcp --read-only

Claude Desktop

Add this to claude_desktop_config.json — %APPDATA%\Claude\ on Windows, ~/Library/Application Support/Claude/ on macOS, ~/.config/Claude/ on Linux.

{
  "mcpServers": {
    "verlex": {
      "command": "verlex-mcp",
      "env": { "VERLEX_MCP_MAX_SPEND": "25" }
    }
  }
}

Codex

[mcp_servers.verlex]
command = "verlex-mcp"
env = { VERLEX_MCP_MAX_SPEND = "25" }

Gemini CLI

{
  "mcpServers": {
    "verlex": {
      "command": "verlex-mcp",
      "timeout": 60000
    }
  }
}

Keep any client request timeout at 60 seconds or more: log tailing holds a request open for up to 25 seconds while it waits for new output.

What you can ask

Once connected, your assistant can do all of this on its own:

• →"What GPUs can I get, and what would an A100 cost me for two hours?"
• →"Run this script on an A100 and show me the output as it goes."
• →"Cancel that job if you see the loss go to NaN."
• →"List my last 10 jobs and what each one cost."
• →"How much have I spent this month, and how much credit is left?"

Safety

The connector is a thin client holding your own API key, so it can only ever see and do what you can. There is no database access and no shared credential. On top of that:

• Anything that spends money or cancels work needs an explicit confirmation step, so it always surfaces as a visible action rather than a silent side effect.
• Every submission is priced first, and the estimate comes back with the job so you see the cost up front.
• A per-session spend cap (default $25), a one-GPU limit and a 60-minute duration limit are enforced before any request is sent.
• Job output is fenced and labelled as untrusted data, so instructions hidden inside a log cannot redirect your assistant.

The verlex command covers the same ground as the Python SDK. verlex --help and verlex <command> --help list every flag.

verlex login                              # save your API key (used by the SDK, CLI and MCP)
verlex run train.py --gpu A100            # run a script
verlex run train.py --gpu H100 --gpu-count 8 --provider runpod --provider gcp
verlex run train.py --spot --pip torch==2.3.1 --python 3.11 --fast
verlex jobs                               # recent jobs with full ids and cost
verlex status <job id>                    # status, machine, cost, error
verlex logs <job id> --follow             # stream output
verlex cancel <job id>
verlex chat serve.py --gpu T4 --interactive
verlex deploy create MODEL --idle-timeout 1800
verlex deploy list                        # also: status, logs, stop, delete
verlex pool list                          # team data pool; verlex env for environments
verlex mcp serve                          # connect an AI assistant

Verlex raises specific exceptions so you can handle failures gracefully:

from verlex import (
    VerlexError,              # Base class for all Verlex errors
    ConfigurationError,       # Bad input or option, raised before submit (also a ValueError)
    AuthenticationError,      # Missing, invalid, frozen or paused key
    InsufficientCreditsError, # Not enough credits, or no payment method
    GPUUnavailableError,      # Pinned hardware has no capacity right now
    ProviderMaintenanceError, # Pinned provider is under maintenance
    JobFailedError,           # Your code or the platform failed the job
    JobCancelledError,        # The job was cancelled elsewhere
    JobTimeoutError,          # Your timeout passed
    OutputDownloadError,      # A required output could not be downloaded
    SerializationError,       # Function or result could not be (de)serialized
    NetworkError,             # Connection to Verlex API failed
    RateLimitError,           # Too many requests
)

import verlex

try:
    result = verlex.cloud(fit, args=[data], gpu="A100")
except ValueError as e:              # bad input, or a ValueError raised by your code
    print(f"Bad input or ValueError on the machine: {e}")
except InsufficientCreditsError as e:
    print(f"Need about {e.required:.2f} USD, have {e.available:.2f} USD")
except JobFailedError as e:
    print(f"Job {e.job_id} failed ({e.remote_type}). Run: verlex logs {e.job_id}")
except JobTimeoutError:
    print("Took longer than timeout= (which includes queueing and provisioning)")
except VerlexError as e:
    print(f"Something went wrong: {e}")

The errors people hit most, and what to do. Every error, its cause and whether you were charged is listed in docs/ERRORS.md in the repository.

Request a GPU by name with gpu="H100", and several GPUs on one machine with gpu_count=8. Verlex finds the cheapest available instance across 11 clouds, including AWS, GCP, Azure, Verda, RunPod, Vast.ai, JarvisLabs, Hyperstack, TensorDock, Lyceum, and Lambda Labs, plus serverless container lanes for small jobs.

If you don't specify a GPU, Verlex auto-detects your code's needs and picks the best option. Availability changes continuously; GET /v1/hardware/gpu-types is the source of truth for the currently schedulable public catalog and live price ranges.

Job inputs, outputs, workspace files, and checkpoints live in Verlex object storage and count against your plan's storage quota:

Workspace upload blobs are stored by content hash and expire automatically after a period of non-use, so re-uploads of unchanged files are fast.

Spot capacity is 60-80% cheaper than on-demand, but the provider can reclaim it at any time. Verlex handles the recovery: anything your job writes into the checkpoint directory is synced to object storage while it runs (every 60 seconds, every 20 seconds on short-notice providers, and immediately when a preemption notice arrives).

When a spot instance is reclaimed, the job is automatically re-enqueued on the next cheapest spot capacity (a different provider whenever one is stocked), the checkpoint directory is restored before your code runs again, and the reclaimed time is not billed. If spot keeps getting reclaimed, or no spot capacity fits the requested hardware anywhere, the job finishes on on-demand capacity so it always completes.

Write checkpoints, resume on restart

import os, pickle

def train():
    # Set inside every Verlex job; synced to object storage while you run.
    ckpt_dir = os.environ.get("GATEWAY_CHECKPOINT_DIR", "./ckpt")
    state_file = os.path.join(ckpt_dir, "state.pkl")

    start = 0
    if os.path.exists(state_file):
        with open(state_file, "rb") as f:
            start = pickle.load(f)["epoch"]

    for epoch in range(start, 100):
        run_one_epoch()
        tmp = state_file + ".tmp"
        with open(tmp, "wb") as f:
            pickle.dump({"epoch": epoch + 1}, f)
        os.replace(tmp, state_file)  # atomic: never syncs a torn file

The same restore also runs when an on-demand instance dies and the job is retried on fresh hardware. On providers with no advance reclaim notice, progress since the last periodic sync (at most the sync interval) can be lost.

Point Verlex at a model on HuggingFace and get back an always-on HTTPS endpoint. The GPU is chosen from the model's own metadata, and Analyze reports the card, the disk and the hourly price before anything is provisioned — deploying is a separate, deliberate step, so the first time you learn what an endpoint costs is never the invoice.

Two kinds of endpoint

Decoder language models are served by a pinned vLLM behind an OpenAI-compatible API, so an existing OpenAI client works by changing base_url and nothing else. Tool calling works too: when a model's own chat template declares a tool-call format Verlex recognises, tools and tool_calls come back in the OpenAI shape, so agent frameworks run unmodified. It is detected from the template rather than guessed from the model family, and left off when the format is ambiguous; pass tool_call_parser to set it yourself, or "none" to turn it off. A model too new for the default vLLM release runs on a newer one instead — the analysis names which, and vllm_version lets you pin one yourself. Set max_replicas to let one endpoint grow onto more GPUs under load and shrink back when it is quiet; each replica is billed like the first machine. Every other supported task — embeddings, text and token classification, speech recognition, image classification, object detection, text-to-image and text-to-video — is served by a FastAPI app Verlex generates from a validated spec. Those expose POST /predict, and GET /schema describes exactly what that route accepts, so the URL you are handed documents itself. Video is the exception to answering in one call: a clip renders for minutes, so POST /predict answers at once with a job id and GET /result?id=… returns the MP4 once it is ready — poll it; it answers 202 with progress until then.

From Python or the CLI

import verlex

endpoint = verlex.deploy("Qwen/Qwen2.5-0.5B-Instruct", idle_timeout=1800)
print(endpoint.chat("Hello!")["choices"][0]["message"]["content"])
endpoint.stop()

# Same from the CLI:
#   verlex deploy create Qwen/Qwen2.5-0.5B-Instruct --idle-timeout 1800
#   verlex deploy stop <deployment id>

A drop-in OpenAI base URL

from openai import OpenAI

# The endpoint the dashboard hands you, plus your ordinary Verlex key.
client = OpenAI(
    base_url="https://api.verlex.dev/v1/deployments/dep_.../openai/v1",
    api_key="gw_your_key",
)

answer = client.chat.completions.create(
    model="my-endpoint",
    messages=[{"role": "user", "content": "Explain spot instances."}],
)
print(answer.choices[0].message.content)

Bring your own app.py

For a model no built-in task fits, pass app_source with your own FastAPI app (and requirements, if it needs any) and it runs in place of the generated one — same machine sizing, same endpoint, same billing. A generated deployment's Source panel has a Use as custom app button, so the usual way in is to start from what Verlex generated and edit it.

• Define a module-level app: the server is started as uvicorn app:app.
• Serve GET /health, returning 200 once the model is loaded — it is how the machine reports ready. Return 503 with {"status": "failed"} if loading fails, and the deployment stops in seconds instead of waiting out the startup deadline.
• Serve POST /predict. It is the only inference route forwarded to a custom app; GET /schema is forwarded too, and worth serving so the URL describes itself.
• Read the model from the VERLEX_MODEL_REF environment variable rather than hardcoding one, so it always matches what the GPU was sized for.
• Requirements come from the same allowlist as generated apps. Anything off it is refused by name at create, not skipped — a skipped import would only fail later, on a machine already billing.
• The source is checked before any machine starts: it must parse, and it runs through the same acceptable-use scan as every job.

A gated repository needs a HuggingFace token when the deployment is created. It is used once to download the weights and is never stored, never logged and never returned by the API, so a replacement machine needs it supplied again.

Corporate accounts get organization-backed teams: a shared org credit pool, email invitations, and roles (owner, admin, member).

Admins allocate credits from the pool to members and can claw back unused allocations; members can also contribute personal credits to the pool. Everything is managed from the dashboard.

Set daily or monthly spending caps from your dashboard to prevent unexpected charges. When a limit is reached, Verlex can block new jobs, warn you, or send a notification.

Configure spending limits in your dashboard settings. You can also enable auto top-up to automatically add credits when your balance gets low.