Automating Vast.ai GPU Rentals: A Script That Spins Up and Tears Down a vLLM Box

If you have ever rented a GPU, you know the annoying part is not the GPU. It is the plumbing around it: searching offers, comparing hosts, creating an instance, waiting for SSH, installing the inference server, and then remembering to destroy the box before the hourly billing eats the money you saved. vast.ai is a GPU marketplace that makes the renting itself cheap and instant. This post is about the plumbing: a single bash script that takes a vast.ai box from search to a working vLLM endpoint, then tears it down when you are done.

The script I use is vast-qwen.sh, a single bash file that needs nothing but curl, jq, and ssh (no vastai CLI, no pip, no Python). It searches the cheapest matching 2x RTX 3090 offers, rents one, waits for SSH and the model endpoint, tunnels the port to your laptop, and destroys the instance when you are finished. The box ends up serving Qwen3.8-27B-FP8 through vLLM as a plain OpenAI-compatible API on localhost:8080, which opencode, aider, LiteLLM, and curl can all talk to.

Before the how, the honest when. My advice these days: unless you already own a GPU, prefer a subscription. ChatGPT, Claude, and friends are cheaper than buying specific GPUs, cheaper than specialized hardware like DGX Sparks, and cheaper than renting for steady everyday use. Renting a GPU on vast.ai makes sense when you need a real GPU for a short, specific job, when you need several GPUs at once, or when you want full control of the serving stack. Keep that trade-off in mind, because it decides whether this script saves you money or spends it.

The short version

• vast.ai is a GPU marketplace with per-hour billing. You rent by the hour from verified hosts, and you stop paying the moment you destroy the instance.
• The script wraps the whole lifecycle in one file: search, create, list, wait, ssh, tunnel, logs, destroy, and a one-shot up that does create + wait + prints the tunnel command.
• It talks to the vast.ai REST API directly with curl, so there is no vastai CLI to install and no Python environment to babysit.
• The result is an OpenAI-compatible endpoint on localhost:8080, so any OpenAI-shaped client works: curl, opencode, aider, LiteLLM.
• The honest cost rule: renting beats buying when the GPU is the point and the job is short; a flat subscription beats renting for steady everyday use.

What vast.ai actually is

vast.ai is a marketplace where GPU owners publish idle hardware and you rent it by the hour. Think of it as a short-term rental market for datacenter GPUs: hosts publish machines with their GPUs, disk, CUDA versions, and prices, and you search, rent, and get an SSH-accessible box running a Docker image of your choice. Two pricing modes matter here: on-demand (fixed price per hour, the default this script uses) and interruptible (cheaper bids that a host can reclaim). Most offers include the NVIDIA driver and let you specify any Docker image; the host handles power, cooling, and uptime.

Because supply is distributed across hundreds of hosts, prices for the same GPU vary from hour to hour. That is exactly why the script searches instead of hardcoding a host: it asks the API for the cheapest verified offer that matches your GPU, count, disk, and CUDA requirements.

Why rent a GPU at all (and when not to)

Renting makes sense when the workload genuinely needs a GPU and does not need it forever. Fine-tuning runs, batch inference jobs, rendering, serving a specific open model with a specific quantization, or a multi-GPU experiment that would sit idle on your desk for weeks. Renting gives you a specific GPU (or eight of them) for the duration of a job, root access to configure it exactly how you want, and no depreciation, no power bill, and no maintenance.

The counterweight is real. For steady everyday use, a flat subscription is almost always cheaper, because you pay one predictable fee whether you use it lightly or heavily, and the provider eats the idle time. Buying a specific GPU or a DGX Spark only pays off if it stays busy, because the hardware cost is sunk whether it runs or not. My rule of thumb: subscribe for the assistant workload, rent for the GPU workload, and buy only when the GPU workload is constant.

The script: vast-qwen.sh

The gist is one bash file with a small set of subcommands, all sharing the same auth and the same CONFIG block at the top. Everything is configurable through environment variables, so you can point it at a different GPU, model, or port without editing the file.

$ ./vast-qwen.sh search
# list the 15 cheapest matching 2x RTX 3090 offers
$ ./vast-qwen.sh create 123456
# rent that offer, launch vLLM, print the instance id
$ ./vast-qwen.sh up
# one shot: pick the cheapest offer, create, wait, print the tunnel command
$ ./vast-qwen.sh list
$ ./vast-qwen.sh wait <instance_id>
$ ./vast-qwen.sh ssh <instance_id>
$ ./vast-qwen.sh tunnel <instance_id>
$ ./vast-qwen.sh logs <instance_id>
$ ./vast-qwen.sh destroy <instance_id>   # stop billing, delete the instance

opencode integration is opt-in on purpose: up and create never touch your local client config, because not everyone runs opencode. Run opencode-config yourself when you want it.

How it works under the hood

The script is a thin wrapper over three vast.ai REST endpoints, which you can also call directly. Auth is a single API key from cloud.vast.ai/manage-keys, sent as a Bearer token; the script reads VAST_API_KEY or falls back to a .vast_api_key file. Nothing else is needed.

Searching offers is a POST to /api/v0/bundles/ with a JSON filter:

curl -sS -H "Authorization: Bearer $VAST_API_KEY" -H "Content-Type: application/json" \
  -d '{"gpu_name": {"eq": "RTX 3090"}, "num_gpus": {"eq": 2}, "disk_space": {"gte": 60}, "cuda_max_good": {"gte": 12.4}, "rentable": {"eq": true}, "verified": {"eq": true}, "type": "on-demand"}' \
  https://console.vast.ai/api/v0/bundles/

The script sorts the results by dph_total (dollars per hour total) and prints the top 15, or takes the first id when you call create without an offer id.

Renting is a PUT to /api/v0/asks/{offer_id}/ with the image, disk, label, environment, and an onstart script:

curl -sS -X PUT -H "Authorization: Bearer $VAST_API_KEY" -H "Content-Type: application/json" \
  -d '{"image": "vllm/vllm-openai:v0.27.1", "disk": 70, "label": "qwen38-27b-fp8-dual", "env": "-p 8080:8080", "onstart": "...", "runtype": "ssh", "target_state": "running"}' \
  https://console.vast.ai/api/v0/asks/123456/

The response returns new_contract, the instance id. The onstart field is a shell script the box runs on first boot; here it launches vLLM in the background with a retry loop. Then GET /api/v0/instances/{id}/ is polled until ssh_host and ssh_port appear, and wait ssh-checks /v1/models until vLLM answers.

Destroying is a single DELETE on /api/v0/instances/{id}/. That is the only call that stops the per-hour billing, which is why the script prints it loudly and why up prints the matching destroy command at the end.

Hands-on: from search to a working endpoint

Prerequisites: curl, jq, and ssh on your machine (all already present on macOS and most Linux boxes), a vast.ai account, and an API key exported as VAST_API_KEY. Optionally export HF_TOKEN if the model requires gated access, and SSH_IDENTITY if your key is not the default.

$ export VAST_API_KEY=xxxx
$ ./vast-qwen.sh search
OFFER_ID     PRICE       GPUS          DISK  CUDA  LOCATION
<offer_id>   $<x>/hr    2x RTX 3090   82GB  12.4  <region>
# rent the cheapest match in one shot, or pass an offer id
$ ./vast-qwen.sh up
instance created: <instance_id>
vLLM ready.
ssh -p <port> root@<host> -L 8080:localhost:8080

$ ./vast-qwen.sh tunnel <instance_id> &
$ curl -s http://localhost:8080/v1/models | jq '.data[].id'
qwen3.8-27b-fp8
$ curl -s http://localhost:8080/v1/chat/completions \n  -H "Content-Type: application/json" \n  -d '{"model": "qwen3.8-27b-fp8", "messages": [{"role": "user", "content": "Hello!"}]}' | jq '.choices[0].message.content'
# when the job is done, stop the meter
$ ./vast-qwen.sh destroy <instance_id>

What runs on the box: vLLM serving Qwen3.8-27B-FP8

The interesting part is the onstart script. On first boot the box runs a small shell loop that retries vLLM up to five times and writes everything to /var/log/vllm.log, so a transient model download or CUDA init failure does not kill the session; it just retries after a 15 second pause, and logs shows you what happened.

export HF_HUB_DISABLE_XET=1
nohup bash -c 'for i in $(seq 1 5); do
  vllm serve Qwen/Qwen3.8-27B-FP8 \
    --tensor-parallel-size 2 \
    --host 0.0.0.0 --port 8080 \
    --served-model-name qwen3.8-27b-fp8 \
    --max-model-len 120000 --max-num-seqs 160 \
    --gpu-memory-utilization 0.92 \
    --enable-auto-tool-choice --tool-call-parser qwen3_coder \
    --reasoning-parser qwen3 --trust-remote-code
  echo "vllm exited (attempt $i), retrying in 15s..." >&2
  sleep 15
done' > /var/log/vllm.log 2>&1 &

Three flags are worth understanding. --tensor-parallel-size 2 splits the model across both RTX 3090s (24 GB VRAM each), which is why the script insists on num_gpus=2. --served-model-name decouples the API model name from the Hugging Face repo id, so clients request qwen3.8-27b-fp8 instead of the full path. --enable-auto-tool-choice with --tool-call-parser qwen3_coder and --reasoning-parser qwen3 turns on native tool calling and reasoning output for the model, which is what makes it usable from agent clients. One caveat worth knowing: a mismatched tool-call parser does not error loudly, it silently breaks tool calling — if you swap in a different model, re-check the parser flag against vLLM's current parser list rather than trusting this one.

The full CONFIG block is overridable through environment variables, so you can swap the model, image, port, or memory utilization without touching the file. The defaults that matter:

Using the endpoint: opencode, aider, LiteLLM, or plain curl

Once the tunnel is up, localhost:8080/v1 is a drop-in OpenAI-compatible API. Point any OpenAI-shaped client at it with a dummy key. opencode-config writes a provider named vast-qwen into ~/.config/opencode/opencode.json (base URL http://localhost:8080/v1, model qwen3.8-27b-fp8) so opencode can pick it up:

$ ./vast-qwen.sh opencode-config
wrote ~/.config/opencode/opencode.json
run: opencode -> /connect -> Other -> id 'vast-qwen' (dummy key) -> /models

aider and LiteLLM just need the same base URL. For any client the pattern is identical: point base_url at http://localhost:8080/v1, use any dummy API key, and set the model name to qwen3.8-27b-fp8.

Costs, honesty, and when to skip the whole thing

Let me be blunt about the money, because this is where most GPU tutorials go quiet. If you do not already have a GPU and your workload is steady, a subscription is cheaper than renting, and renting is cheaper than buying for almost everyone. Specialized hardware such as a DGX Spark only wins when the GPU is busy near-constantly; otherwise the purchase price amortizes over idle weeks. The reason this script exists is not that renting is cheap, it is that renting is the only option that scales to zero: you pay per hour, and destroy ends the bill instantly.

Concrete case: you want to serve an open model with your own quantization and tool-calling settings, or run a fine-tune that needs 2x 24 GB. That is a GPU workload, and renting for the weekend beats buying a box you will not keep busy. But if the answer to "what are you using it for" is "the same assistant tasks I do every day", use the subscription and keep the money.

The real cost of renting is not the hourly rate, it is the forgotten instance. The script is built around that failure mode: up prints the destroy command at the end, and destroy is a single command that ends billing for good. Get into the habit of tearing the box down when the job is done, and the hourly math works in your favor.

Official sources

• The gist, vast-qwen.sh: https://gist.github.com/MyWay/164746b52dfbec326a47a0589c4b1dec
• vast.ai API Hello World (search and create): https://docs.vast.ai/api-reference/hello-world
• vast.ai destroy instance: https://docs.vast.ai/api-reference/instances/destroy-instance
• vast.ai managing instances guide: https://docs.vast.ai/guides/instances/manage-instances
• vLLM tool calling: https://docs.vllm.ai/en/latest/features/tool_calling/
• Qwen serving with vLLM: https://qwen.readthedocs.io/en/latest/deployment/vllm.html
• Our local AI tooling comparison: https://systhoughts.com/posts/lm-studio-vs-ollama-vs-anythingllm-vs-unsloth-studio
• Our AI agent token field guide: https://systhoughts.com/posts/reduce-token-usage-ai-coding-agents

Are you renting GPUs on demand, or has the subscription won your workload? What does your spin-up and tear-down workflow look like? Drop it in the comments.

Until next time, keep your systems thoughtful.