Start with the basic Modal deployment guide if you have not run a function yet. This article covers the scaling and cost patterns specific to batch jobs.

How do you parallelize a batch in Modal?

Use a function that takes one item and a driver that maps it over the whole list. Modal lets a function spawn many parallel containers, so the wall-clock time depends on concurrency, not list length.

1. Write a process_one(item) function with the GPU decorator.
2. In a driver function, call process_one.map(items).
3. Set max_containers to control how wide it scales.
4. Collect results as they return.

A batch of 10,000 short items can finish in minutes if you let it scale wide.

How do you avoid loading the model repeatedly?

Loading a large model on every call wastes time and money. Modal keeps containers warm while a batch runs and can cache the model weights in a volume or the image cache. Load the model once at container start and reuse it for every item that container handles.

How do you keep costs predictable?

• Cap concurrency with max_containers so a big list cannot spin up hundreds of GPUs.
• Use gpu="any" to accept cheaper hardware when the model tolerates it.
• Right-size the GPU — a small model on an A100 is money burned.
• Test on 100 items before running 100,000.

How do you store and read results?

Write outputs to a Modal volume for persistence, or push to your own S3/GCS bucket. Volumes survive between runs, so a crashed batch can resume instead of restarting. For huge datasets, stream results out rather than holding everything in memory.

Modal vs. a rented GPU box

If your workload runs continuously, a rented GPU instance is cheaper and simpler. Modal wins when the job is intermittent — nightly embeddings, a weekly re-index, occasional fine-tuning. You skip provisioning, and the meter stops when the batch ends. The trade-off is cold-start latency and a platform to learn.

Key Takeaways

• Modal maps one function over a list to run batch inference in parallel.
• Cache model weights so each container loads them once.
• Cap concurrency with max_containers to control cost.
• Use volumes to persist results and resume failed batches.
• Best for intermittent jobs; a continuous workload is cheaper on a rented box.

Last updated October 2026. More at Modal.

It is aimed at data and ML developers who want a function to run somewhere fast without DevOps overhead. If you have ever waited on a Kubernetes setup just to run a batch job, Modal is the shorter path.

What is Modal?

Modal is a serverless compute platform for Python. You define an app, attach an image with your dependencies, and deploy. Modal builds the container, runs your function on demand, and scales to zero between calls so you do not pay for idle time.

How do you install and run your first function?

1. pip install modal
2. Run modal setup to authenticate in the browser.
3. Write a file that imports modal and defines an app plus one function.
4. Run it locally with modal run app.py — it executes in the cloud.
5. Deploy it persistently with modal deploy app.py.

The first run pulls and builds an image, so it takes longer than later calls.

How do you add dependencies and a GPU?

Point the image at packages and the runtime picks it up:

• modal.Image.debian_slim().pip_install("torch") for libraries.
• @app.function(gpu="A100") to request a GPU type.
• Use gpu="any" to take whatever is available at a lower cost.

GPUs bill per second of use, so a 30-second job on an A100 costs cents.

How do you expose it as a web endpoint?

Add a decorator that turns the function into an HTTP route, then deploy. Modal gives you a URL that scales from zero. This works for an API, a webhook handler, or a simple inference endpoint. You can also schedule functions to run on a cron.

What should you watch out for?

Cold starts happen when a deployed function has been idle, and a GPU cold start can add several seconds. Long-running state does not survive between calls unless you use Modal’s storage primitives like volumes and dicts. And the pricing model rewards bursty work — a function running 24/7 may be cheaper on a plain VM.

Key Takeaways

• Modal runs Python functions serverlessly with no Dockerfile or Kubernetes.
• Install with pip install modal, authenticate with modal setup.
• Request GPUs with one decorator argument; billed per second.
• Functions can be HTTP endpoints or scheduled jobs.
• Free tier gives about $30/month compute; best for bursty workloads.

Last updated October 2026. Details at Modal.