How to Connect Label Studio to a Machine Learning Pipeline

October 12, 2026

Hand-labeling every image is a brick wall. The fix is pre-labeling: let a model make a first guess, have a human correct it, then retrain on the corrections. Label Studio supports this through its ML backend interface.

Quick answer: Run an ML backend server that implements /predict and /train, point Label Studio at its URL in project settings, and enable auto-annotation. Predictions appear as editable boxes.

The loop pays off fast. Correcting a rough box takes seconds; drawing one from scratch takes a minute. On a 5,000-image set that difference is weeks.

How does the ML backend work?

You stand up a small HTTP service with two endpoints. Label Studio calls /predict to pre-label imported tasks and /train after you finish a batch. The predictions come back in Label Studio’s own result JSON.

from label_studio_ml.model import LabelStudioMLBase

class MyModel(LabelStudioMLBase):
    def predict(self, tasks, **kwargs):
        preds = []
        for t in tasks:
            box = self.model.detect(t["data"]["image"])
            preds.append({"result": [{
                "from_name": "label", "to_name": "image",
                "type": "rectanglelabels",
                "value": {"x": box.x, "y": box.y,
                          "width": box.w, "height": box.h,
                          "rectanglelabels": [box.cls]}
            }]})
        return preds

Coordinates are percentages of image size, not pixels. Getting that wrong is the classic first bug — boxes land in the corner.

How do you connect it to a project?

Run the backend with label-studio-ml start ./my_backend.
In your project, open Settings, then Model.
Add a model, paste the backend URL.
Enable the model and turn on auto-annotation.

Verify with a validation call from the settings screen before importing thousands of tasks.

How do you close the retraining loop?

After annotators finish a batch, call /train or trigger it on a schedule. The backend pulls the latest labeled tasks, fine-tunes, and serves the new weights on the next /predict. Accuracy improves as the model sees corrected examples it got wrong before.

What infrastructure do you need?

Component	Small setup	Larger setup
Label Studio	Local pip install	Docker + Postgres
ML backend	Same machine	GPU worker
Storage	Local files	S3 or GCS

A handful of annotators can share one small server. Inference at batch scale wants a GPU, and that is where cost appears.

What does it cost to run?

Label Studio open source is free. The ML backend is your own code, so cost equals compute — a GPU instance is typically $0.5-$2 per hour, and a model that pre-labels in under a second per image keeps that cheap. Retraining frequency is the main lever on spend.

What goes wrong?

Coordinate systems: pixels instead of percentages puts every box in the wrong place.
Label mismatches: the class names in your model must match the XML config exactly.
Stale models: forgetting to trigger /train means predictions never improve.
Confirmed bias: if annotators accept good boxes too quickly, errors in the model’s blind spots never get labeled.

Who this suits: teams with an initial model already and a growing dataset. Who should skip it: if you are starting from zero and have no model yet, label a first batch by hand; there is nothing to pre-label with.

Quick Summary

An ML backend pre-labels tasks so humans correct instead of draw.
Implement /predict and /train and point Label Studio at the URL.
Use percentage coordinates, and match class names to the config exactly.
Retraining on corrected data is what actually improves accuracy.
Open source is free; GPU inference is the recurring cost.

Tags:

Label Studio