Your product already uses NestJS for users, permissions, and APIs. Your model training code uses Python. Serving the model does not require choosing one language for everything: expose inference through a small internal service and keep product decisions in the public API.
This design introduces a network boundary, so it is useful when independent deployment, Python dependencies, or separate scaling justify that cost. For a tiny application, a simpler single-runtime deployment may be easier to operate. The goal is a clear contract, not an impressive number of services.
Follow one prediction
The example scores numeric URL features. It does not fetch a submitted URL, and it does not claim that these three features form an effective phishing detector. Keeping the example small makes the service contract easier to inspect.
Validate
NestJS authenticates the caller, authorizes the operation, and validates a versioned request. It enforces limits before forwarding any inference work.
1. Version the input contract
1{2 "request": {3 "schemaVersion": "url-features-v1",4 "urlLength": 132,5 "digitCount": 8,6 "hasIpHost": false7 },8 "response": {9 "modelVersion": "url-risk-2026-10-01",10 "score": 0.7311 }12}
Define feature order, units, valid ranges, missing-value behavior, and the positive class. “Length” could mean bytes or characters; make the choice explicit in the feature specification. A model trained with one interpretation must not silently receive another. Give the fitted preprocessing and model a shared versioned artifact.
NestJS can validate public request DTOs using its ValidationPipe. Configure the policy for unexpected properties deliberately. Validation is separate from authentication, authorization, request-size limits, and rate limits; the application still needs those controls.
Load once, predict many times
FastAPI’s lifespan mechanism lets you load shared resources before serving requests and release them during shutdown. Its request-body support uses Pydantic models for validation. The sketch below assumes Pydantic v2 and a trusted, previously fitted binary classifier whose positive label is 1.
1from contextlib import asynccontextmanager2from typing import Literal3import joblib4from fastapi import FastAPI5from pydantic import BaseModel, ConfigDict, Field67MODEL_VERSION = "url-risk-2026-10-01"89@asynccontextmanager10async def lifespan(app: FastAPI):11 # Artifact and dependency versions are pinned by the deployment.12 app.state.pipeline = joblib.load("/models/url-risk.joblib")13 yield14 del app.state.pipeline1516app = FastAPI(lifespan=lifespan)1718class Features(BaseModel):19 model_config = ConfigDict(extra="forbid", strict=True)20 schemaVersion: Literal["url-features-v1"]21 urlLength: int = Field(ge=1, le=8192)22 digitCount: int = Field(ge=0, le=8192)23 hasIpHost: bool2425class Prediction(BaseModel):26 modelVersion: str27 score: float = Field(ge=0, le=1, allow_inf_nan=False)2829@app.post("/v1/predict", response_model=Prediction)30def predict(features: Features):31 row = [[features.urlLength, features.digitCount,32 int(features.hasIpHost)]]33 pipeline = app.state.pipeline34 positive = list(pipeline.classes_).index(1)35 score = float(pipeline.predict_proba(row)[0, positive])36 return Prediction(modelVersion=MODEL_VERSION, score=score)
Deploy this only behind an authenticated internal boundary; service-to-service identity and network policy are omitted from the sketch. Validate cross-field rules too, such as a digit count no larger than URL length. The feature limits are illustrative contract choices, not universal URL rules.
Startup failure should keep the instance out of readiness. Run a small compatibility prediction before declaring it ready, including the expected feature schema and output class. Each server worker normally loads its own copy, so account for model memory when choosing the process count.
Make the network call bounded
Put the client behind an injectable provider so controllers do not duplicate networking logic. NestJS documents its configurable HTTP client; the illustrative provider below uses Node’s built-in fetch. The configured URL must come from trusted deployment configuration, never a user-supplied target.
1import { Injectable, ServiceUnavailableException } from '@nestjs/common';23type Features = {4 schemaVersion: 'url-features-v1';5 urlLength: number;6 digitCount: number;7 hasIpHost: boolean;8};910function validPrediction(value: unknown): value is {11 modelVersion: string; score: number;12} {13 if (typeof value !== 'object' || value === null) return false;14 const item = value as Record<string, unknown>;15 return item.modelVersion === 'url-risk-2026-10-01'16 && typeof item.score === 'number'17 && Number.isFinite(item.score)18 && item.score >= 0 && item.score <= 1;19}2021@Injectable()22export class InferenceService {23 async predict(features: Features, requestId: string) {24 try {25 // Validate required config during application startup.26 const response = await fetch(process.env.INFERENCE_PREDICT_URL!, {27 method: 'POST',28 redirect: 'error',29 headers: {30 'content-type': 'application/json',31 'x-request-id': requestId,32 // Supply service identity through your deployment's auth mechanism.33 },34 body: JSON.stringify(features),35 signal: AbortSignal.timeout(1500),36 });37 if (!response.ok) throw new Error('Inference returned an error');38 const result: unknown = await response.json();39 if (!validPrediction(result)) throw new Error('Invalid inference result');40 return result;41 } catch {42 // Emit a redacted, correlated failure event in the real provider.43 throw new ServiceUnavailableException('Prediction is unavailable');44 }45 }46}
This provider deliberately returns an unavailable error for dependency failures. In a complete implementation, classify upstream validation failures separately, cap response size, and log a redacted failure category. The TypeScript request type helps developers, but the public route and Python endpoint still need runtime validation.
Keep failure and capacity visible
- Budget time: the internal timeout must fit inside the public request’s deadline. A client abort does not guarantee that Python has stopped computing; use concurrency limits and bounded queues.
- Retry sparingly: prediction should be free of business side effects, but repeated inference still consumes capacity. Retry only appropriate transient failures and keep a shared time budget.
- Separate score from policy: a score is model output. The product’s warning or blocking threshold should be evaluated, versioned, and monitored as its own policy.
- Observe versions: attach a request ID and model version to logs and metrics. Track latency, unavailable responses, invalid results, and missing-feature rates without logging sensitive raw URLs.
For expensive batch inference, a queue and an asynchronous job status may fit better than a long HTTP request. For small synchronous predictions, begin with the bounded service call, load test realistic concurrency, and scale from observed bottlenecks.
Verify a known prediction, an invalid input, a wrong model version, a slow response, and a model-load failure. The success criterion is both a correct prediction path and an honest failure path: users should be able to tell the difference between “low risk” and “the detector could not run.”