How to add AI features to an existing SaaS without a rewrite

Step 1: pick the first feature with two questions

Score candidate features on two axes: how much value a correct output creates, and whether the data the model needs is already in your system and accessible. The first feature should score high on both.

"Users can see and correct the result" matters more than it looks. A feature where a person reviews the AI output before it is used is far more forgiving than one that acts on its own — and it generates the labelled examples you will use to evaluate and improve it.

Step 2: four architecture defaults

1. A provider-agnostic model layer

Put model calls behind one small interface in your codebase, with the model chosen by configuration. Providers, prices and model quality change every few months; switching should be a config change plus an evaluation run, not a refactor. SDKs such as the Vercel AI SDK or LangChain already give you this abstraction across Anthropic, OpenAI and Google models.

2. Structured output, validated against a schema

If software consumes the result, never parse free text. Ask for output that matches a schema and validate it. In the Vercel AI SDK this is generateText with an Output.object schema; the parsed, validated value comes back as output, and a response that cannot be parsed into the schema throws a NoObjectGeneratedError you can handle explicitly:

import { generateText, NoObjectGeneratedError, Output } from 'ai';
import { z } from 'zod';

const Invoice = z.object({
  vendor: z.string(),
  invoiceNumber: z.string(),
  dueDate: z.string().describe('ISO 8601 date'),
  total: z.number(),
  currency: z.string().length(3),
});

export async function extractInvoice(text: string, model: string) {
  try {
    const { output } = await generateText({
      model, // e.g. from config, so the provider is swappable
      output: Output.object({ schema: Invoice }),
      prompt: 'Extract the invoice fields from this document:\n\n' + text,
    });
    return { ok: true as const, invoice: output };
  } catch (error) {
    if (NoObjectGeneratedError.isInstance(error)) {
      // Route to a human review queue instead of saving bad data.
      return { ok: false as const, reason: 'unparseable', raw: error.text };
    }
    throw error;
  }
}

We used the same principle in an AI assessment app we built: assessment templates are generated as schema-constrained JSON, so the app never has to guess whether the model "probably" returned valid structure.

3. Streaming where users wait, background jobs everywhere else

Interactive features (an assistant, a draft the user watches being written) should stream. Everything else — extraction, classification, report generation — belongs in a background job with retries, so model latency and provider outages never block a web request. Your existing queue is usually fine.

4. Ship behind a feature flag

Roll out to internal users, then a few friendly customers, then everyone. Capture a thumbs-up/down or an edit signal on every output from day one; it is the cheapest evaluation data you will ever get.

Step 3: get multi-tenancy right before launch

Most AI tutorials assume one user and one dataset. A SaaS has many tenants, and the AI layer must respect the same boundaries as the rest of the product:

• Isolation in retrieval. If the feature searches customer content, filter by tenant inside the search query itself — never retrieve broadly and filter afterwards. Storing embeddings in PostgreSQL with pgvector keeps vectors next to your existing tenant and permission columns.
• No cross-tenant prompts. Prompt templates, few-shot examples and caches must never include another customer's data.
• Per-tenant usage metering. Record tokens, model and feature for every call, tagged with tenant and user. You need it for pricing, for abuse limits and to answer "why did our bill double?".
• Per-tenant limits and kill switches. One customer's bulk import should not exhaust your rate limits or budget for everyone else.

Step 4: know exactly what data goes where

Your customers' security questionnaires will ask which data reaches which AI provider. Answer it on purpose:

• Send the minimum. Redact or drop fields the model does not need. It improves accuracy and cost as well as privacy.
• Use paid API tiers and read their data terms. As of October 2026: OpenAI states that data sent to its API is not used to train its models unless you opt in, and that abuse-monitoring logs are kept for up to 30 days, with zero data retention available to approved customers. Anthropic states it does not use inputs or outputs from its commercial products for training by default. Google's Gemini API terms say prompts and responses from paid services are not used to improve its products, while unpaid-tier data may be used and reviewed by people.
• Document it. Keep a short data-flow note per AI feature: fields sent, provider, retention, region. Update your subprocessor list and privacy policy.

Step 5: put evaluation in CI

Every prompt edit or model upgrade can make the feature better on the example you were looking at and worse on ten you weren't. Treat AI output like any other behaviour under test:

1. Collect 30–100 real inputs with the output a domain expert considers correct.
2. Score automatically: exact checks for structured fields, and an LLM-as-judge rubric for free text — but only after checking the judge's verdicts against human labels, so you know when it can be trusted.
3. Run the suite in CI on every change to prompts, models or retrieval, and block merges that regress.
4. Add every real-world failure your users report to the suite.

Mistakes we see when teams add AI themselves

• Starting with the most impressive feature instead of the most valuable one. A general assistant demos well; extraction or classification usually pays back first.
• Putting raw tables in the prompt. Models do better with less, well-labelled context — the relevant records, not the whole table. It is cheaper too.
• Trusting a single happy-path test. A prompt that works on the example you wrote it against tells you very little. An evaluation set of real inputs tells you a lot.
• Hard-coding one provider everywhere. When prices drop or a better model ships, you want to switch in an afternoon.
• Forgetting the failure path. Timeouts, rate limits and refusals happen in production. Decide what the user sees when they do, and make sure the rest of the product keeps working.
• No owner after launch. Model behaviour drifts as providers update models and your data changes. Someone needs to watch the feedback signal and the evaluation results.

A rollout checklist

None of this requires a new platform or a rewrite. It does require engineers who treat AI as production software. That is what we do in our AI integration service — inside your codebase, following your conventions — and if you would rather have an engineer join your team directly, you can hire AI developers from us. For the SaaS foundations around it, see SaaS development.