Request -> Gateway -> [Meter Event] -> Kafka/Redis Stream
                                            |
                                    Meter Aggregator Service
                                            |
                                    Stripe Meter API (hourly batch)
                                            |
                                    Stripe Invoice (monthly)
                                            |
                                    Customer Payment

Meter event structure: Each request generates an event containing tenant ID, request type (chat, RAG, embedding), model used, input tokens, output tokens, and timestamp. The aggregator batches these into hourly summaries and reports to Stripe.

Stripe Meters (v2 metering): Create a meter in Stripe for each billable dimension — API calls, input tokens, output tokens, storage GB. Stripe's metering API accepts events and handles aggregation, proration, and invoice line item generation. You do not build aggregation logic yourself.

Cost-plus pricing calculation: Your internal cost per 1M tokens (including GPU compute, API fees, and infrastructure overhead) is the floor. Apply a 3-5x markup for the customer price. At $0.003 per 1K tokens internal cost for GPT-4o-mini, your customer price of $0.01-$0.015 per 1K tokens yields 70-80% gross margin on inference.

Pricing Models That Work for AI Products

The tiered flat-rate with overage model works best for enterprise sales because procurement departments understand monthly line items. Pure per-token pricing works best for developer platforms where users want granular control and predictability.

Self-Hosted Models: When to Build vs Buy Inference

The build-vs-buy decision for model inference determines your cost structure, latency profile, and operational complexity:

Use hosted APIs (OpenAI, Anthropic, Google) when: - Your request volume is under 1M requests/month - You need frontier model quality (GPT-4o, Claude Opus) - You cannot justify dedicated GPU infrastructure costs - Latency requirements are above 500ms p99

Self-host with vLLM on Kubernetes when: - Request volume exceeds 1M/month (cost crossover point) - Open-source models (Llama 3.1 70B, Mistral) meet quality requirements - You need sub-200ms p99 latency - Data residency requirements prohibit sending data to external APIs - You want predictable, fixed infrastructure costs instead of variable API bills

The cost crossover math: At 10M requests/month with average 500 input + 200 output tokens per request, GPT-4o-mini via API costs approximately $7,500/month. Self-hosted Mistral 7B on three A100 GPUs via reserved instances costs approximately $5,800/month with capacity for 20M+ requests. At 10M requests, self-hosting saves 23%. At 20M requests (same infrastructure), savings reach 61%.

vLLM on Kubernetes is the production standard for self-hosted inference. Key configuration decisions: set gpu-memory-utilization to 0.90 (90% for KV cache, 10% buffer for spikes), readiness probe at 120 seconds (model weight loading time), and enforce-eager mode to avoid CUDA graph memory fragmentation in multi-tenant workloads.

Explore our Cloud Architecture Toolkits for vLLM deployment templates, Kubernetes GPU scheduling configurations, and inference cost calculators.

The Metrics That Determine Whether Your AI Product Prints Money

Traditional SaaS metrics (MRR, churn, LTV) still apply, but AI products add three critical metrics:

1. Gross Margin per Tenant

Calculate inference cost, infrastructure overhead, and support cost per tenant, then compare to their revenue. Healthy AI SaaS targets 65-75% gross margin. If a tenant's inference costs consume 60% of their subscription revenue, they are unprofitable at their current plan tier.

Action: Build a cost attribution dashboard that calculates real-time per-tenant margins. When margin drops below 50%, either upsell the tenant to a higher tier, apply intelligent caching to reduce inference calls, or route their requests to cheaper models.

2. Inference Cost per Dollar of Revenue

Track total inference spend (API fees + GPU compute + vector database) as a percentage of total revenue. Early-stage AI products often run at 40-50% cost-of-revenue. Mature products optimize to 15-25% through caching, batching, model routing, and self-hosting crossover.

The optimization loop: Semantic caching (embedding-based similarity matching of previous requests) reduces redundant inference by 15-30%. Request batching groups multiple requests into a single GPU forward pass, improving throughput 3-8x. Model downgrades for simple queries (routing "what is your return policy?" to a small model instead of GPT-4) save 90% per request with no quality impact.

3. Time to Value per Tenant

How quickly does a new tenant go from signup to their first meaningful inference result? For enterprise AI products, this includes data ingestion, document embedding, and initial RAG index building. If a tenant uploads 10,000 documents and waits 4 hours for embeddings, you have lost them. Target: first meaningful result within 15 minutes of signup.

Architecture for fast onboarding: Pre-compute embeddings during upload using a dedicated embedding queue with auto-scaling. Show partial results as soon as the first batch of documents is indexed. Use progressive enhancement — the RAG system improves as more documents are processed, but it works (with reduced coverage) immediately.

Learn to build production AI products with our free AI & ML Engineering course covering model serving, billing integration, and multi-tenant architecture patterns.

From MVP to Production: The 90-Day Playbook

Days 1-30 (Foundation): Ship a single-tenant MVP using hosted LLM APIs (Anthropic or OpenAI). No billing integration. No GPU infrastructure. Validate that customers will pay for the AI capability by charging flat monthly fees manually via Stripe invoices. Focus on model quality and user experience.

Days 31-60 (Multi-tenancy): Add tenant isolation (API keys, namespace-separated vector stores, per-tenant rate limits). Integrate Stripe metering for usage tracking. Implement the inference router with cost-aware model selection. Ship usage dashboards so tenants can see their consumption.

Days 61-90 (Optimization): Add semantic caching to reduce inference costs by 15-30%. Implement request batching for throughput. Build the cost attribution dashboard for per-tenant margin tracking. Evaluate self-hosted models for the cost crossover point. Add provider failover for reliability.

The temptation is to build the full platform on day one. Resist it. Ship the MVP, validate revenue, then optimize. Every week of premature infrastructure engineering is a week you could have spent talking to customers.

Frequently Asked Questions

What gross margin should an AI SaaS product target to be sustainable?

Target 65-75% gross margin at scale. Early-stage AI products (under $100K MRR) typically run at 40-55% gross margin because inference costs are high relative to revenue and you have not yet optimized caching, batching, and model routing. The path to 65%+ margin involves three levers: semantic caching (saves 15-30% of inference calls), intelligent model routing (send simple queries to cheap models, complex queries to premium models), and self-hosted inference for high-volume workloads (saves 40-60% versus API pricing at 10M+ requests/month).

How do I prevent tenant data leakage in a multi-tenant RAG system?

Use namespace isolation as the minimum: each tenant's documents are embedded and stored in a dedicated namespace within your vector database (Pinecone, Qdrant, or Weaviate). Every query includes a mandatory namespace filter that the application code cannot override. For regulated industries (healthcare, financial services), use collection-level or database-level isolation where each tenant has physically separate indexes. Add integration tests that specifically attempt cross-tenant queries and verify zero results. Log every vector search with tenant ID and result tenant IDs for audit trails.

When should I switch from hosted LLM APIs to self-hosted model inference?

The cost crossover typically occurs at 5-10M requests per month, depending on your average prompt length and model choice. Below 5M requests, hosted APIs (OpenAI, Anthropic) are cheaper when you factor in the engineering time to operate GPU infrastructure. Above 10M requests, self-hosted open-source models (Llama 3.1 70B, Mistral) on reserved GPU instances cost 40-60% less than equivalent API pricing. The decision also depends on latency requirements (self-hosted can achieve sub-100ms p99), data residency constraints, and whether open-source model quality meets your use case.

Ready to build AI products that generate revenue? Browse 320 premium AI and cloud architecture blueprints or start with our 17 free courses covering AI engineering, cloud infrastructure, and enterprise architecture.