Success Story

Moving Real-Time Content Moderation to Amazon SageMaker: Platform Exit Ahead of Deadline at About 70% Lower Run Cost

Background

Our customer is an audience-engagement technology company that runs the conversation layer for online publishers, including national news brands. It hosts, moderates and ranks reader comments at scale. Its data-science team operates a fleet of real-time machine-learning classifiers: toxicity, hate speech, spam, harm, sentiment, content categorisation and more. LLM-based moderation and reply services run alongside them. The models serve the company's own publisher platform and an external moderation API used by media partners.

Until 2026, all of this inference ran on a third-party managed MLOps platform. The vendor operated the platform on Kubernetes inside the customer's AWS account.

The Challenge

The platform vendor changed its pricing, and the customer's contract was due to end. That created a hard deadline on a latency-critical production path. Every comment on the publisher network and every partner API call depended on the platform.

No elasticity. Streaming deployments ran on fixed replica counts without autoscaling. At the daily afternoon peak, the LLM moderation service queued for over 100 seconds. A partner onboarding was expected to add 40–50% more traffic.
No ownership. The vendor operated the Kubernetes cluster, the serving stack and the network peering. None of it was defined as infrastructure as code the customer controlled.
No cost visibility. The team could not see or attribute the real production cost of serving each model.
Reliability. In its final days on the old platform, the partner moderation API logged hundreds of failed model calls.

The goal was to exit the platform before the contract ended, with no loss of service, no change to model or application code, latency at parity or better, and a materially lower run cost.

Our Solution

MyOps ran the engagement following the AWS Migration Acceleration Program (AWS MAP) sequence. The Mobilize phase was funded through AWS MAP, at no cost to the customer.

Assess

MyOps inventoried 26 model-serving workloads and assigned each a 7R disposition: 23 to replatform, 2 to retire and 1 out of scope.
We right-sized and right-priced each workload on AWS, mapping every model to its accelerator rather than its vCPU count, to preserve latency. T4 GPU models moved to ml.g4dn.xlarge and A10 GPU models to ml.g5.xlarge.
The total-cost-of-ownership analysis projected a 43% saving.
A proof of concept put the first production workload on AWS and set the migration pattern every later wave followed.

Mobilize

We planned the migration waves and the governance around them.
We enhanced the customer's existing multi-account AWS environment and reviewed the security of the target estate. The review covered secrets handling, IAM roles and network paths. The customer's DevOps team accepted the findings.
We restored the observability previously provided by the old platform's console, streaming Amazon CloudWatch metrics to the customer's observability platform.
We defined cost-allocation tags so management could see what the migration delivered.
We wrote the operating runbooks.

Migrate

The migration pattern MyOps set:

Model weights are stored in Amazon Simple Storage Service (Amazon S3) and served from Amazon SageMaker real-time endpoints with autoscaling and full data capture.
Kafka-driven streaming consumers became AWS Lambda functions that read from the same Amazon Managed Streaming for Apache Kafka (Amazon MSK) topics.
The backend's calls to the old platform were re-pointed to an internal moderation API on Amazon API Gateway.
Each workload ran in parallel before cutover, and its verdicts were compared with the old platform's. The LLM moderation service ran a 23-hour production shadow run with 100% verdict coverage and zero errors before it switched over.

The customer's ML team executed the classifier waves on this pattern, and MyOps reviewed each cutover. MyOps also built and launched a new LLM-based reply service on AWS.

AWS services used: Amazon SageMaker, AWS Lambda, Amazon API Gateway, Amazon MSK, Amazon S3, Amazon DynamoDB, Amazon CloudWatch, AWS Secrets Manager, Amazon Elastic Container Registry (Amazon ECR) and AWS Identity and Access Management (IAM).

The Result

Out ahead of the deadline. The first production cutover came in August 2026, and the last workload left the old platform 23 days before the contract ended. There was no loss of service to publishers or media partners.
Reliable at scale. The fleet runs on 15 SageMaker real-time endpoints and serves about 5 million model invocations a day. Over one week, it handled 33 million invocations at a 0.00% model error rate. The partner moderation API ran with zero errors after its cutover, at about 2.5 million requests a week.
Faster and more stable. Fleet median (p50) latency is 28 ms, well inside the 300 ms design ceiling. The customer's team described latency as "more stable", with much lower tail latency.
About 70% lower run cost. Measured in AWS Cost Explorer, the migrated services run about 70% below the previous platform's annual cost.
Owned and visible. The serving stack now runs on AWS managed services in the customer's own accounts, with autoscaling and cost-allocation tags that attribute spend per workload.