Most consumer platforms still run Trust & Safety like it is 2015: too many manual reviews, too much reliance on offshore contractors, too many fragmented tools, and not enough automation that can scale judgment while reducing cost and meeting compliance demands.
That model is breaking.
AI-assisted abuse is scaling faster than human review teams can keep up. And the biggest operational burden is no longer just detection. It is the messy, manual, high-consequence work that happens after the alert: investigation, evidence gathering, policy reasoning, enforcement, appeals, and compliance documentation. Not solving for this exponentially escalating problem is not an option anymore. That is why we believe Trust & Safety is becoming one of the most important software layers in the consumer platform stack.
At Dreamit, we look for markets where the pain is real, the budget already exists, and the workflow is structurally broken. Trust & Safety checks all three boxes.
But what makes the category especially interesting now is not just the pain. It is the timing.
Frontier AI models have changed what is technically possible. For the first time, organizations can realistically embed agentic workers into high-stakes investigative workflows and unlock the kind of step-function gains that can drive 100x ROI. But the model alone is not the product. The hard part, and where the real value will accrue, is the last-mile architecture: systems that plug directly into existing business processes, work across fragmented tools and SOPs, preserve human oversight, and improve speed, scale, and precision without disrupting operations.
In high-stakes environments, enterprises do not want a rip-and-replace AI layer. They want automation that fits inside the workflows they already use today, get immediate ROI from their AI investments, while creating a path toward AI-native work tomorrow.
That is the real opportunity.
The next wave of Trust & Safety infrastructure for consumer platforms spans a broad set of high-stakes investigative workflows across content moderation, fraud operations, and governance, risk, and compliance. In one consumer marketplace example we reviewed, these workflows included product/service listing verification, new user onboarding verification, marketplace guarantees/claims review, video and chat moderation, account fraud and profile authenticity, payment fraud and membership abuse, ID verification and compliance checks, and continuous background screening.
Different labels, same core problem: open a case, gather evidence across systems, apply policy, make a decision, document the rationale, and be ready to defend it later.
Like a SOC, Trust & Safety teams detect, investigate, respond, escalate, and document. They sit at the center of fraud, abuse, scams, unsafe content, policy violations, and compliance events. But unlike modern security operations, many T&S teams still operate with brittle manual workflows and labor-heavy review models. Detection has improved, almost to a fault, generating more alerts and cases than operations teams have the capacity to manage. The investigation layer has not. And that middle layer is now the largest cost center and one of the biggest sources of operational risk.
This is why the category is ready now.
AI-native abuse is rising. BPO-heavy operations remain low quality, expensive and slow. Appeals, compliance, and regulatory scrutiny require audit trails and defensible reasoning. Platforms need to reduce cost without sacrificing judgment as evidenced by a recent spike in Meta/Instagram ATO spike. That is a very different problem than simple automation. It requires systems built for high-stakes decisions, not just faster classifiers.
The winning platforms in this market will not just detect more risk. They will modernize the operating layer where investigators actually spend time.
The architectures we find most compelling are embedded directly into existing workflows, usable by non-technical teams, and designed to keep experts in control. They read the case where it already lives, gather evidence across fragmented tools, reason over policies and SOPs, recommend actions, escalate outliers, and improve over time through expert feedback. The point is not to replace the human decision-maker overnight. The point is to eliminate repetitive casework around that decision-maker, compress cycle time, and improve consistency without compromising oversight.
That is where the ROI becomes undeniable.
In one marketplace example, the pre-automation environment involved roughly 10 million users and 20,000 cases per day. In that setting, one T&S platform reportedly automated 90% of cases, increased throughput from 10 to 1,000 cases per hour, and reduced staffing from 100 to 5 on the target queue while expanding into additional Trust & Safety functions. Whether every company sees that exact curve is a diligence question. But the broader point stands: the investigation layer is dramatically under-automated relative to its cost, complexity, and strategic importance.
This is why “100x ROI” is no longer a slogan. It is becoming a credible outcome in workflows where labor, rework, fragmentation, and compliance burden have historically overwhelmed software leverage.
And importantly, that 100x outcome does not come from some abstract AI overlay. It comes from solving the last mile well.
The near-term win is fitting into the business process as it exists today: existing case tools, existing review queues, existing escalation paths, existing policies, existing compliance requirements. The longer-term prize is much bigger: a path from human-heavy review operations to AI-native work that delivers dramatically greater scale, velocity, and precision.
In other words,
The best companies will not just automate tasks. They will bridge the enterprise from legacy manual operations to AI-native operating models.
This is also why the budget argument matters.
Trust & Safety automation is not asking companies to invent a new spend category from scratch. The spend is already there across moderation, fraud operations, compliance reviews, appeals handling, identity verification, and outsourced investigation teams. Consumer platforms have largely built T&S operations in-house out of necessity, stitching together a cobweb of integrations, manual processes, and internal tools to manage these functions at workforce scale. But these home-grown systems were never designed to be a durable foundation for AI-native investigative work. Over time, they will need to give way to a dedicated T&S operating system that can unify workflows, embed agentic workers, and deliver far greater scale, speed, and precision.
We think this becomes especially obvious in a B2B2C environment with explicit review policies, public community guidelines, branded safety centers, message and content monitoring, and dedicated safety teams. Those are strong signals of real operational pain, real workflow complexity, and real budget ownership. These teams do not need more AI theater. They need infrastructure that lowers cost, improves consistency, speeds decisions, and holds up under appeals and compliance scrutiny.
That is the shift underway.
Trust & Safety is moving from support function to core infrastructure.
The winners in this market will not be point tools built for a single abuse category. They will become the operating system for high-stakes investigative work across fraud, policy enforcement, compliance checks, claims, appeals, and identity workflows.
