Rewiring Systems to Scale AI From Demos to Deliverables - Nina Edwards of Prudential Insurance

Nina explains that AI initiatives stall not because of weak technology, but because enterprises measure AI using pre-AI assumptions. Productivity gains like faster code generation or automated documentation get trapped in legacy quarterly release cycles and approval chains, making ROI appear flat even when work velocity has radically improved. The key issue is measuring time saved rather than velocity gained.

• •AI generates massive productivity gains but leadership doesn't see ROI because systems can't absorb the velocity
• •Engineering teams generate thousands of lines of code faster with AI, but deployment still follows quarterly cycles, trapping the value
• •Measuring 'time saved' in fragmented increments (3 minutes here, 5 hours there) doesn't capture the real productivity story
• •ROI metrics must match the workflows AI actually creates: faster, iterative, and insight-driven
• •Service teams draft 80% of customer replies instantly, but 2-week legal review SLAs prevent cycle time improvements from showing in dashboards

Nina describes how system-ready enterprises create protected sandboxes with spend caps, redacted data, and automated logging to slash approval cycles from months to days. A retail bank example demonstrates how tiered sandboxes enabled teams to capture ROI in 'time to operationalization' rather than just hours saved, fundamentally changing how value is measured and demonstrated.

• •Tiered sandboxes with spend caps and automated logging reduce approval cycles from months to days
• •Protected environments allow teams to experiment safely before embedding AI into production processes
• •ROI shifts from hypothetical 'hours saved' to measurable 'time to operationalization'
• •Sandboxes enable employees to understand AI tangibly before it becomes embedded in their workflows
• •Safe spaces for experimentation help humans learn to work side-by-side with AI systems

A major healthcare payer created an enterprise KPI glossary that unified definitions of cycle time, exception rates, automation percentage, and rework avoided. This standardization enabled leadership to quantify which AI pilots produced deployable capacity versus one-off time savings, unlocking scale through comparable ROI signals across the organization.

• •Enterprise KPI glossaries standardize definitions of cycle time, exception rates, and automation percentage
• •Unified metrics enable comparison of AI pilots to identify which produce deployable capacity
• •Leadership gains comparable ROI signals for the first time, enabling scaling decisions
• •Standardization extends to skills, role names, and workflow terminology across the enterprise
• •Embedding glossaries in systems like Workday and Jira ensures staffing, funding, and evaluation speak the same ROI language

Nina outlines how enterprises must reorganize humans around AI's new speed by shifting roles from doing to deciding. An insurance example shows underwriters moving from data gathering to decision sequencing, tracking time-to-decision instead of documents reviewed. This fundamental workflow change cut cycle times from days to hours and transformed the ROI story.

• •Humans shift from doing to deciding: gathering to governing, processing to prioritizing, checking to sequencing
• •AI handles high-frequency, repetitive, and deterministic tasks while humans focus on judgment
• •Underwriters moved from data gathering mode to decision sequencing, fundamentally changing their role
• •Tracking time-to-decision instead of documents reviewed cut cycle times from days to hours
• •The human-centered model is critical for converting AI speed into measurable business outcomes

Nina emphasizes that traditional L&D training on AI fundamentals and prompt engineering is necessary but insufficient. Organizations must provide consistent messaging from leadership, safe sandboxes for hands-on experimentation, and opportunities for employees to play with technology before it's embedded in processes. This creates a two-way street where AI learns from humans and humans learn to work with AI.

• •Ground employees in AI fundamentals: the journey from AI ten years ago to today's capabilities
• •Leadership must communicate consistently and transparently about AI strategy and impacts
• •Create safe spaces for experimentation between foundational learning and production deployment
• •Employees need to play with technology tangibly, not just read about it and get tested
• •AI-human collaboration requires a two-way learning process: tech trained on humans, humans trained on tech

Nina introduces outcome charters as reusable frameworks that define business value targets with AI-ready KPIs rather than model accuracy targets. A claims team defined ROI around exception rates and cycle time instead of hours saved, enabling a clear financial story within one quarter that leaders could invest behind, solving both current measurement problems and future pilot approval challenges.

• •Outcome charters focus on business value targets, not model accuracy targets
• •Claims teams defined ROI around exception rates and cycle time, not hours saved
• •Clear financial stories emerged within one quarter, enabling leadership investment
• •Charters are reusable frameworks that reduce upfront pain in getting board approval
• •Telling the ROI story upfront prevents post-deployment measurement disappointments

A financial institution replaced monthly model review meetings with weekly operational AI reviews tracking time-to-decision, error rates, customer impacts, and rework avoided. This shift transformed ROI conversations from hypothetical to measurable, providing continuous evidence of AI value rather than periodic assessments disconnected from business operations.

• •Replace monthly model reviews with weekly operational AI reviews
• •Track time-to-decision, error rates, customer impacts, and rework avoided in real-time
• •Shift ROI conversations from hypothetical projections to measurable operational metrics
• •Frequent cadences keep AI value visible and connected to business outcomes
• •Continuous measurement enables faster iteration and course correction

Nina illustrates how the same term 'analyst productivity' means completely different things across departments: operations measures claims processed per analyst (30% volume increase), while finance measures cost per analyst (8% cost decrease). Without standardized definitions, these metrics can't be compared or rolled up to justify enterprise investment, creating noisy signals that prevent scaling decisions.

• •Same terms like 'analyst productivity' have completely different meanings across departments
• •Operations measures volume (claims per analyst), finance measures cost (cost per analyst)
• •Both metrics may show positive results but can't be compared or aggregated
• •Language fragmentation creates noisy signals that prevent justification of scaling investments
• •Standardizing definitions across the enterprise is critical for coherent ROI storytelling