Turning Consumer Goods Data into Real-Time Business Decisions - with Michael Finley of AnswerRocket

Finley distinguishes real agents from chatbots by their ability to make decisions and take actions autonomously. Enterprise agents need three core elements: understanding business strategy (not just RAG over documents), access to valid structured data that models can properly consume, and well-designed tools that prevent hallucination. The critical insight is that LLMs haven't been pre-trained on specific business problems like pricing lollipops, so proper data presentation through tools is essential.

• •Agents must have agency - ability to make decisions and take actions, not just respond to queries
• •Train agents on business strategy as you would humans: videos, audio, clear goals (e.g., price optimization, ad allocation)
• •Valid data access requires tools that focus model attention correctly - no pre-training exists for your specific business problems
• •MCP and tool standards are just the 'pages of a book' - what matters is the actual content and implementation

A common failure pattern occurs when teams repeatedly narrow agent capabilities after each demo variation, reducing scope from 100% to 30% to eventually 1% of original intent. This happens when teams add excessive guardrails due to nervousness about non-deterministic outputs. The result is agents that feel inferior to consumer ChatGPT, not due to competence but poor engineering. The solution is treating agents as battle-hardened software requiring proper packaging and testing rather than progressive restriction.

• •Agents often get watered down from 100% capability to 1% through repeated guardrail additions after demos
• •Teams get nervous when agents produce slightly different (but correct) outputs, leading to over-restriction
• •Users complain enterprise AI feels inferior to ChatGPT at home - this is an engineering problem, not capability issue
• •Proper software engineering approach prevents the need for excessive constraints while maintaining reliability

Enterprises must avoid binding too tightly to specific model providers' APIs and ecosystems despite hyperscaler commitments. The competitive landscape means no single provider will always have the best model. Critical governance includes controlling model versions (not auto-subscribing to 'latest'), regression testing model changes like software updates, and treating LLMs as software requiring packaging, testing, release cycles, and ongoing monitoring. Model selection should be based on the simple contract: prompts + tools = actions.

• •Hyperscaler commitments for discounts shouldn't extend to API/model lock-in - competitive landscape shifts constantly
• •Avoid auto-subscribing to 'latest' model versions - major providers have caused production issues with unannounced changes
• •Treat language models as software: package, test, release, and monitor with proper governance
• •Maintain model independence by requiring only: prompts, tools, and actions - nothing provider-specific above that layer

While data governance remains critical (authorized access, version control, proper provisioning), LLMs eliminate the need for extensive data harmonization projects. Unlike traditional systems requiring 18-month normalization efforts to join disparate sources, LLMs can consume multiple data sources separately and synthesize them in context - similar to how humans read multiple reports and draw conclusions. This creates network effects where value builds exponentially across sales, competitor, warehouse, distributor, and weather data without requiring unified schemas.

• •Governance still required: track AI users, ensure authorized data access, use blessed corporate data sources and microservices
• •LLMs eliminate need for extensive data harmonization - can consume 5 separate sources and synthesize like an MBA analyst
• •Network effects: value builds exponentially when relating weather, distribution, manufacturing, demand, and competitor data
• •Incremental dataset additions don't require exponential integration work as in traditional data warehouse approaches

Success indicators focus on actual usage - the agent being 'ripped out of hands' because of value delivered. Critical approach: deploy in 6 weeks not 6 months, then iterate 10 times over that period. Choose problems where money is currently being spent (not new initiatives) to ensure measurable ROI. Monitor what's NOT being used after AI deployment. Avoid chatbot-only strategies where ROI measurement is unclear - focus on jobs AI can partially automate with human oversight.

• •Target metric: high usage indicating value so compelling users demand access
• •Deploy in 6 weeks, iterate 10 times over 6 months rather than spending 6 months before first deployment
• •Choose problems with existing spend - if you're not spending money on it, AI won't save money
• •Monitor what stops being used after AI deployment and focus on measurable job automation with human supervision

Enterprises have decades of business knowledge embedded in Excel sheets, reports, notebooks, and utilities across departments. These become LLM tools or agents that can be accessed across functions. Example: supply chain commodity pricing risk tools should inform advertising spend decisions. This eliminates 90-day, 12-gate processes by giving LLMs access to existing departmental tools. Agents can orchestrate entire workflows (like 90-day supplier onboarding) and escalate to humans only when exceptions occur - the 'vibe admin' pattern.

• •Leverage 50-100 years of business knowledge embedded in existing tools, Excel sheets, reports, and utilities
• •Cross-functional value: supply chain tools inform marketing decisions without traditional 12-gate, 90-day processes
• •Existing forecasting models, risk assessments, and chatbots become tools/agents accessible to other LLM agents
• •'Vibe admin' pattern: agents orchestrate entire workflows (supplier onboarding, quality testing) and escalate only exceptions to humans

Agent-to-agent communication leverages existing enterprise integration patterns from Java Bean era - nothing fundamentally new for enterprise IT. The shift is from people communicating to applications communicating to now agents/LLMs communicating. While user experience may look similar to chatbots, agents are fundamentally more powerful: cognitively loaded, making decisions, taking actions, and going as far as possible before requiring human authorization. The key is proper design ensuring human oversight at critical decision points.

• •Agent communication uses existing enterprise bus/integration techniques - familiar territory for enterprise IT
• •Evolution: people → applications → agents communicating, but underlying patterns remain similar
• •Agents are 'smarter, deeper, more cognitively loaded' than previous chatbot implementations
• •Proper design: agents should proceed as far as possible autonomously, returning to humans only for final authorization