Beyond the Model: The Tools That Will Make Agentic AI Work
AI agents are on the rise, but without the right tools, companies struggle to adopt them. Large Language Models (LLMs) are transforming industries, and businesses are racing to deploy Agentic AI—AI-powered systems capable of autonomous decision-making and execution.
Startups are moving fast, building AI agents as core products. Enterprises, on the other hand, remain cautious, slowed down by concerns over security, compliance, governance, and reliability.
For Agentic AI to scale, companies need better tools—tools for reliability, security, and governance. This presentation examines how nimble startups "move fast" to integrate LLM capabilities, while established enterprises "take it slow" with caution and oversight.
Stephen Li
Startups vs. Enterprises: LLM Adoption Status
Startups are rapidly adopting AI agents, focusing on speed and innovation rather than security and risk management. Big companies see the potential of Agentic AI, but concerns over security, compliance, and accuracy prevent rapid adoption.
Startup Approach
Moving fast to capitalize on generative AI with a "ship it now, fix it later" mentality. Over 40% of recent Y Combinator startups mention agents in their intro page.
Enterprise Approach
Taking a cautious "let's think step by step" approach. Only 29% of enterprise leaders have a near-term vision for company-wide AI adoption, while 46% say it will take more than three years.
Model Preferences
Startups primarily use OpenAI models, while enterprises prefer Azure OpenAI due to security validations and data residency within their own Azure subscription.
Startups: Moving Fast
Rapid Prototyping
Startups can stand up an LLM-powered prototype in weeks and deploy it to users quickly.
OpenAI Preference
Most rely on OpenAI as their primary model, with some using Claude for coding and purpose-built models for specific use cases.
Cost vs. Speed
Startups prioritize go-to-market speed over cost, with many leveraging Microsoft/OpenAI's startup program credits.
Model Selection
Most still use GPT-4o/4o-mini, with few adopting O1 for reasoning due to cost concerns and limited need for advanced logic capabilities.
Startups are less bogged down by legacy systems or strict policies, allowing for rapid experimentation. They're building AI agents for semiconductor design, healthcare, real-time translation, GTM tools, robotics, and fintech solutions.
Enterprises: Taking It Slow
Document Summarization
The most common enterprise use case is making massive amounts of text more digestible, like "chatting with a 1,000-page manual."
Risk Mitigation
Data security, regulatory compliance, and brand reputation concerns lead to controlled pilot programs in isolated environments.
Traditional ML Dominance
Traditional Analytics & Machine Learning still deliver far more business value than LLMs or GenAI—an 80/20 split in favor of traditional methods.
Data Quality Focus
Without strong data management, neither traditional analytics nor LLMs can deliver meaningful results.
Most enterprise AI applications remain internal rather than customer-facing. High-profile failures like Air Canada's chatbot disaster have made many companies hesitant to fully trust AI-driven agents.
The Missing Piece: Specialized AI Tools
Security & Guardrails
Protection against prompt injection and unauthorized actions
Observability & Monitoring
Real-time tracking and logging of AI agent's thought process
Evaluation & Testing
Scenario-based testing for AI agents
Orchestration & Scaling
Coordinating multiple AI agents working together
Both startups and enterprises are discovering that specialized support tools are the missing pieces. Even the most advanced LLM by itself cannot meet all the practical requirements of a real-world business application.
Think of the LLM as the brain of the operation; these tools act as the nervous system, eyes, ears, and safety harness that allow the brain to function effectively in a business environment.
Why Do We Need These Tools?
Accuracy becomes critical in multi-step processes. For example, in a 10-step business process like tracking a software bug, each step could be handled by a different AI Agent. If each step has 95% accuracy, the overall process accuracy drops to just 60%—unacceptable for real-world operations.
Security is another key concern. New threats like prompt injection have introduced AI-specific security risks. Traditional security methods like Role-Based Access Control and logging are not enough in this AI-driven landscape.
What Tools Are Being Used Today
Startups
Building custom tools; LangChain ecosystem popular but insufficient
Enterprises
Focused on pilot projects; less urgency for specialized tools
Cloud Providers
Each offering different frameworks for building LLM-powered agents
Specialized Startups
Emerging to fill gaps in orchestration, observability, security, and evaluation
LangChain and its ecosystem have grown rapidly, with many startups adopting it. However, over 70% of YC startup founders reported that their workloads are too complex for existing evaluation/monitoring/security tools, leading them to build custom solutions.
Enterprises are not highly focused on these specialized tools yet, as most of their AI applications are still internal at the pilot stage. Many executives expressed interest in a platform to enable business users for AI application development.
The Future: AI Platforms for Business Users
Fragmented Tools
Current state: specialized tools for each function
Consolidation
Next phase: integration of tools into comprehensive platforms
Business User Platforms
Future state: low-code/no-code platforms for non-technical users
The future of AI in the enterprise is heading toward integrated platforms that hide the complexity of LLMs and their support tools behind user-friendly interfaces. These platforms will handle security, compliance, monitoring, and orchestration.
70% of S&P 500 CIOs interviewed expressed needs for an AI platform for business users to build AI applications without IT bottlenecks. Just as cloud computing abstracted away server management and enabled the SaaS boom, AI platforms will abstract away model management and could enable an "AI-as-a-service" boom.