Horizontal horizon

An horizontal horizon. Image generated with gpt4o

When the Transformation Officer asked, ‘Why is our AI spend soaring while our results feel so fragmented?’ the Enterprise Architect didn’t have a simple answer. The truth was, every department had its own pet AI project, its own budget, and its own version of the truth. AI investment is at an all-time high, yet as a recent McKinsey report notes, many leaders are privately disappointed. Employees are adopting AI faster than their leadership can strategize, creating a patchwork of progress instead of a unified leap forward. This fragmented approach, as a 2025 McKinsey workplace report highlights, often stems not from a lack of technical skill, but from leadership inertia and a lack of alignment.

The problem isn’t the technology itself; it’s the architecture of our thinking. We’re building vertical AI “towers” in finance, marketing, and HR, when we should be weaving a horizontal “reasoning fabric” across the entire organization. This article provides a blueprint for shifting both your enterprise architecture and your organizational mindset.

The High Cost of Vertical AI: A Patchwork of Missed Potential

Today’s approach to enterprise AI looks dangerously familiar. We bolt AI onto existing departmental structures, creating powerful but isolated tools. Finance gets an AI for forecasting, marketing gets one for personalization, and HR gets another for talent acquisition. Each operates in its own world, with its own data and its own logic.

Imagine if, in the early days of the internet, every department had built its own private, incompatible network. Marketing couldn’t see finance’s data, and HR couldn’t talk to operations. It would have been an unmitigated disaster of inefficiency. Yet, this is precisely how most organizations are deploying the most transformative technology of our generation.

The costs of this vertical approach are piling up, even if they’re hidden in different budget lines. We’re paying for redundant data pipelines and duplicative model training. Worse, we’re getting conflicting AI outputs—for instance, the marketing AI’s ideal customer profile doesn’t align with the finance AI’s risk assessment. The organization’s intelligence never compounds. Each silo learns on its own, repeating mistakes and relearning lessons the rest of the company already knows. Instead of a single, cohesive intelligence, we’re building a collection of expensive, isolated brains.

The Horizontal Shift: Weaving an Enterprise ‘Reasoning Fabric’

The alternative is to stop building towers and start weaving a fabric. Horizontal AI isn’t about one giant, monolithic AI. It’s a foundational layer of shared intelligence, context, and governance that serves the entire enterprise. It’s less like a departmental tool and more like a utility—as fundamental as the company network or the cloud.

This “reasoning fabric” has a few core components, explained not in code but in business terms:

• Shared Foundational Models: This is the core engine of reasoning for the entire enterprise. Instead of each department buying or building its own, the organization invests in a central set of powerful models that everyone can access via APIs. This becomes the shared intellect of the company.

• Shared Knowledge Base: This is the “enterprise memory.” (usually as a vector database) It stores information—from financial reports to customer service logs—in a way that allows the AI to understand relationships and context across departments. When any model needs to know something, it queries this shared memory, ensuring every part of the organization works from the same contextual understanding.

• Shared Guardrails: This is the “corporate policy engine.” It’s a central layer that enforces rules for safety, brand voice, and regulatory compliance on every single AI interaction, no matter where it happens. This ensures AI-generated content is always aligned and compliant, significantly reducing brand and legal risk.

• Shared Feedback Loops: This is the mechanism for continuous, organization-wide learning. When a sales agent corrects an AI’s answer about a product, that feedback doesn’t just improve the sales AI; it improves the core model for everyone, from marketing to product development. The entire organization gets smarter with every interaction, fostering compound intelligence.

Siloed mode versus horizontal fabric. Image generated with o3

The Real Roadblock: It’s the Org Chart, Not the Algorithm

The technical components of a horizontal fabric are well-understood. The real challenge is getting out of our own way. As one McKinsey report bluntly states, “The biggest barrier to scaling AI is not employee resistance but leadership inertia and lack of alignment.” While nearly every company is investing in AI, a scant 1% believe they have reached maturity. This bottleneck is in the C-suite, not the data center.

Why this inertia? It’s a profoundly human problem. Department heads often fear losing control over their budgets and operational autonomy. Legacy budget structures are built for vertical profit-and-loss statements, not horizontal platform investments. The pressure for short-term, departmental wins often overshadows the long-term, enterprise-wide strategic advantage. Leaders are being asked to fund a utility that benefits everyone, but which they don’t exclusively own.

This is where the Chief Human Resources Officer (CHRO) and the Transformation Office become the most critical players. The shift to horizontal AI is not merely an IT project; it is a fundamental organizational change. It requires breaking down silos, redesigning incentives, and fostering a culture where cross-functional collaboration is the default, not the exception. Ultimately, it’s about changing how people work, share, and create value together.

Designing the New Operating Model: From CoEs to Value Streams

To support a horizontal fabric, the old operating model must be retired. For years, companies have relied on a central “Center of Excellence” (CoE) for AI. These CoEs were often service centers, taking on projects for different business units. But this model reinforces the very silos we need to dismantle. The challenge lies in transitioning from a project-centric CoE to a product-centric platform team.

The new model looks radically different. It replaces the large, project-based CoE with a lean, central “AI Platform Team.” This team doesn’t run individual projects; instead, they manage the horizontal fabric as a product. Their customers are the internal business teams. Their primary job is to make the shared models, data, and guardrails powerful, reliable, and easy to use across the enterprise.

Empowered by this platform, the organization can orient its work around horizontal “Value Stream Teams.” Instead of a marketing team and a sales team working separately, a “Customer Acquisition” value stream team—comprised of people from both functions—can now use the same shared AI fabric to create a seamless journey from first ad impression to final sale.

For the Enterprise Architect, this means designing for platforms, APIs, and service orchestration, not just standalone applications. It’s about enabling seamless data flow and intelligent workflows across the business. For the CHRO, it means defining new roles like the “AI Platform Product Manager,” upskilling business teams to become “AI-literate” consumers of the platform, and redesigning career paths to reward cross-functional impact over siloed success.

Reality Check: Why a Pure Value-Stream Model Can Backfire?

The horizontal fabric story is compelling—but it glosses over three stubborn realities. First, value-stream teams seldom start with equal AI maturity. Marketing may have seasoned data scientists, while HR is still wrangling spreadsheets. Pushing every stream to “self-serve” overnight creates lopsided outcomes: early adopters sprint ahead while late movers stall, breeding frustration and shadow tech. Without a small but seasoned core to mentor lagging streams and enforce baseline standards, the fabric frays at the edges.

Second, shared accountability often dissolves into shared ambiguity. When every stream is free to tweak prompts, fine-tune models, or recut features, even minor deviations can snowball into governance chaos—think duplicate PII fields, conflicting fairness thresholds, or rogue agents overspending on API calls. A lightweight CoE—retooled as a “governance and enablement cell” rather than a service factory—keeps a tight loop on policies, reference architectures, and drift-detection playbooks. It doesn’t run projects, but it does sign off on guardrails and audits, ensuring one team’s clever hack doesn’t become another’s breach.

Finally, value streams excel at customer outcomes but struggle with cross-stream synergies. Who stewards an enterprise-wide feature store? Who orchestrates multimodal models that cut across acquisition, fraud, and employee engagement? A minimal CoE acts as the fabric’s custodian of shared capital—owning the long-tail road-map items (e.g., global embeddings refresh, vendor negotiations, compliance certifications) that no single stream will prioritize but everyone relies on.

This CoE + Embedded Value Streams. Image generated with gpt4o.

In practice, the winning pattern is a mixed operating mode: 90 % of delivery capacity embedded in value streams, 10 % retained in a thin, product-minded CoE. The CoE tackles enablement, governance, and asset stewardship; the streams own experimentation and business impact. Instead of dismantling the CoE, right-size it and measure it on platform adoption and cross-stream velocity—not ticket throughput. That balance keeps expertise centralized enough to stay coherent, yet distributed enough to stay agile.

Your First 90 Days on the Horizontal Horizon

This is a multi-year journey, but it starts with a single step. You can build momentum and demonstrate value quickly without boiling the ocean.

1. Step 1: Audit & Map (Days 1-30): Your first task is to find the money. Create a comprehensive inventory of every AI initiative across the company. Map the tools, the data sources, the vendors, and the costs. The redundancy you uncover will become the cornerstone of your business case for change.

2. Step 2: Convene & Evangelize (Days 31-60): You can’t do this alone. Form a cross-functional working group with leaders from Enterprise Architecture, HR, Finance, and at least one influential business line. Use the audit findings to build a shared understanding of the problem and a compelling business case for the horizontal platform approach.

3. Step 3: Pilot a Value Stream (Days 61-90): Don’t try to build the entire fabric at once. Pick one critical, customer-facing process that is notoriously fragmented—like customer onboarding or resolving a support ticket. Re-imagine and rebuild that single value stream using a thin slice of the shared AI fabric. Prove the value on a small, highly visible scale.

4. Step 4: Communicate the Vision (Ongoing): Use the pilot’s success—the reduced friction, the faster resolution times, the improved customer feedback—as your primary tool for evangelism. Turn that small win into a powerful story that builds momentum and secures the executive buy-in needed for the broader transformation.

Conclusion

Shifting from vertical AI towers to a horizontal reasoning fabric is not a simple technical upgrade. It is a fundamental rewiring of the enterprise operating system. It challenges our assumptions about control, collaboration, and how value is created. The companies that cling to their siloed AI projects will continue to see fragmented returns on their massive investments. However there a need for strong leadership in common questions, so the idea of having a number of a group of experts in certain areas of knowledge leading its adoption and controlling its governance, is key for the proper rollout of an horizontal organization. The ones that master this shift will unlock compound intelligence and build a sustainable, unassailable competitive advantage. The choice is stark. Stop bolting AI onto the past. Start weaving it into the fabric of your future.