Where’s IT in the Gen AI debate?

MC+A Insight Guest Article. Original Article can be found here.

Background

Vivek Sriram

Chief Product Officer @ Bookend AI

The corporate role most frequently missing from conversations about Gen AI is that of the Chief Information Officer. Everywhere else, there is certainly no shortage of glitz and energy about the transformative impact that Gen AI will have — from sales and marketing to support and every other interaction in the customer lifecycle. It is however surprising that the backbone to any GenAI induced transformation — namely, IT — is continuously sending cautionary signals about how immediate and how widespread that transformation is going to be. It is curious that IT is so obviously missing from this discussion. No matter, until their very real concerns are met, any enterprise-wide adoption of Generative AI will be slow to non-existent.

Expanding Gap between IT and Business

First the good news. In a survey of over 500 CIOs done by Salesforce and IDC, 2/3 of those surveyed IT leaders say they are prioritizing Gen AI tech over the next 18–24 months. 1/3 of those surveyed say that Gen AI is indeed their top priority. This is the corporate equivalent to the change an earthquake leaves — deep, dramatic, quick and permanent. The interest in Gen AI isn’t matched with a desire to leap forward with corporate wide rollouts, however. 2/3 of CIOs do not think that they have the capacity or the skills to successfully use Gen AI tech in their organizations and a full 99% believe that businesses must take measures to better equip themselves to successfully leverage the technology.

It’s easy to be mesmerized by the shiny objects that promise automation and efficiency. Until those promises are reconciled with the very real risks that Gen AI comes with, enterprises will tread very cautiously. CIOs worry about security implications and about administrative overhead (in part resulting from capacity and knowledge gaps). Right now the infrastructure for managing Gen AI isn’t quite mature enough to satisfy requirements across these two critical dimensions. Look deeper.

From CIO,” …for about 71% of IT leaders, angst about security creates a barrier to adoption, mandating that approaches, infrastructure, data strategies and security be appropriately aligned.”

This alignment in other transformational technologies has happened because platforms obfuscated the underlying technical challenges inherent in implementing them and let developers focus instead on building applications. That is notably not the case in Gen AI. The capacity and knowledge gaps from those who know how to perform alchemy to those who have to deal with the aftermath is wide and getting wider. Current platforms by and large do not solve these problems. Despite the buckets of VC cash being dumped into “prompt engineering” startups or the plethora of low-code and no-code tools targeted at enterprise analysts, marketing staff and product managers, there is virtually nothing out there that offers IT a end-to-end tooling that can handle security and administration. Consequently, the very limited supply of engineering talent which can work on deep tech is demanding exorbitant wages to frequently work on undifferentiated heavy lifting.

Generative AI is not Enterprise Ready

Wide adoption of Generative AI in the enterprise will not happen until risk is more predictable. Policies around security, governance and compliance are rapidly evolving. The tools and infrastructure aren’t evolving at the same pace. In fact the gap is widening. This leaves CIOs with only one possible path for meeting the needs of the business — that is to use policies, gated by people, to keep bad stuff from happening. It’s ironic that technology that is both revered and feared for an ability to automate everything is being managed and safeguarded with the most manual of constraints.

This excellent risk matrix from BCG is a good model to consider how to think about evaluating the risk and promise of Gen AI in the enterprise. While the whole piece is worth a read this callout in particular is instructive: “companies need policies that help employees use generative AI safely and that limit its use to cases for which its performance is within well-established guardrails. Experimentation should be encouraged; however, it is important to track all experiments across the organization and avoid “shadow experiments” that risk exposing sensitive information.”

Right now the overwhelming majority of IT leaders are cautious about Gen AI adoption in the enterprise, in some substantial measure because the systems to manage these applications have not yet emerged. 99% of IT leaders feel that business must take measures to better equip themselves to successfully leverage the technology. For this number to drop, enabling Gen AI specific tech like infrastructure needs to emerge.

A future where Generative AI is Enterprise Ready

The current landscape of Generative AI tools is by and large made up of tech data scientists are building for other data scientists. Obviously this is important as data science is frequently the crucible where concepts are crystallized. But it’s not enough. The security, administrative and integration considerations that enterprise developers, system operators and IT leadership have are a reflection of wider corporate priorities that data science alone doesn’t quite reflect. Most notably, enterprises lack the capability for administering and securing Gen AI apps simply and reliably. AWS and Google are starting to offer some new services and maybe this heralds a new era of IT innovation. Right now both are preview announcements with limited features and seem to be designed for keeping their current customer base locked into their large platforms. Gen AI is a fast growing ecosystem with innovation happening everywhere. If ever there was a tech crying out for heterogeneity, this is it. Right now, there isn’t a multi-cloud, heterogeneous hosted Gen AI service that is truly serving the needs of IT while delighting developers. Maybe one day.

Recent Insights

After App Search: A Modernization Blueprint for AI-First Organizations

Essential considerations for a successful migration away from App Search, emphasizing the importance of assessing your current implementation and aligning search with your business goals. Discover how modernizing your search technology can enhance relevance, expand use cases, and operationalize insights.

Why Federated Search is Still Relevant Today

Glean’s recent blog post claims that federated search is on its way out. However, federated search is more relevant today than ever, and protocols like MCP are enhancing its effectiveness

What’s New in Elasticsearch 9.0: Key Innovations in Search, Observability, and Security

Elasticsearch 9.0 is finally GA, and it’s packed with powerful new features that push the boundaries of search, observability, and AI-driven analytics. Whether you’re building semantic search applications, managing massive datasets, or monitoring LLM performance, this release delivers innovations designed to meet modern data challenges head-on. Here’s a breakdown of the most impactful updates in Elasticsearch 9.0 from our perspective:

Trusted Advisor

Go Further with Expert Consulting

Launch your technology project with confidence. Our experts allow you to focus on your project’s business value by accelerating the technical implementation with a best practice approach. We provide the expert guidance needed to enhance your users’ search experience, push past technology roadblocks, and leverage the full business potential of search technology.