Making Human + AI Collaboration a Workforce Strategy

The most important AI choices facing technology leaders aren’t about technology. They’re about people. Every decision about agents, models, or token budgets ultimately comes down to one question: how can our people extract meaningful value from these tools?

According to Karat’s most recent CIO survey, organizations estimate that AI is driving a 34% increase in overall productivity. But that productivity increase isn’t distributed evenly. Nearly three-quarters of CIOs say that their strongest software engineers are now worth at least 3x their total compensation. Meanwhile, more than half say weaker engineers deliver negative or net-zero value. AI isn’t serving as an equalizer for engineering orgs. It’s a multiplier.

This insight matters because software engineering is just the tip of the iceberg for how AI adoption is spreading across the enterprise. AI isn’t just changing how code gets written. It’s changing how work itself is allocated between humans and machines.

When Technology Strategy Becomes Workforce Strategy

This shift is already reshaping executive ownership.

Last year, Moderna merged its technology and HR leadership into a single executive role overseeing both digital systems and workforce strategy. The rationale was simple: in an AI-enabled enterprise, decisions about systems and decisions about people cannot be separated.

And because AI is multiplying output, the productivity gap between top performers and lower performers is growing.

This means the stakes of talent decisions are higher than ever.

Legacy Hiring Systems Are Now a Technical Liability

Here’s the disconnect technology leaders are wrestling with:

●     Most organizations still prohibit AI use in interviews.

●     Most have not updated talent evaluations to account for AI.

●     Yet, most leaders believe candidates are using AI.

In a pre-LLM world, evaluating coding productivity was relatively straightforward. A working solution to a well-defined problem produced a strong hiring signal.

Today, a simple solution to a basic coding question can be generated in seconds. As such, a fully-working and optimized coding solution no longer produces the same hiring signal it once did.

Candidates who play by the rules are at a disadvantage that is difficult (though not impossible) to detect. What’s more, the talent quality signal these interviews produce becomes less predictive.

Rethinking Talent Measurement in the Human + AI Era

When talent evaluations don’t reflect working conditions, predictability collapses. And predictability is what technology leaders ultimately depend on. This is why forward-looking teams are redesigning talent evaluation to mirror real-world work.

Instead of banning AI, they introduce structured evaluation scenarios that observe how candidates reason with AI-generated output, validate and refine machine suggestions, navigate ambiguity and model failure, and balance efficiency with correctness.

The goal isn’t to isolate people in AI-free cleanrooms to conduct interviews, nor is it to create an AI-interviewing chatbot that can outsmart candidates. The goal is to shift talent measurement from a static output to a dynamic collaboration (which sounds a lot like the way most people work, right?).

The organizations with the highest AI adoption rates in our survey were the ones that took a blended human + AI approach to talent measurement. Leaders who lean on live (human) interviews that allow candidates to leverage AI also anticipate fewer coding errors and faster production cycles.

In other words, the CIOs and CTOs seeing the strongest results from their AI initiatives are the ones who are evolving talent systems alongside their AI investments. They understand that if talent measurement can’t detect AI fluency, their AI deployments won’t have the impact they want.

The bottom line is that AI is amplifying engineering ROI, not commoditizing it. For technology leaders, this means it’s time to stop treating talent measurement as an HR process and to start treating it as infrastructure.

---

Gordie Hanrahan, Communications & Content Leader, Karat

To explore the global data behind these shifts, download Karat’s AI Workforce Transformation Report.

And for leaders ready to operationalize AI-native evaluation, learn how to create Human + AI interview rubrics designed to measure real-world performance in an AI-enabled enterprise.