Skip to Sidebar Skip to Content

AI Doesn’t Make You Faster. Using It Daily Does.

AI Doesn’t Make You Faster. Using It Daily Does.

Microsoft studied its own engineers as they adopted AI coding agents early this year. Those who adopted them merged roughly 24% more pull requests than they otherwise would have. The bigger finding was further down the paper. Engineers who used the tools three days a week saw a lift of about 15%. Those who used them five or more days saw +50.1%.

That is more than three times the gain for adding two days a week. The size of the AI dividend tracked how often people used the tools, not which tool they used.

The study that measured itself

The paper comes from Microsoft researchers Emerson Murphy-Hill, Jenna Butler, and Alexandra Savelieva. It follows the company's rollout of two command-line agents, Anthropic's Claude Code and GitHub's Copilot CLI, from January 5 to April 29, 2026, across what the authors describe as "tens of thousands of engineers."

They didn't rely on surveys. The output measure was merged pull requests, units of finished code accepted into a product. A statistical model estimated what each adopter would have produced without the tools, and the gap came to a +24.0% lift, with a 95% confidence interval of +14.5% to +33.7%. A placebo test on an earlier date showed essentially no effect, which suggests the model isn't just picking up noise.

The authors are careful with their own claims. "Merged PRs are an imperfect proxy for throughput and reward small, frequent PRs," they write, and they read the dose-response curve "as an association, not a cause." People who use a tool daily may simply be the ones it suits best. Even with that caveat, a curve that keeps rising with frequency is hard to wave away.

The tool mattered less than you'd think

One result should make any team hesitate before an expensive vendor bake-off. Copilot CLI users saw roughly 2.2 times the lift of Claude Code users inside Microsoft. The researchers' own explanations are modest: the tools may be used for different kinds of tasks, or Copilot CLI may simply fit Microsoft better, "because Microsoft owns GitHub, the maker of Copilot CLI, but is a buyer of Claude Code."

Put another way, how well a tool fits your environment and your daily routine may count for more than its benchmark scores. A tool your team opens every morning will probably beat a better one it opens on Fridays.

Habits spread through peers, not mandates

How did the habit spread? "Adoption is substantially social," the authors conclude. An engineer whose skip-level peers were largely using Copilot CLI had +216% higher odds of trying it. When a direct manager used it, the odds of trying it rose 82%, but the odds of sticking with it rose only 22%.

So a manager's example gets people to try a tool, and seeing peers use it is what makes it stick. A memo announcing a new tool works less well than a colleague showing you what they shipped with it this morning.

Microsoft's broader research points the same way. Its 2026 Work Trend Index finds that organizational factors, "culture, manager support, talent practices," account for 67% of AI's real impact, against 32% for individual mindset and behavior. Moor Insights analyst Melody Brue summed up the report in her Forbes headline: AI productivity "is not enough."

Why most companies aren't seeing it

The opposite case is now well documented. Glean's Work AI Index, a survey of 6,000 digital workers in the US, UK and Australia, finds that 87% use AI at work and 75% say it makes them more productive. Only 13% say their organization is performing significantly better as a result.

Part of the gap is hidden labor. Workers spend an average of 6.4 hours a week "botsitting," which Glean defines as "feeding it context, supervising its output, debugging its mistakes, and cleaning up after it." And 69% admit to "botshitting," which means shipping AI work they haven't verified.

The detail that matters most is this: Glean's high AI achievers spend more of their time botsitting than low achievers, 40% versus 33%. The people getting the most from AI aren't avoiding the supervision work. They do it every day, until it becomes routine.

Daily use, not blind trust

Using AI every day shouldn't mean handing it everything. Microsoft found that its most advanced users, the "Frontier Professionals" who make up 16% of AI users surveyed, are more likely to do some work without AI on purpose to keep their skills sharp (43% versus 30%). They are also more likely to pause before starting a task to decide what goes to the machine and what stays with a person (53% versus 33%).

In the report's words, these professionals "refuse to outsource their thinking." Using the tools daily and stopping to decide what to hand off go hand in hand.

What this means for your team

For founders and managers, the practical takeaway is to plan for frequency of use, not for licenses. Pick one recurring workflow, such as the Monday pipeline review, the first draft of every client proposal, or the daily code review. Make AI part of it five days a week instead of an occasional extra.

Next, make the habit visible. The Microsoft data suggests that seeing a colleague use a tool does more than any rollout announcement. A short weekly demo, where someone shows what they shipped and how, costs almost nothing.

Finally, budget for the supervision work. If your best people spend 40% of their AI time checking and correcting its output, that isn't waste. It is what the gains cost.

The open question, as the Microsoft authors admit, is quality: "whether this added throughput yields better software." For now, teams that use these tools every day will learn the answer sooner than teams that don't.

Zilck Team Zilck Team
Navigate effortlessly between news, knowledge and entertainment.

Become a Member Now

Join Zilck Pro today and unlock all content.