How to Use AI Without Losing Your Technical Judgment

Recently, a PhD student studying computer science at Cornell shared an awkward experience from her Microsoft internship. She said her collaborators seemed overly dependent on agentic tools: whenever a problem came up, their first instinct was to ask the agent to fix it. The issue was that they could not clearly explain how the system actually worked, and they lacked a basic understanding of where she was getting stuck.

It got even more awkward when they tried to solve the problem together. They offered to meet in person and help, but once they sat down, the session turned into either them prompting their own agent while she watched uncomfortably, or asking her to prompt her agent while they watched uncomfortably.

What Is Cognitive Surrender?

This brings to mind a recent University of Pennsylvania study on cognitive surrender. Cognitive surrender refers to the behavior of treating AI output as your own output without checking it, questioning it, or adding your own perspective.

In the study, researchers asked participants to complete different questions and tasks. Some participants were allowed to use an AI assistant, while others were not. The groups were randomized, so the main intervention was whether the participant had access to AI.

During the experiment, researchers secretly manipulated some of the AI’s answers so that the model would give incorrect responses. They found that when the AI’s answer was wrong, participants who used AI were more likely to give the wrong answer too.

In other words, they used AI and absorbed its mistakes at the same time.

The more concerning part came afterward. After participants gave those incorrect answers, researchers evaluated how confident they were. The AI-assisted participants were actually more confident than those who did not use AI. The reason is straightforward: AI often explains things in a highly confident tone. After reading a complete, polished, and confident explanation, people become more likely to believe it and feel certain that the answer is correct.

Now imagine this happening in real work. The consequences can be serious.

For engineers, suppose you ask an AI agent to write a piece of code, and you fall into the kind of cognitive surrender described in the study. You treat that code as your own output without doing any code review. Because the AI’s code and explanation are both delivered with confidence, you trust it.

The result is that you may submit broken code while feeling very confident that you submitted high-quality work. That is why cognitive surrender is dangerous.

More Engineers Are Losing Technical Judgment

This is not limited to Microsoft. A recent Reddit post described a tech lead who used to spend hours at a whiteboard drawing complex system designs. He would explain every tradeoff and make sure the team understood the reasoning behind each decision. Now, according to the post, he just pastes things into ChatGPT and asks it to explain them.

In that same team, a race condition recently slipped into production. When the original poster pointed it out, the lead replied, “But the AI said it was thread safe.”

Both examples show the same problem. Even as AI agents become more capable, that does not mean engineers can stop paying attention and enter a state of cognitive surrender. From a team perspective, the practical question is: what mechanisms can help teams use AI efficiently without sacrificing understanding and learning?

How to Avoid Cognitive Surrender

There are several ways to reduce cognitive surrender. One of the best ideas comes from an engineer who previously worked at Notion. Before submitting AI-generated code, he asks AI to create several questions based on the changes. If he cannot answer those questions correctly, he does not submit the code. This forces him to verify that he actually understands the details of what the AI produced instead of shipping something he cannot explain.

This practice is useful for individuals, but it is even more valuable when adopted by a whole team. Many teams already use AI to write Git commit messages or even open PRs. The quiz mechanism can be built directly into those workflows. Whenever someone uses AI to create a commit or PR, the AI first asks a few questions. Only after the engineer passes the quiz does it proceed with the commit or PR. This helps ensure every team member understands the details behind AI-generated changes before submitting them.

Another useful approach comes from Amazon’s response after a similar incident. They introduced a mechanism requiring human sign-off before a PR can be merged, similar to the common code review practice of requiring approval from specific reviewers. For changes produced by AI agents, Amazon requires an actual human engineer to review and sign off before the PR can be merged.

This sign-off mechanism reinforces a culture of accountability. Simply telling people to be responsible for AI output may not be enough. But when your name is attached to the change, the incentives become different. If an incident happens, everyone can see which engineer reviewed and approved the problematic change. That alone pushes people to inspect AI output more carefully and avoid becoming the person who failed to do the necessary review.

Support ExplainThis

If you found this content valuable, please consider supporting our work with a one-time donation of whatever amount feels right to you through this Buy Me a Coffee page.

Creating in-depth technical content takes significant time. Your support helps us continue producing high-quality educational content accessible to everyone.