How to Build High-Quality Agent Skills for Your Team
November 19, 2026
Summary
Agent skills should address tasks your AI agents actually struggle with, rather than add instructions for their own sake. This article explains how to evaluate skill quality through trigger behavior, task success and consistency, and execution efficiency, so your team can keep improving its skills and retire those it no longer needs.
A reader in the E+ Discord community recently asked us to write about how to create good agent skills and how to recognize a high-quality skill.
As development teams adopt AI agents more widely, the quality of their agent skills directly affects the results they get. We think this is a great question. In practice, we also recommend reviewing your team's skills regularly: keep improving them, and retire those that aren't useful or aren't actually being used.
In this article, we'll revisit what agent skills are, discuss when to use them, and look at how to assess their quality and keep improving the skills your team relies on.
What Are Agent Skills, and When Should You Use Them?
When agent skills first launched, we published “Building Agents — What Are Agent Skills, and What Problems Do They Solve?” We also covered them in our online course lesson, “1-2 Using Agent Skills to Make AI Output More Predictable.” We recommend reviewing those resources before reading on.
In Anthropic's talk “Don't Build Agents, Build Skills Instead,” Barry Zhang, who helped introduce agent skills, described a frustration many of us have experienced: AI models are powerful enough to solve PhD-level problems, so why do they sometimes struggle with the tasks we ask them to do?
Zhang's view is that even a person with strong general abilities can struggle with a specialized problem if they lack the relevant domain knowledge and skills. AI agents face the same issue. Agent skills give them specialized knowledge and capabilities so they can complete tasks more effectively.
Concretely, an agent skill is a folder containing instructions, supporting resources, scripts, and other material. The agent uses these resources while working to help it complete a specific task.
Before adding a skill, first check whether the AI agent can already complete the task successfully without one. If it can, there's no need to add a skill. Each new generation of models becomes more capable, and many problems that earlier models couldn't solve on their own are now within reach.
In those cases, providing extra material simply takes up space in the context window, consumes more tokens, and increases costs. Start with problems you actually encounter. Identify tasks where an agent left to its own devices falls short of your expectations, then introduce skills to address those gaps.
How Can You Evaluate the Quality of an Agent Skill?
As we discussed above, a skill may help when an AI agent can't complete a task successfully. That brings us back to our reader's question: how can a team tell whether a skill is any good?
To recognize a high-quality skill, it helps to understand the principles behind writing one. Of the many resources available online, our top recommendation is Anthropic's “Skill authoring best practices.” It explains the principles in detail and provides concrete examples. You can review a skill against the points in that guide.
You can also click Copy page on the guide's page, give the text to an AI model, and ask it to assess whether your skill follows the recommendations.
In practice, though, following the guide alone can leave you with some difficult decisions. For example, it says skills should be concise, but also warns against vague instructions. Being specific can require more words, while cutting too much can make the instructions unclear. How do you know whether you've found the right balance?
Put another way, is there a more objective way to judge skill quality, instead of relying on how the instructions feel when you read them?
To answer that, we first need to define what makes a good skill. Based on our experience at ExplainThis, we believe a high-quality skill needs to meet expectations in three areas. Let's look at each one.
The first is whether the skill is invoked when it should be. If it isn't invoked when needed, it won't help. If it's invoked when it isn't needed, it takes up unnecessary space in the context window.
AI agents use a skill's description to decide whether to invoke it for a given task. To evaluate this, prepare two sets of tasks: one where the agent should invoke the skill, and another where it shouldn't.
Then run those tasks with different models and check for incorrect trigger behavior. If you find problems, revise the skill. For example, suppose the original description is too long and you want to shorten it. After editing it, run both sets of tasks again to check whether the agent still invokes the skill appropriately.
💡 If you only change a skill's body or scripts, prioritize rerunning evaluations of task outcomes. If you change its name or description, switch models, or change the skill configuration, recheck trigger behavior as well.
The second area is how accurately and consistently the agent achieves the intended outcome. Prepare representative tasks and compare performance on the same tasks with and without the skill. Does using it improve the success rate? If the results are about the same, something needs to change. If both versions succeed, the model is already capable enough and you can remove the skill. If both fail, the model still needs help, but the skill isn't providing it effectively and needs improvement.
Run each task multiple times to check consistency. An agent succeeding once doesn't mean it will succeed again. If it completes a task successfully in only half of your test runs, the skill needs work to make its behavior more reliable. When comparing results, keep the model version, tools, initial data, and settings fixed. Reset the execution environment between runs so files left over from one run don't affect the next.
The third area is execution efficiency, which has two sides. One is speed. If two skills both let an agent complete a task accurately and reliably, but one requires fewer steps and finishes faster, that skill is more useful to the development team. The other is token efficiency: completing the same task with fewer tokens reduces the team's actual costs.
Alongside reviewing a skill against authoring guidelines, the most direct way to assess its quality is to put it to work and evaluate it in these three areas.
Support ExplainThis
If you found this content helpful, please consider supporting our work with a one-time donation of whatever amount feels right to you through this Buy Me a Coffee page.
Creating in-depth technical content takes significant time. Your support helps us continue producing high-quality educational content accessible to everyone.