Agent Skills Masterclass

The AI Daily Brief is a daily podcast and video about the most important news and discussions

in AI.

“All right friends, quick announcements before we dive in.”

First of all, thank you to today's sponsors, Recall.ai, robots and pencils, blitzy and super intelligent to get an ad-free version of the show, go to patreon.com/aadlibreef. And if you are interested in sponsoring the show, send us a note at [email protected]. Now one other note, today's episode of Course Features, the one and only new far-guest bar, walking us through a masterclass in Agent Skills.

For anyone who listen to my Agent Skills primer, it's a really good part too for that, that gets much more practical with a whole framework for how to use skills and how to use them well. Now you can get all of this companion data, including things like the anatomy of an effective skill, over on play.aadlibreef.ai, that's where we keep the companion experiences for the

show.

And if after that you want even more new far, we have just opened up the second cohort

for Enterprise Quar, which is an Agent and Agent team building program. I'll have links to all of that in the show notes.

“For now though, let's dive in and up your skills with skills.”

All right, new far, welcome back to the show. We're talking skills. How you doing? I'm good. Happy to be here.

Yeah. We are, man, it is a, since the last time you were on, the things that matter in terms of teaching, being up to speed, getting up to speed with AI are, you know, some of them are obviously that there's fundamentals of teams and, you know, expectations and things like that that remain.

But God, the last time we were here that stuff we were talking about feels like ancient compared to where we are now. The human element is the same, the technology is completely different. So what we're talking about today is I did a couple of weeks ago on the show, kind of an introduction and a primer to Agent skills.

It's a standard, a sort of primitive for the Agent era that helps Agent figure out how to do things that you need them to do in very simple terms. But obviously there's a lot more complexity in how you use them and how you use them well.

“And that's what we're going to be talking about today.”

So tell us a little bit about what we're going to go through and then let's dive in. Okay. So obviously you did a very good job in your skill episode. You talked about what they are, the entropy categories and the various things that are currently like a landscape overview.

But today I want to go much deeper and make it more of an operator cut because I want to give people the actual playbook on how to build skills that work, what my text, kill them and what organization and opportunity really looks like.

So we made it fun, like we always try to do and we structure it in our five-level journey.

So by the end, hopefully you will go from understanding what skills are to knowing how to build an organizational skill library and everything is accessible to you guys on the play at AI Daily Brief, which we will demo in a minute. Okay. So we have five levels from apprentice to architect.

So to make sure that we are all on the same page and give a reminder of what skills are. At the core, skills are just folders, not just markdown files, folders that contain instructions, scripts and resources that give AI tools and agents the actionable playbooks to execute various tasks.

But here there's something that many people are kind of missing and that is that skills are not just for agents to it. They work in two modes. An agent can discover the skills that you enable in the environment and it can do so automatically and invoke them on its own or as humans can trigger them manually either by using the

slash commands and most tools or we can just provide verbal cues and the tools will know to pick up the skills that we intend them to use. So for example, you may say research this topic and it fires a very specific research skill that you built that is very specific to what you like in terms of doing the research. So that's something to it also show in a minute.

And the very good thing about skills is that they are highly portable. Most of us have built many custom GPTs or gems over the last few years. The problem with them is that they were locked inside the church GPT or the Gemini Enterprise Forever. The skills basically serve that they are folders that you can just take with you between

tools, they are human readables. So there is no proprietary format and anyone in your team can open a skill file ready to understand it, edit it and you don't need any engineering degree and you can just take it between tools. Why are we saying that skills is not only the present but also the future of AI and agent

tools because we are already seeing that all major companies are supporting skills currently

support tools recently.

“No one said so and many other tools already announced that.”

We of course include in the tools that support skills, the open cloud, the cloud, the cursor, the windself, get up and many, many other tools they are also supporting that but not just coding tools. And then we have people that have been basically using and building skills effectively

for a while and they will tell you that this is probably the complete game changer to

our AI and agent work for you and also it's quite addictive like when you start realizing the power of skills or the ability will create more and more and more. And I do want to flag out wanting very explicitly here is that third party skills wanted to acquire from somewhere in the internet whether it's an open cloud market place or other places they are code and as such they can run with a lot of your agent permissions and

if you download that it can execute scripts and sometimes it can be a malicious script.

“So be very, very careful whenever you're getting a skill that you have not verified the”

source, read it very carefully and try it like installing any software package on your machine and especially if it's a work machine be very careful and pay attention so you will not bring any malicious software back into your organization. All right, so this is the basic let's talk about when to build skills. So the question is when should you build skills and I wanted to start with three obvious

signals either when you do something more than three times that's to make the good indication that now is a good time to build a skill or you keep basically pasting the same instructions and getting very frustrated with your tool and that's another good one and also when you need a consistent output. But the two additional things that you want to consider first of all this is a great opportunity to standardize things across either the way you do

the work or others do the work with you. It's a great opportunity for you to think of all the things that you ever wanted to be more consistent of or get more consistent behavior by others and just build a skill to get others to behave the same way and lastly and that's something that often also when I'll double it talks about skills are not just a way for you

to be more productive. It's also a way for you to unlock opportunities of things that you always

wanted to do and just didn't have the bandwidth or the ability to do so. So think outside the box of water some of the research tasks that you never had the opportunity to do or water some of the work and the business changes that you could never ever sort of because you didn't have the know how and the bandwidth. Okay, I want to talk about two other things. One is that skills cannot be ten different things. So one skill per task if you find yourself getting

to a point where it's completely separate jobs separate them to different skills and lastly when it comes to the question of reuse versus creating a skill I know that there are many marketplaces out there and especially in the open cloacos system we know that there are an abundance of them similarly with the entropic skills repo. However, it's very hard to navigate some of these skill marketplaces and find the exact fit and often it will find yourself wasting

so much more time just trying to read what others created versus creating your own. So especially

“if you want to hone your ability to create skills I will actually recommend that you sit down”

build the skill for yourself leverage some of the best practices that we're showing here and at least you will learn how to do it. Well later on you can of course go and search what

others are building because some of the skills that people are building are amazing but I would

advise to lean on more heavily towards the building at this damage. One note on that front I agree entirely I also think that by virtue of them being sort of marked you know just mark down files you can also treat even skills that you download as templates not things that you have to copy wholesale. So in the next show that I do this week it's going to be a personal context portfolio and I'm sharing a GitHub repo that has basically templates for 10 files about yourself and it's sort of

it's not meant to it's not something you would copy it's about yourself so you have to use it like a template but I think there's a lot of resources out like that and so I think it's sort of puts a fine point on the idea of wanting to have the skills to build because it actually unlocks using all of these things that are out there in different ways that aren't just sort of blindly copying it into your projects and hoping it works. Yeah and by the way a versus custom

GPT is that we're black boxes if you were to use others now you'll get the full visibility into how the skills instructed so if you don't like some of it just change it. I just wanted to note that Claude created an amazing and tropical that created an amazing skill creator that they recently released and I definitely encourage you to go and use it because it's generally impressive it interviews you to extract your expertise it runs if as it does a

be testing and benchmarking so if you are a Claude user you can definitely leverage their

to understand what is the anatomy of a very effective skill we created at least for you so every skill

“should have some of these elements and I want to emphasize a few of them. The most important part”

is the beginning and that is the trigger. The trigger is how you instruct the tool on when to

discover and when to basically fire this skill and it's probably the most important line because if

your trigger is not very precise or very make then your skill will just not be used and selected by the agent so I would advise actually that you make it louder rather than quieter because the models will sometimes skip past more subdued descriptions so trigger words exact descriptions about when do you expect to be used and be more explicit than implicit here that will go along way and then we have the body and what most people go on with the body that they write pros and skills are

like playbooks so favor and numbered steps or bulleted lists. Claude and all of the AI tools they really like structured instructions dramatically because that will also turn to be their action

“plan if it's very very concrete so try to make it as literal as possible that's how the tools”

like to follow the instructions however I want you to also set the right level of freedom so if a task is very fragile like a database migration coding according something that has to be very precise be very prescriptive with a step by step but if it's more of a creative task like writing a strategy doc or something that is more open to interpretation give the guidance but do leave some room for the tool to be creative because if you overaloding the model you will not get as good results.

We also encourage you to make sure to include an output format and here it's even better if you just include an output example so show the model don't just describe if you want a template include it if the output is a table show a table and headers if it's a document show the section structure so that's very useful for you to get exactly what you want out of it and another section that and tropical commanded very strongly is the Gacha section. This is probably the highest signal

content in any skill because it's the area where it gets the model to go out of its own patterns because you're looking to put here things that where the model will typically go wrong or

“what assumption it might make that they each shouldn't and you need to say something like”

I know you want to do X but don't here's Y and every failure that I've seen is probably something that you should document here after you stress test your skill. A few things not to include are some of the classical prompting skills like don't include the persona and stuff like that that's not useful the tools are looking to get playbooks a few skill killers that you should avoid

first of all it's the trigger if the trigger is not well set the skill will never be picked for

usage second over defining the process like we said don't railroad the model also don't stay the obvious don't waste tokens on things that the model already knows and we strongly recommend that you don't skip the Gacha section because this is often when your skill will go off or will create a sub-optimal results and lastly don't do like a monolithic blob everything crammed into one file instead of using more of a folder structure so speaking of folder structure the

recommendation is to keep skill under 500 lines because it's a playbook not the integral idea of everything that you do for work if you have reference materials or context that are very important for the skill move them outside of the skill file into a separate set of files within the skill folder when it's relevant if you also have very long input and output examples and you should include input and output examples you can also put them in a separate examples dot md files

inside the skill repo or the skill folder that will help you a lot and that's probably the most effective way to communicate the desired format in terms of the discussion of whether or not I should append a bunch of files into my actual skill folder that will be bundled with the skill wherever the skill goes versus perhaps just pointing the skill to my other files and other systems the deciding factor should be if this is something that is context specifically for this skill that

should always come with the skill whenever I'm offloading the skill to someone else then put it within

the skill folder otherwise when it's stuff that's more general about you or about your company that can be pointed to an external source why is there always a meeting bot in your zoom call

products like klui, hub spot and click up they handled the hard infrastructure work capturing

“clean recordings transcripts and metadata across zoom Google meet microsoft teams in person meetings”

and more so developers don't have to build with themselves if you're building a meeting note taker or anything involving conversational data recall dot ai is the api for meeting recording get started today with the hundred dollars in free credits at recall dot ai slash a_idb that's recall dot ai slash a_idb today's episode is brought to you by robots and pencils a company that is growing fast their work as a high growth a_w_s_ and data breaks partner

means that they're looking for elite talent ready to create real impact at velocity their teams are made up of a_i_native engineers strategists and designers who love solving hard problems and pushing how a_i_ shows up in real products they move quickly using robber works their agentic acceleration platform so teams can deliver meaningful outcomes in weeks not months they don't build big teams they build high impact number ones the people there are wicked

smart with patents published research and work that's helped shape entire categories they work in velocity pods and studios that stay focused and moved with intent if you're ready for career defining work with peers who challenge you and have your back robots and pencils is the place explore open roles at robots and pencils dot com slash careers that's robots and pencils dot com slash careers one to accelerate enterprise software

development velocity by five x you need blitzie the only autonomous software development platform built for enterprise code bases you're engineers to find the project a new feature refactor

or greenfield build blitzie agents first ingest and map your entire code base

then the platform generates a bespoke agent action plan for your team to review and approve once approved blitzie gets to work autonomously generating hundreds of thousands of lines of validated end-to-end tested code more than eighty percent of the work completed in a single run blitzie is not generating code it's developing software at the speed of compute your engineers review refined and ship this is how fortune 500 companies are compressing

multi-month projects into a single sprint accelerating engineering velocity by five x experience blitzie first hand at blitzie dot com that's bli tz y dot com it is a truth universally acknowledged that if your enterprise AI strategy is trying to buy the right AI tools you don't have an enterprise AI strategy turns out that AI adoption is complex it involves not only use cases but systems integration data foundations outcome tracking

people and skills and governance my company super intelligent provides voice agent driven assessments that map your organizational maturity against industry benchmarks against all of these

“dimensions if you want to find out more about how that works go to be super dot AI and when you fill”

out the get started form mention maturity maps again that's be super dot AI let's show you a concrete skill that is slightly more advanced and that is a meeting prep skill obviously all of us know what we do we have to get ready for meetings so I wanted to show

case an interesting skill here first of all description when to use when use a prep for the meeting

meeting prep and so on so it's offering quite a few options so the skill will be picked up in almost all scenarios in terms of context required so as part of the skill folder a bundle stakeholder context so either it's going to be transient or if you have regular and stakeholders that you work with that can be fixed context pilot comes with the skill it should get some email history for instance is so that can of course be pulled directly from the use of systems calendar

and other open action items involving the attendees in terms of the steps identify all the attendees from the calendar all the the other inputs collect the context analyze the agenda lan scenario analysis so part of what the skill does is kind of preparing you for what can go on with your meeting and generate a brief and the output will be structure that is defined in an attached file because it's a very long and specific output and in broad talks the structure will be executive

summary and so on a few got you that can happen when you're getting ready for a meeting with such a skill sometimes I assume it's a tendency already just from title and if someone is a VP there

assume that they're the most important person in the room and attributes unnecessary wait for them

don't fabricate company details don't prefer generic talking points don't skip the what could go wrong analysis and so on these are things that happened when triggering the skills

“without it so that's why they're here and the way this overall folder is structured we have”

this file we also have like I said this take all the context is an external pointer to be sure the course relevant skills the brief format scenarios examples and also there is an ested skill and in this nested skill there is a sub skill of how to simulate the actual

come with the six to seven different scenarios of what could could go wrong we'll see like if you someone is joining your meeting and has a agenda I will you address them someone is asking you difficult questions so it will literally help you get ready for difficult questions if you use it for a sales call it can come up with difficult questions around the sales and so on so that's an example of how you would build a skill that also has context and perhaps refers to

another skill moving on so obviously there are many many skills that people have but I wanted to include here for ideas of skills that might be useful for anyone who is a knowledge worker which is

most of the audience here so first of all as part of the material that we provide you we

included an example of research with confidence that's skill that not only does research that is very precise to what you care about time horizon specific sources but also it has a built in fact checking methodology where it will compare sources and do a deeper dive into specific things that seems off as well as giving you confidence scoring about how securities with the findings

“so you can decide how deep to go in so that's one skill that I think every person who does any type”

of research which is all of us that should build or reuse another one that I really like is devil's advocate this is basically a skill where we say takes any proposal and systematically stress test what makes the version that we included a little bit more special is that it takes explicitly looks for blind spots and biases both on your side and on the AI side because we know that the

models have many of their bases so it's explicitly tries to avoid those and it always ends up with

something that is more constructive so it's not just kind of finding holes with anything that you want to do but also helps you to be back to something that is actually actionable and another skill that we created is a morning briefing that's another classical one it pulls together your priorities calendar pending item relevant news and the thing that makes it more powerful is that it binds your personal context files with the skills including the the goals the current project

and stakeholders and as part of the materials that we provide with this episode we also included a poem that lets you create one for yourself that will interview you and make sure that you will be able to create your own morning briefing and another one that I strongly recommend that you will

build is a board of advice or skills so either one or several of those basically will simulate

perspectives coming from multiple expert archetypes so it won't be just like think like a CFO but rather you can think of all the different perspectives that will help you make decisions so if you are a starter founder then perhaps your board of advice also will have someone coming more from a VC background someone for more of an entrepreneurial background your imaginary advisor and various other perspectives and you basically create a skill that gets them all to advise and assist you

by providing various perspectives on any decision that you would like to make a few more advanced patterns that for people who are already built skills for a while I want to take it one step further first of all having a dispatcher skill which is a meta skill that reads all of your requests and

“routes them to the relevant skills it's like a traffic controller basically and it's very important where”

in your library of skills go past 10 or 15 active skills that you regularly use often I would advise that you will create this dispatcher instead of hoping that the agent will read to all the available skills and pick up the right one and this is especially important when you have nuanced similar skills that you want to be picked up in completely different scenarios another thing that you can do is you can chain skills one after the other either automatically by having a skill that basically calls

one skill after the other or manually you will take the output of one skill and it becomes the input to another skill so in the examples that we've shown before maybe you start with research with confidence and then the output you take it to the devil's advocate to poke holes in the research and then you take it to another skill that does an executive summary and deck preparation for you

“the only thing here is that skills need the clean input and output so that's an important thing in”

order to change them well obviously recently we've been seeing more and more the emergence of loops agentty clubs and other loop patterns so you can also create skills that create stuff like that that they will iterate check act check again and then iterate and it's becoming very interesting also for an untechnical stuff because you can think for example on marketing a campaign optimization for example you will monitor your ed performance adjust the beads, recheck, flag when the specific

so you have like an endless loop of some of the optimizes your campaigns and you can also create

skills that basically orchestrate multiple agents or multiple sub agents execution you can just

explicitly prompt them to spin up multiple agents our research skill also does that so you can take a look at that and of course the sky's the limit and we'll be curious to see if you have other advanced patterns that you've discovered that are working well with you. I want to make sure that you don't just create skills but that you test them and make sure that they're working well for you over time I think the easiest test for you is if you find yourself having to iterate after

you get the output of the tool that uses your skill that means that your skill is not good enough

“because ideally a skill should create a ready to use output and if this is not the case you have to”

go back and fix the skill and this becomes even more important when you're about to share it with

50 different people that's the case for you to treat it like any other AI product and basically run

proper evaluation and the rigor of course you'd match the stakes if it's something that also updates your CRM then make sure that the skill is well tested if it's customer facing make sure that it's well tested and there are some ideas here on how to test it but in general every time that you have a new model or that you have a different tool that will be using the skills you have to go back and reevaluate okay let's talk about the organizational perspective so up until now skills

were at least it could be inferred with primarily talking about skills as a personal asset however organizations that are very AI forward we already realized that skills are the future of how to

“streamline work and how to get everybody to get more value from AI and as you can hear this is where”

I get genuinely excited because it's basically the pipe dream of every knowledge manager that finally can become real and you can think about it at the following you can standardize the way work gets done you can get a lot of the work done autonomously or some extent and you can bundle that also with organizational knowledge so everything can be bundled into a single portable artifact that both the humans can read and new employees can onboard using that as well as the agentic tools that

will be using them and doing the work for you so what have seen happening in some organizations they do skill hackathon where they create skills for their relevant teams they are maintaining skills in shared libraries like they would maintain code they make sure the skills are having clear ownership and use the course various people and those organizations are seeing massive uplift with the quality and results that they're getting and the ones that are still not there there

people are kind of reinventing the will every time and that they have a conversation with the I or even if they create skills just for themselves eventually and we've already seen that with the cloud core and most organizations will have the set of plugins so in cloud core we're seeing plugins for specific professions but you can create a plugin which is comprised of typically skills and connections and perhaps some context for each and every department or each and every group in your organization

and all of a sudden everybody enjoys the same world view and the same goodness so to be a little bit more prescriptive here what I would recommend that you do at the org level you will start with this cover running work audits or understanding where people do repeating work or where people are not getting optimal value from AI and where they're wishlist that are not being covered so you will have a list of opportunities to create skill and then you will curate them and build skills

using the best possible methods maybe with cloud skill creator or just some of the best practices that we discussed here I then want to encourage you to do a lot of validation especially if those are going to be shared across many people then perhaps the person who created the skill will replace with another person who created another skill and they will stress test each other and of course

“people should use to poke holes in the skill itself then you should package them into plugins”

and reusable elements and lastly skills have to have clear owners whether they are AI champion those subject matter experts be reviewed every time they're being updated and when they're no longer relevant or no longer serving us they should be deprecated because otherwise we will very quickly have a

stale system that was amazing at the beginning but is no longer the case here so that's the five

levels hopefully got you from a skill apprentice to architect everything we talked about plus full skill template and a lot of bonus content and some advanced patterns are all packaged very nicely

artifact awesome thank you so much so what one sort of just mental framework that I wanted to

maybe close on is the last thing you said about deprecating skills when they no longer serve I feel like the even more were used to infrastructure which is what skills are as being

“sort of you know semi permanent or long duration and I think skills feel like one of the first”

infrastructure primitives of the AI era that exemplify one how iterative things are going to be

to the sort of shorter half lives that we have to assume for things that are valuable and I think that what that means is you're not going to have an initiative to design a bunch of skills for

“yourself and for your organization where you do a sprint and then it's done it feels like it's”

going to be something that is just now a new recurring ongoing part of working with these systems

and basically requires constant upkeep is that is that what you've found so far in your

experience with them yes and even in the materials one thing that I included was saying that like reevaluate skills in the following scenarios and when one month has passed because that's about the time horizon where things might become a little bit stale nowadays so at least until and I don't know if it's going to happen in the foreseeable future but until things will be more stable

“and you will have a more self healing system for skills management you have to collectively go”

and revisit everything that is created including by the way the context as part of the skills often the skill itself will remain relevant but maybe the examples or the context that the skill refer to that's the problem of why we're not getting good results anymore interesting we need a little agent that sits there giving a rating to how stale skills our skills are based on when they were last updated I'm sure people that the advanced organizations are already building these systems

of automations of skill reviews and suggestions for improvement awesome all right well new far thank you so much for this short tons of useful stuff for everyone to dig in can't wait to have you back