I'm a little late to the party on this one! I'm not the first person to write about it, and I'm sure I won't be the last, but in this post, I get my hands-on the Microsoft Copilot for Azure preview and take it for a spin.
From looking at the docs, the official name for this product is Microsoft Copilot for Azure, but for ease, I'll refer to it as Azure Copilot throughout this post.
All credit where it's due, I was inspired to write this post having read 'Exploring: Microsoft Copilot for Azure' by fellow MVP Joe Carlyle. Like Joe, I've been able to get access to the Azure Copilot preview as an Azure MVP.
What is Azure Copilot?
Microsoft Copilot for Azure (preview) is an AI-powered tool to help you do more with Azure. With Microsoft Copilot for Azure (preview), you can gain new insights, discover more benefits of the cloud, and orchestrate across both cloud and edge. Copilot leverages Large Language Models (LLMs), the Azure control plane, and insights about your Azure environment to help you work more efficiently.
Azure Copilot is currently in preview with limited access to enterprise customers only. I was able to get access by playing the Azure MVP card! If you fancy having a look at the preview application form, you can find it here.
The capabilities page goes into a tad more detail, stating that this Copilot can help you better understand your Azure environment, deploy resources, and author code (such as CLI scripts and KQL queries).
The homepage breaks down the features into four key categories, below:
At this point in the post, I'm going into this somewhat blind. I'm honestly not sure if this Copilot is going to blow my socks off with kick-ass AI enabled features, or if it's simply going to pair up Azure Advisor and Microsoft Learn.
Before I move on, I'd like to share John Savill's video on Microsoft Copilot for Azure:
John goes into way more detail that what I will on how Azure Copilot works, including the important topics of permissions and enforcement. I'm quite mindful of how much power this Copilot has over my environment, and what damage it could cause if something goes wrong. I know from this video that Azure Copilot acts on behalf of the user, so it will assume the same permissions I do, but what if my prompts get misinterpreted?!
Azure Copilot operates at the control plane, not the data plane. This means I can use it to interact with resources such as storage accounts from a management perspective, but not the likes of containers or blobs within them.
For reference, the control plane is used to manage resources in your subscription, such as creating, updating, deleting, or configuring them via the Azure Resource Manager. The data plane is specific to an instance, such as a storage account, and handles operations such as reading, writing and querying data.
First Impressions
I'm running Azure Copilot on my personal tenant, which hosts three Azure subscriptions and multiple resources. It's where I host The Cloud Pirate's Azure resources, which consists of resources such as Azure Functions, Logic Apps, Cosmos DB and an Azure Open AI instance.
The first impressions are fine. I throw in prompts to change the portal theme from dark to light, which it handles without issue, and I ask it to list all my storage accounts. All it does there is take me to the Storage accounts portal page.
I do a quick check and confirm it has access to all my subscriptions, as it inherits the same RBAC (role-based access control) permissions I do.
As I'm messing about with it, I realise that my prompts are numbered out of 10, and it clicks that this preview must limit your interactions. Revisiting the capabilities page confirms this:
For each user, interactions are currently limited to ten questions per conversation, and five conversations per day.
There are other limitations to be aware of too, such as the fact some responses that display lists will be limited to the top five items, and that for some tasks, a full resource ID will be required, not just its name.
Design
I've decided to break down my testing into the four key categories taken from the Azure Copilot homepage: Design, Operate, Optimise and Troubleshoot.
Starting with Design, I put the Azure Copilot to work:
As you can see, it didn't go exactly as I had planned.
Maybe I should have said please!! 🤣
In the end, I resort to a prompt asking it to give me the correct Azure CLI to create a new storage account, which was much better:
Clicking Run launched the Cloud Shell in PowerShell mode, however, the query resulted in an error as it could not locate the pre-existing resource group. It didn't have the correct subscription context! Fair enough I suppose, but it would have been nice if it could prompt me for that as a prerequisite.
Interestingly enough, I followed up my prompt by asking it to provide both the Terraform and Bicep config to make the same storage account - which worked for both! I honestly expected it to throw a wobbly when asked to produce Terraform, so that was a nice value add!
I throw down a couple more prompts to see how it responds (if at all):
This is much better than earlier, although now it's just assuming I'll be using the Azure CLI.
Next, I tell it that I need to deploy a highly available web app with a SQL backend, and ask for its help:
This is more along the lines of what I expected this Copilot to be like - helpful guidance tied back to Microsoft Learn.
Ok - I have a good feel for how the Design aspect of the Azure Copilot works now, so let's see how it handles operations.
Operate
In my opinion at least, understanding costs is a key part of operating a cloud environment, so let's see how the Azure Copilot handles something like that:
....oh! I'm running some PAYG subscriptions here that I know it should be able to analyse. This could be an issue with the preview capabilities perhaps. Remember, this product is still in preview.
I try again, but with the full resource ID this time (knowing using names only was a listed limitation), but to no avail.
Let's try something else:
....oh (again)!
The query is provided me is invalid! It should have been:
resources
| where isnull(tags)
| summarize count()
Moving on again, I decided to see what Azure Copilot could do with Entra ID:
And much like before when trying to create a new Storage Account, it failed to reply.
Here's another good one:
Note that with the below, I'd previously been chatting with it about a specific Storage Account (apologies, my screenshots aren't in chronological order).
But the answer to my question is in the query itself! Argh!!
At this point, from the Operations side of my testing, I'm giving up. I'm starting to get a good feel for what Azure Copilot can do, and more importantly, what it cannot do.
Optimise
The Optimise section is where I've found Azure Copilot really show its strengths. Here's a great example:
The guidance it has provided is great, although I would have hoped for more pressing recommendations, such as restricting the network level access to the account.
Following up on the recommendations however is a different matter entirely:
I can however use the Azure Copilot to delete the Storage Account for me (tried and tested), so it certainly should have enough permissions to apply a lock!
The below is more along the lines of what I was expecting earlier. Prompt engineering is a key skill!
Cost management was a dead-end though:
And guidance on reliability also fell short:
The Azure Advisor was more informative than this!
I'm putting a lot of these issues down to the product still being in preview, or perhaps I'm pushing the boundaries of its capabilities due to my high expectations.
Also, these issues could be down to how I've written the prompts. For example, if I had asked for generic guidance, rather than guidance on a specific resource or resource group, the output would have been extracts from Microsoft Learn or other official guidance, which is pretty handy - just not what I wanted.
Troubleshoot
Troubleshooting is the feature I'm most excited about to be honest. My expectations are that the Azure Copilot will help me better understand logs and metrics and will suggest fixes and improvements. Having said all that, so far, the reality of the Azure Copilot has fallen way short of my expectations.
To trial this feature, I stopped one of my existing Azure Functions, and asked the Azure Copilot to help diagnose why it wasn't triggering. Here's how it handled it:
This took me to the 'Availability and Performance' section of the 'Diagnose and solve problems' blade within the Function App portal page. This is fair enough I suppose, but nowhere did it actually suggest turning the darn thing on in order to get it back up and running. Perhaps my expectations are too high.
I change approach and ask it something I know how to get myself via a logs query:
The Events Logs page ins't what I had hoped for. I was hoping it would write me a query, but perhaps this response is down to how I wrote the prompt.
As a last ditch attempt and getting something useful beyond taking me to portal pages, I ask the Azure Copilot to help me determine why a specific Function App cannot read secrets from an Azure Key Vault.
Thankfully, the response was quite useful, although I feel it's the equivalent to what an online search would have provided.
Final Thoughts
Being blatantly honest, I had higher expectations for Microsoft Copilot for Azure (aka Azure Copilot), and I'm left underwhelmed. This statement, as you can appeciate, is highly opinionated.
The issues I experienced, such as the outright lack of responses on occasion, KQL queries being invalid, and a lack of subscription contexts, are a cause for concern, but please remember that this product is still in preview, and that my testing is subjective to my own use-cases and expectations. In all honestly, I was testing this in such a way that I almost expected it to break, which likely skewed my results somewhat, so please bear that in mind.
From my testing, I definitely got a good feel for how it could be used, and in my opinion, it added the most value when prompted to handle generic requests, such as providing guidance on resiliency and security. Targeted requests, such as asking it to analyse logs or perform cost optimisation analysis, fell well short of my expectations.
As an Azure professional, I can't see myself using this tool. I certainly wouldn't pay for it, especially in its current form.
I can see myself however recommending it to my customers, and to those new to Azure. It's a value-add tool rather than something one would depend on, and being able to quickly look up references and code snippets is a nice to have feature.
If you have any questions on this, please feel free to reach out.