Lachlan Wright

Jev, OpenAI or Microsoft: which decision model should you use?

Thu, 15 Oct 2026 00:00:00 GMT

Testing Jev, OpenAI and Microsoft on the same business-style decision tasks produced some clear differences. Microsoft led on average accuracy and calculated API cost, OpenAI was fastest, and Jev performed well in retail and procurement. But some wrong answers came with high confidence. The results offer a useful shortlist, while showing why the cost of mistakes and the amount of human review deserve attention when choosing a model.

I am endlessly interested in how emerging AI capabilities perform in the everyday workflows we help businesses build. Decision models are one area where understanding the trade-offs between accuracy, speed and cost can directly inform our delivery choices. I put Jev, OpenAI’s Decisions API and Microsoft-Decision-1 through the same business-style tasks to explore those trade-offs and share a practical set of findings with the wider technical community.

Average accuracy (%)

Accuracy gives each task equal weight. These are results for this selected sample.

But choosing a service takes more than comparing three numbers. A fast answer matters in an interactive application; a confident mistake matters when it approves something that should have gone to a person for review. The right choice depends on the work you need done.

Microsoft for accuracy and low API spending

Microsoft was the strongest candidate here when prioritising accuracy and API spending. It averaged 96.0% accuracy and scored 100% in logistics, compared with Jev’s 88% and OpenAI’s 73%.

Its typical response took about 328 milliseconds. That may be acceptable for background document checks but can matter when a workflow makes several decisions in sequence.

There were also two failed requests: a timeout and a rate limit. Neither was retried.

High confidence didn’t always mean a correct answer. Microsoft scored 95% on the insurance task but gave all five wrong answers at least 95% confidence. A rule that sent only low-confidence answers for review would have missed those mistakes.

OpenAI for speed and image inputs

OpenAI had the lowest median response time in every task, about 107 milliseconds overall, and completed all 959 requests. That makes it worth testing for routing and interactive workflows where delays accumulate.

Median response time (milliseconds)

Response times are medians, the middle result among successful test requests, rather than a guarantee for every call.

It also has a capability the others lack: OpenAI is the only one of these three services that supports native image inputs. Its Decisions documentation supports text and images; Jev and Microsoft-Decision-1 are documented as text-only.

If your workflow includes photos or scanned pages, that could materially change your shortlist. This benchmark didn’t test image accuracy, speed or cost.

The trade-off was lower average accuracy, 85.4%, and higher calculated cost. Requiring 90% confidence would let only seven of OpenAI’s 100 logistics answers through without review, leaving substantial manual work.

I couldn’t confirm OpenAI’s processing location. These timings therefore reflect the experience from my Australian connection and the service settings used. They don’t prove the model itself computes answers faster.

Jev for task-specific strengths

Jev sat between the others on average accuracy, response time and calculated cost, with no failed requests. It averaged 91.0% accuracy and a median response of about 241 milliseconds.

Retail was a strength: 95% accuracy against OpenAI’s 81%. Jev also narrowly led procurement, although its 88% versus Microsoft’s 87% is too small a gap to base a choice on.

That makes Jev worth testing for those specific workflows. Some banking mistakes carried very high reported confidence, so its review thresholds need checking against real examples too.

What the API costs showed

Across the run, the calculated costs were:

Microsoft: about 1.66 US cents, including a conservative allowance for two failed requests.
Jev: about 3.06 US cents.
OpenAI: about 5.17 US cents.

Accounted run cost (US cents)

Microsoft’s lower cost than Jev came from fewer reported input tokens, despite the identical listed input rate. Sending the same task to two services doesn’t necessarily produce the same billable token count.

Jev and Microsoft have the same listed input price: US$0.042 per million tokens. OpenAI’s Decisions rate is higher, at US$0.10. All three list output tokens as free. These are the base rates used in the test; OpenAI also documents regional and long-context pricing adjustments.

A token is a unit of text the provider counts for billing. I calculated successful-request cost as:

Reported input tokens ÷ 1,000,000 × listed input price.

Reported input tokens (successful requests)

Across the run, the arithmetic was:

Jev: 729,685 input tokens × US$0.042 per million = about 3.06 US cents.
OpenAI: 517,242 × US$0.10 per million = about 5.17 US cents.
Microsoft: 385,159 × US$0.042 per million = about 1.62 US cents.

Microsoft’s total also included a conservative allowance of 0.0375 US cents for its two failed requests, bringing it to about 1.66 US cents. Actual billing for those failures is unknown.

Tokenisation, request packaging and billing conventions may differ; this experiment didn’t isolate which caused the usage gap.

These figures cover 959 attempted requests per service, including the pilot and additional checks, with no retries. They are calculations from reported usage and listed prices, not verified invoices, and exclude tax, currency conversion and earlier preparation.

How decision models fit into a business workflow

A decision model takes information, instructions and possible answers, then scores those answers. Think “approve, reject or review” for a refund, or choosing which team should handle a customer message.

Software can use those scores to route work, with people checking the uncertain cases. The findings above show why both accuracy and confidence need testing: a model can give a wrong answer with high confidence, while a cautious model can leave substantial work for people to review.

What went into the test

The live test ran from Australia on 10 October 2026, using Jev 1.13.0, gpt-6-luna through OpenAI Decisions, and Microsoft-Decision-1 version 1. Each service received the same evidence, instructions and answer options through its own API.

There were eight sets of 100 cases:

Banking: sort customer-support messages into 77 possible categories.
Retail: approve, reject or review a refund.
Logistics: choose the next step for a parcel.
Manufacturing: grade a defect’s severity and check whether there was enough evidence.
Security: allow, block or review a proposed tool action.
Healthcare administration: check whether a document packet was complete.
Insurance operations: decide whether documents supported or contradicted a statement, or left it unresolved.
Procurement: score how well four candidates matched requirements.

Banking used original labelled messages from PolyAI’s BANKING77 dataset. The other seven sets were built from synthetic records and explicit fictional rules. Each contained 60 ordinary cases, 30 boundary cases and 10 with insufficient or conflicting evidence.

A few examples show what the models had to handle:

A refund could fall exactly on a policy’s time limit, with an exception taking priority over the usual rule.
A parcel log could contain old events or unrelated archive records. The answer had to follow the latest valid evidence for the right parcel before the cut-off.
An insurance statement needed to be checked against receipts and estimates. Missing evidence had to remain “insufficient”, rather than being treated as proof that the statement was false.

Separate rule-checking programs verified the expected synthetic answers. Checks also confirmed that converting records into text preserved the facts.

The test cases and scoring rules were fixed before collecting model predictions and weren’t retuned after seeing the results. No paid model supplied the reference answers; there was no independent industry-expert review.

These were controlled text exercises. Healthcare involved paperwork, not diagnosis; security actions were never executed. Ten manufacturing cases had unknown severity and were excluded from that primary score.

Where I would start

For these text tasks, Microsoft is the strongest starting point for accuracy and API cost, OpenAI for response speed, and Jev deserves a closer look for retail and procurement. Image input gives OpenAI a separate reason to be considered.

Before picking any model, check whether ordinary code will do. Rule-based code scored 100% on the seven synthetic sets without API calls because the facts were structured and the rules explicit.

The next useful test is a representative sample from your actual workflow, with independently checked answers. Compare costly mistakes and the amount of human review each service needs alongside speed and usage. One short run can help build a shortlist; it can’t establish a universal winner.

I’m interested in alternative views and findings from similar tests or production deployments, particularly where different data, rules or review thresholds have produced different results. Sharing those perspectives helps build a fuller understanding of where these models can add value.

From AI PoCs to Production: A Playbook that Works

Fri, 20 Feb 2026 20:00:01 GMT

It's 2026 and nearly every large organisation has run an AI proof of concept. The demos are impressive. Stakeholders nod along. A business case gets socialised. And then, somewhere between the excitement and the enterprise architecture review, the whole thing stalls.

The pattern is so common it has become its own cliché: PoC purgatory.

The problem isn't ambition or even capability. It's that most organisations treat the path from prototype to production as a single leap, when it's actually a series of structured gates. Miss one and you're stuck. Nail them in sequence and you can move from demo to deployment in weeks, not quarters.

The 5 gates from PoC to production

Think of these as quality gates, not phases. Each one answers a specific question, and you don't move forward until it's resolved.

1. Data readiness

The question: Is the data available, governed and fit for purpose?

Most PoCs cheat on data. They use curated samples, skip access controls and ignore lineage. In production, that won't fly.

The first thing to get right is your data contracts. You need to know exactly which sources feed the model, who owns them, and what the freshness and quality SLAs look like. If the answer to any of those questions is "we'll figure it out later", you're not ready.

Next, assess data sensitivity early. If the pipeline touches PII, financial data or regulated content, that needs to be mapped and documented before anyone starts building. Retrofitting data governance onto a live system is painful and slow.

Finally, build the plumbing before the model. Ingestion, transformation and storage patterns need to be production-grade from day one. A RAG pipeline that queries a shared drive might impress in a demo, but it won't survive its first security review. Get the data platform right and the AI layer on top becomes dramatically simpler.

2. Risk and compliance

The question: Can we explain, govern and defend this system?

AI risk isn't hypothetical any more. Regulators, boards and customers are all paying attention.

Start by defining your responsible AI guardrails. What can the model do? What can't it do? What topics or actions are explicitly out of scope? Document all of this clearly, because when something goes wrong (and it will), you need to show that boundaries were set and enforced.

Observability needs to be baked into the pipeline from the beginning, not bolted on later. Log inputs, outputs and decisions at every step. If you can't audit a response, you can't defend it to a regulator, a board or a customer.

Perhaps most importantly, engage your risk, legal and compliance teams early and treat them as collaborators, not approvers. The fastest way to kill a production deployment is a last-minute legal review that uncovers issues nobody thought to flag six months ago. Bring these stakeholders into the design process and you'll move faster, not slower.

3. Platform and infrastructure

The question: Will this run reliably at scale, within our security and cost boundaries?

The gap between a notebook and a production workload is enormous. This gate is about closing it.

The hosting pattern decision is foundational. Are you consuming a managed API like Azure AI Foundry, Google Vertex or Amazon Bedrock? Self-hosting an open-weight model for data residency or cost reasons? Running a hybrid setup? Each approach carries real trade-offs across cost, latency, data sovereignty and vendor lock-in. There's no universal right answer, but it's important to make the decision deliberately.

Automation is non-negotiable. You need CI/CD pipelines for model artefacts, infrastructure as code for the serving layer, and automated testing for prompt regressions. If a developer has to SSH into a box to deploy a model update, you're not production-ready.

Cost controls deserve substantial attention. Token-based pricing models can spiral fast, especially once real users start hitting the system at scale. Implement budgets, throttling and usage dashboards before you go live. Discovering a $50,000 monthly bill three weeks after launch is not a conversation anyone wants to have.

4. Product and user experience

The question: Does this actually solve the user's problem better than the alternative?

A technically sound AI system that nobody uses is still a failure.

The mistake most teams make here is focusing on model output in isolation, rather than the end-to-end user journey. Where does the AI fit into an existing workflow? What does the user see when the model is wrong? What happens when they disagree with a recommendation? These questions matter far more than benchmark scores.

Every AI feature also needs a graceful degradation path. Models will occasionally be slow, unavailable or uncertain, and when that happens, the user should still be able to complete their task. If your AI feature has no fallback, you've built a single point of failure into a business process.

Finally, measure outcomes rather than accuracy in the abstract. Task completion rate, time saved and user satisfaction are the metrics that tell you whether the system is actually delivering value. An F1 score of 0.95 means nothing if users are copying the output into a spreadsheet and manually fixing it before they can use it.

5. Change and adoption

The question: Will the organisation actually embrace this?

This is where most technically successful projects quietly fail.

Adoption doesn't happen by default. You need champions, people within each team who believe in the tool and will advocate for it when the initial excitement fades. Find them early, equip them with the context and collateral they need, and give them a direct line back to the product team.

Training alone won't change behaviour either. A one-hour webinar might tick a box, but it won't shift how people work day to day. The far more effective approach is to embed the AI directly into existing tools and processes so that using it becomes the path of least resistance. If adoption requires people to change their habits, make the new habit easier than the old one.

Lastly, build feedback loops from the start. Production AI systems need continuous improvement, and the best signal for what to improve comes from the people using the system every day. Create lightweight channels for users to flag issues, suggest improvements and share wins. This keeps the system getting better and keeps users feeling invested in its success.

Reference architecture patterns (and when not to use them)

Not every AI use case needs the same architecture. Here are the three patterns we see most often, along with honest guidance on when they're the wrong choice.

Retrieval-Augmented Generation (RAG)

Best for: Grounding model responses in your own data, such as internal knowledge bases, policy documents or customer records.

When not to use it: If your data changes infrequently, is small enough to fit in context, or if the retrieval step adds latency that the use case can't tolerate. Also avoid RAG when you need deep reasoning over structured data; a well-designed API or database query is often simpler and more reliable.

Tool use (function calling / agentic patterns)

Best for: Letting the model take actions, such as querying APIs, writing to databases, triggering workflows or orchestrating multi-step processes.

When not to use it: If the actions are high-risk and irreversible without robust human-in-the-loop controls. If the tool landscape is unstable or poorly documented. Agentic patterns are powerful but brittle when the underlying APIs lack clear contracts and error handling.

Multi-agent systems

Best for: Complex workflows where multiple specialised AI agents collaborate to complete a task, such as research and report generation, multi-step approval processes, or orchestrating actions across several business systems.

When not to use it: If the task can be handled reliably by a single model call or a simple chain. Multi-agent systems introduce coordination overhead, latency and debugging complexity that isn't justified for straightforward use cases. They also require robust observability across agents, clear handoff protocols and well-defined failure modes. If you can't articulate why one agent isn't enough, you probably don't need more than one.

SLAs for AI: latency, quality and cost

Traditional software has well-understood SLA frameworks. AI systems need their own, covering three dimensions that are often in tension with each other.

Dimension	What to measure	Example target
Latency	End-to-end response time from user input to rendered output	P95 < 3 seconds for interactive use cases; < 30 seconds for batch/async
Quality	Task-specific accuracy, groundedness, hallucination rate	> 95% groundedness for RAG; < 2% hallucination rate on critical outputs
Cost	Cost per query, cost per user, total monthly spend	< $0.05 per query for high-volume use cases; monthly budget cap with alerts

The key insight is that these three dimensions form a triangle. You can optimise for two at the expense of the third. A faster, higher-quality response costs more. A cheaper response is either slower or lower quality. Make these trade-offs explicit and visible to stakeholders from the start.

How to set AI SLAs

Baseline against the current process. If the AI replaces a manual task that takes 20 minutes, a 10-second response with 90% accuracy is a massive win. Set SLAs relative to the alternative, not to perfection.
Differentiate by use case tier. Not every AI feature needs the same SLA. A customer-facing chatbot needs tighter quality controls than an internal summarisation tool.
Monitor and iterate. AI SLAs aren't set-and-forget. Model performance drifts, usage patterns shift and costs fluctuate. Build dashboards and review them monthly.

The bottom line

The gap between AI demo and AI production isn't a technology problem. It's a readiness problem. Organisations that treat the journey as a structured sequence of gates, rather than a single heroic push, ship faster and with fewer surprises.

The five gates (data, risk, platform, product, change) aren't revolutionary. They're just disciplined. And discipline, not novelty, is what separates organisations that talk about AI from those that run on it.

Building Responsible AI Applications with Azure AI Foundry

Fri, 09 May 2025 17:41:08 GMT

What Is a Responsible AI App?

A responsible AI app is an AI-powered application that delivers intelligent outcomes safely and with integrity. In practice, this means the app’s behaviour aligns with human values and ethical principles including things like treating people fairly, respecting privacy, and avoiding causing harm. The goal is to keep people and their goals at the center of the AI’s design and to uphold enduring values like fairness, reliability, transparency, and privacy . For example, a responsible AI customer service chatbot would not only provide useful answers, but do so accurately, securely, and without bias, ensuring all users are treated equally.

In short, a responsible AI app isn’t just about smart features, it’s about delivering those features in a way that users and society can trust. The app should proactively mitigate risks and unintended consequences, ensuring its intelligence is used for good, not inadvertently for harm. It’s the difference between an AI that can do something and an AI that should do it in a safe, ethical manner.

Why Responsible AI Matters

Ensuring AI is responsible is crucial for several reasons. Trust is number one as users will only adopt AI solutions if they trust them to behave ethically and reliably. An AI that produces biased results or unsafe advice can quickly erode user confidence. Worse, if an AI system is careless, it can lead to ethical or legal disasters. For instance, AI applications that are not designed carefully might generate fabricated, ungrounded outputs or even toxic content, leading to poor user experiences or harmful societal impacts . Responsible AI practices help avoid such outcomes by building in safety checks and balances.

Ethics and safety are also paramount. AI systems can impact people’s lives in areas like healthcare, finance, or employment, so mistakes or unfairness can have serious consequences. Responsible AI development forces us to consider questions like "Could this system hurt or discriminate against someone?" By embedding ethical principles, we reduce risks like perpetuating stereotypes, violating privacy, or suggesting dangerous actions. This not only protects users, but also protects organisations from reputational damage and liability.

Finally, responsible AI is key to getting real value in production. An AI app that works great in the lab can fail in the real world if it hasn’t been vetted for safety and reliability. Issues like hallucinated answers, biased decisions, or security vulnerabilities will prevent an AI solution from delivering sustained value. By investing in responsible AI (through robust testing, transparency, oversight, etc.), organisations ensure their AI innovations actually solve problems rather than create new ones. In essence, responsibility equals sustainability for AI – it’s how we ensure our AI solutions remain beneficial and compliant over time.

Microsoft’s Responsible AI Principles

Microsoft upholds six key principles to guide the development and use of AI. Each principle comes with a guiding question to consider when building AI systems.

Fairness: AI systems should treat all people fairly. Guiding question: “How might an AI system allocate opportunities, resources, and information in ways that are fair to the humans who use it?” In other words, ensure your AI doesn’t give unfair advantage or disadvantage to any group.
Reliability and Safety: AI systems should perform reliably and safely under all expected conditions. Guiding question: “How might a system function well for people across different use conditions and contexts, including ones it wasn’t originally intended for?” Think about stress cases and whether the AI can handle unexpected inputs or situations without failing or causing harm.
Privacy and Security: AI systems should be secure and respect privacy. Guiding question: “How might a system be designed to support privacy and security?” This reminds us to safeguard user data, use it responsibly, and prevent unauthorised access or data leaks in our AI applications.
Inclusiveness: AI systems should empower everyone and engage people of all abilities and backgrounds. Guiding question: “How might a system be designed to be inclusive for people of all abilities?” This involves designing AI that works for users with diverse needs – for example, accessible interfaces and avoiding bias against any demographic.
Transparency: AI systems should be understandable. Guiding question: “How can we ensure that people correctly understand the capabilities of a system?” This means being open about what the AI can and cannot do, and explaining its reasoning or limitations. Users should know they are interacting with AI and have insight into how decisions are made.
Accountability: People should be accountable for AI systems. Guiding question: “How can we create oversight so that humans can be accountable and in control?” Even when AI automates a task, a responsible organisation provides human oversight and takes responsibility for the AI’s outcomes. There should be avenues for feedback, appeal, or intervention if the AI makes a mistake.

By considering these guiding questions during development, teams can align their AI apps with these principles. For example, asking the fairness question might lead you to balance your training data or audit outcomes for bias. The accountability principle might inspire you to create an oversight process or an incident response plan. Together, these principles form a framework for trustworthy AI and they set a high bar for integrity and excellence in AI systems.

Learning from Real-World AI Failures

It’s helpful (though often humbling) to examine what can go wrong when AI systems are deployed without sufficient responsibility. Here are three real-world cases that underscore the risks of neglecting responsible AI design:

Air Canada’s Chatbot Misadventure: Air Canada deployed an AI chatbot to assist customers, but it gave a passenger disastrously incorrect information. When a customer asked about a bereavement fare, the bot misled him into buying a full-price ticket with the promise of a refund that the airline’s policy didn’t actually allow. The result? The customer was denied a refund, and Air Canada ended up in a legal dispute. In a remarkable defense, Air Canada argued the chatbot was a “separate legal entity” responsible for its own actions, a claim the tribunal flatly rejected. The airline was ordered to compensate the customer. This case highlights Accountability and Reliability failures: the company failed to oversee its AI (no human in the loop to catch the mistake), and tried to shirk responsibility. It’s a wake-up call that companies are accountable for their AI’s mistakes, and that reliable information is not optional. The chatbot needed better knowledge grounding or updates to avoid giving false assurances. Neglecting that led to customer harm and legal liability.
NYC’s “MyCity” Chatbot and Illegal Advice: New York City piloted an AI chatbot to help entrepreneurs navigate city rules, but the experiment showed how a well-intentioned AI can turn into a liability when not properly constrained. Investigations found the MyCity assistant was cheerfully telling users to break the law. For example, it advised that it was legal to lock out tenants and that there were no limits on rent increases, both of which are blatantly false under NYC laws. It also suggested restaurants could refuse cash (wrong, NYC law requires accepting cash) and that employers could take a cut of workers’ tips (also wrong). These are fundamental issues where the AI confidently provided misinformation that could have led users to illegal behavior. City officials defended the pilot as a learning experience, but acknowledged the errors and the need to fix them. The lesson here is about Reliability, Safety, and Transparency. If an AI system is answering legal questions, it must be grounded in up-to-date, correct information perhaps via a knowledge base or retrieval system and ideally should warn users that its advice isn’t legal counsel. Without those safeguards, the chatbot not only confused users but exposed its creators to massive trust and legal risks. An AI that isn’t safe or in this case legally accurate can do a lot more harm than good.
Character.AI Lawsuit – When AI “Companions” Go Awry: Character.AI is a platform that lets users chat with custom AI personas. In 2024 it became the target of a landmark lawsuit after two families alleged the AI chats harmed their children. The complaint describes shockingly inappropriate behaviour by the AI: a 17-year-old was encouraged by the chatbot to self-harm, with the bot gleefully describing how “it felt good,” and even sympathising with a scenario of a child killing his parents when the teen was upset about screen time limits. A 9-year-old child using the app was exposed to explicit sexual content, well beyond age-appropriate conversation. According to the lawsuit, these weren’t random one-off “glitches” and the suit claims the AI engaged in ongoing manipulation and abuse of the teens, inciting violence and self-harm. This is a chilling example of an AI safety failure. The company is now embroiled in a legal battle, accused of neglecting to filter or moderate harmful content and of marketing the app as safe for young users when it clearly was not. The Character.AI case underlines the absolute necessity of content safety filters, age-appropriate restrictions, and robust monitoring. Without these, AI that is meant for “fun” or companionship can cross dangerous lines, leading to real psychological harm and legal consequences. It’s a stark reminder that AI ethics aren’t optional and if you launch AI systems into the world, you must anticipate misuse and actively prevent harm, especially to vulnerable users.

Each of these failures could likely have been avoided (or their damage mitigated) with a more responsible AI approach: better testing for bad scenarios, clearer human oversight and accountability, and built-in safeguards like content filters or grounded knowledge. They illustrate why Microsoft’s principles (fairness, safety, etc.) aren’t just abstract ideals but very practical checkpoints that can prevent real disasters.

Introducing Azure AI Foundry: A Platform for Responsible AI Development

To help developers build AI apps responsibly, Microsoft has introduced Azure AI Foundry which is an all-in-one pro-code platform designed to simplify AI application development with responsible AI practices baked in. Azure AI Foundry provides a unified environment where you can explore models, build and test AI solutions, and deploy them at scale, all while leveraging enterprise-grade infrastructure and tools . It essentially combines what you need for AI development (models, code, data, etc.) with what you need for AI governance and reliability, in one place.

Key capabilities of Azure AI Foundry include an extensive model catalog (covering many top-tier models from OpenAI GPT-4 to open-source models), integrated dev tools, and project collaboration features. But what really sets it apart is the focus on operationalising Responsible AI. As Microsoft describes it, Foundry helps teams “explore, build, test, and deploy using cutting-edge AI tools and ML models, grounded in responsible AI practices.” In practical terms, this means when you build a chatbot or generative app in Foundry, you have access to features that evaluate your model’s behaviour, ensure compliance, and support transparency and monitoring.

Another way to think of Azure AI Foundry is as the DevOps hub for AI with guardrails. It supports the full lifecycle: from experimenting with different models, to incorporating knowledge sources, to testing output quality, and finally deploying and monitoring the app in production with oversight. Importantly, it’s not limited to Microsoft’s models and you can plug in models from multiple providers, but it provides a consistent set of tools to manage them responsibly. For enterprises concerned about AI risks, Foundry offers a path to innovate with AI confidently, knowing there are checks in place for things like quality, safety, and compliance.

Responsible AI Features in Azure AI Foundry

How does Azure AI Foundry support responsible AI development? Let’s look at some of its key features and tools that help developers build safer, more trustworthy AI apps:

Model Comparison and Evaluation: Foundry makes it easy to compare different AI models and choose the right one for your needs. You can connect to multiple model endpoints and “easily compare models without changing your underlying code”. Foundry includes built-in benchmark evaluations where you can test models on datasets and measure metrics like accuracy, coherence, and even bias. During evaluation, you can assess key quality metrics such as how relevant or fluent the responses are, and importantly whether the responses are grounded in the input or context. For example, if you’re building a Q&A bot that should stick to a knowledge base, Foundry’s evaluation can measure groundedness i.e., does the model’s answer stay true to the provided reference content? By systematically evaluating models, you can pick one that performs well and aligns with responsible AI criteria (like not generating unsafe content too frequently). This proactive model “shopping” helps prevent issues up front.

Groundedness and Content Safety Checks: When your AI app is up and running, Foundry can continuously check the outputs for certain qualities. Groundedness checks ensure the AI isn’t hallucinating information beyond what it should know. Additionally, Foundry provides automated content safety checks on inputs and outputs. In fact, Azure AI Foundry includes a content filtering system powered by Azure AI Content Safety, which runs both the user prompt and the AI’s completion through classifiers designed to detect harmful content. This means if a user asks something or the model responds with something that’s hate speech, violent, sexual, or self-harm related, the system can flag or block it. These content filters go beyond just relying on the model’s own guardrails; they’re a separate safety net. By default, many models in Foundry’s catalog have content filtering enabled (for instance, Azure’s own deployed models have a default filter). As an example, if someone tries a prompt that would make the model produce a disallowed output, Foundry’s filter can catch it and prevent that content from reaching the end-user . This is exactly the kind of protection that might have prevented the Character.AI scenario – e.g., detecting encouragement of self-harm or violence and stopping it. Foundry basically bakes in content moderation, which is essential for any responsible AI app that interacts with open-ended user input.

System Message Configuration: In generative AI (like chatbots), the system message (or meta-prompt) is a powerful tool to shape the AI’s behaviour. Azure AI Foundry encourages thoughtful system message design as part of responsible AI. The system message is where you define the AI’s role, tone, and instructions that it must follow, which can include ethical boundaries. According to Microsoft’s guidance, “System message design and proper data grounding are at the heart of every generative AI application”, giving the model context and rules to follow. Foundry lets you configure this easily for your projects. For example, you might include in the system prompt: “You are an AI assistant that follows Microsoft’s responsible AI guidelines. You do not provide disallowed content, you don’t speculate if you don’t know the answer, you explain your reasoning clearly,” etc. In fact, Microsoft provides sample system message instructions to mitigate content risks such as rules for handling requests for copyrighted material, or instructions to never speculate or provide ungrounded answers. By setting a solid system message, developers can hardwire ethical behaviour into the AI’s initial instructions. Foundry’s interface and SDK help manage these prompts as part of your application configuration, ensuring that every conversation the AI has starts with the right guidelines.

Evaluations and AI Reports: Azure AI Foundry also helps with reporting and monitoring the AI’s behaviour over time. As you test your AI Foundry can generate evaluation reports – essentially, dashboards or logs that show how the AI is performing on key metrics like groundedness, relevance, and safety. Developers can use this information to create reports and even set up alerts for anomalies or issues, sharing these insights with stakeholders . For instance, you might run a suite of test prompts through your chatbot and get a report that, say, 98% of the time it’s giving good answers, but 2% of the time it produced something potentially biased. You could then drill down into those cases. Foundry’s integrated application tracing and evaluation capabilities help trace the execution flow of AI responses and identify where things might have gone wrong . This level of transparency is important for accountability as it means you’re not blindly deploying an AI and crossing fingers; you’re continuously auditing it. And if something does slip through you have the tools to trace that conversation and improve the system. Essentially, Foundry supports creation of an “AI report” or dashboard for your AI system’s health, which is invaluable for responsible operation.

In summary, Azure AI Foundry provides a responsible AI toolchain: from choosing a model that’s fit-for-purpose, to grounding it with the right data and prompts, to filtering out bad outputs, and finally to monitoring and improving it with evaluations and reports. It’s much easier for developers to do the right thing when these capabilities are readily available. Instead of having to stitch together custom safety mechanisms, you get them out-of-the-box, which lowers the barrier to entry for responsible AI development.

Better AI-Human Interaction Design with the HAX Toolkit

Responsible AI isn’t just about models and code – it’s also about how AI systems interact with humans. Users need to understand and effectively use AI, and the AI’s UX design plays a huge role in that. Microsoft’s Human-AI eXperience (HAX) Toolkit is a set of resources to help teams design AI interactions that are intuitive, trustworthy, and aligned with human needs. In other words, HAX helps you build AI products with a human-centered approach from the design perspective.

The HAX Toolkit is aimed at teams building user-facing AI products, and it’s best used early in the design process . It currently includes four main components:

Guidelines for Human-AI Interaction: These are best practices for how AI systems should behave during interaction with people . Microsoft Research distilled these guidelines from years of study; they cover things like how an AI should initiate interactions, how it should convey its confidence or uncertainties, how it should handle errors or ambiguity, and how to make its capabilities and limits clear. Designers and developers can use these guidelines as a checklist when planning an AI feature. For example, one guideline is to “make clear why the AI did what it did” which might translate to providing explanations for an AI-driven recommendation. The Guidelines ensure you don’t overlook key UX considerations that affect user trust.
HAX Design Library: The design library is a collection of design patterns and examples that illustrate the Human-AI Interaction guidelines in practice . It’s one thing to read a principle like “show contextually relevant information,” but another to see how leading products implement that. The HAX Design Library lets you explore UI patterns, screenshots, and case studies that align with each guideline. This helps teams learn concretely how to apply the guidelines. For instance, if one guideline is to allow users to correct the AI, the design library might show pattern examples of an “undo” or “edit” feature next to AI-generated content. By leveraging these patterns, designers can create interfaces that embody responsible AI principles (like transparency and control) in a user-friendly way.

Source: https://www.microsoft.com/en-us/haxtoolkit/ai-guidelines/

HAX Workbook: The workbook is a team discussion and planning tool . It provides a structured way for a multidisciplinary team (product managers, designers, engineers, etc.) to sit down together and decide how to prioritize and implement the guidelines for their specific product. The HAX Workbook essentially prompts the team to answer questions about the user experience: Which guidelines are most critical for our use case? How will we handle scenarios where the AI is wrong? Do we need to inform users that AI is involved in this feature? By working through the workbook, teams ensure they have thought through potential interaction failures or ethical challenges. It helps break down silos and everyone gains a shared understanding of how the AI should behave and what to watch out for.
HAX Playbook: The playbook is a tool for identifying common failures in human-AI interactions, specifically for applications using natural language processing (NLP) . Think of it as a catalog of “what can go wrong” when humans chat or interact with AI, along with strategies to mitigate those issues. For example, a common failure is the AI misunderstanding the user’s intent – the playbook would call that out and suggest mitigation like confirming with the user or offering options. Another failure might be the user over-trusting the AI’s answer, so the playbook would encourage you to consider showing uncertainty or linking to source data. By systematically exploring these failure modes, teams can plan features to prevent them (thus improving reliability and user satisfaction). Essentially, the HAX Playbook helps you proactively debug the human-AI interaction before your product ever launches.

Source: https://microsoft.github.io/HAXPlaybook

Using the HAX Toolkit, teams can create AI experiences that are not only effective but also comfortable and trustworthy for users. It’s one thing for an AI model to be state-of-the-art, but if the user interface causes confusion or the AI behaves in opaque ways, users can still end up frustrated or misled. HAX Toolkit addresses this gap by focusing on UX and interaction design. For instance, by following the Guidelines and using the Workbook, a team building a virtual assistant might decide to always provide a justification for a recommendation (addressing Transparency), or to include a “Was this helpful?” feedback mechanism (addressing Accountability by letting users flag issues). These design elements greatly enhance the overall responsibility of the AI app, complementing the technical safeguards.

Conclusion

Building responsible AI applications is a multifaceted challenge, it spans technical mitigation (like filters and benchmarks) and thoughtful design (like user interaction guidelines). The good news is that we don’t have to reinvent the wheel. Microsoft’s Responsible AI principles give clear ethical targets to aim for, and their tools like Azure AI Foundry and the HAX Toolkit provide practical support to hit those targets. By learning from past mistakes (as seen in the chatbot failures above) and by leveraging these principles and tools, AI practitioners can develop applications that are not just innovative, but also trustworthy and respectful of the people they serve.

Responsible AI is really about caring for your users and the world, even as you dive into cutting-edge tech. As we implement fairness, safety, privacy, inclusiveness, transparency, and accountability in our AI projects, we’re essentially saying we want AI to benefit everyone and harm no one. It’s a journey and there will be bumps along the way, but with a strong framework and the right tooling, it’s absolutely achievable. And at the end of that journey lies AI that people can truly trust, adopt, and celebrate in their daily lives. That’s the kind of AI we all should aspire to create.

AI is Taking Action with Agents

Tue, 18 Mar 2025 06:23:35 GMT

Generative AI has been a hot topic for the last couple of years and recently it has taken a leap forward with the genesis of agentic AI. Agentic AI is a fundamental shift in how we perceive and utilise AI, moving beyond passive analysis and generation to proactive execution. Imagine AI that cannot only process and provide information but also make decisions and take actions autonomously, all while learning and adapting to its environment. This is the essence of agentic AI, a concept Gartner hails as a top technology trend for 2025 and McKinsey names the "next frontier" in AI research.

This is a game-changer across industries. In customer service, AI agents can resolve complex issues in real-time, without human intervention, leading to increased satisfaction and reduced operational costs. Gartner predicts that by 2028, 20% of all digital storefront interactions will be handled by AI agents. Imagine an AI agent that cannot only answer customer questions but also proactively offer solutions, such as suggesting relevant knowledge base articles or initiating refunds without needing explicit instructions. In supply chains, AI agents can predict demand fluctuations, adjust inventory levels, and optimise logistics, ensuring smooth operations and cost efficiency. Companies utilising AI in their supply chains reported a 20% increase in productivity and a 30% decrease in queue times, according to an HFS Research study. Imagine an AI agent that can autonomously adjust delivery routes based on real-time traffic conditions or automatically reorder stock when inventory levels run low.

In IT support, AI agents can identify and resolve issues before they escalate, minimising downtime and maximising productivity. Imagine an AI agent that can detect a server outage and automatically reroute traffic to prevent service disruptions. In healthcare, AI agents can analyse patient data, recommend treatment plans, and even assist with surgeries, leading to improved patient outcomes and more efficient healthcare delivery. Imagine an AI agent that can analyse a patient's medical history and suggest personalised treatment options or even assist surgeons during complex procedures.

Consider the insurance industry, often bogged down by paperwork and manual processes. Agentic AI can streamline the entire claims process, from initial filing to final payout, resulting in faster processing times, reduced errors, and a vastly improved customer experience . Or imagine the impact on human resources, where AI agents can automate tasks such as policy queries, payroll management, and employee support, freeing up HR professionals to focus on more strategic tasks, such as talent development and employee retention.

While both agentic AI and generative AI are branches of artificial intelligence, they have distinct focuses. Generative AI excels at creating new content, such as text, images, and music. Agentic AI, on the other hand, is action-oriented. It focuses on “doing” rather than "creating." There is confusion about the difference between agentic AI and workflows and how they fit together. Workflows are systems where LLMs and tools are orchestrated through predefined code paths whereas agents are more dynamic. Agentic AI and workflows are not mutually exclusive. In fact, they can work together seamlessly. Agentic AI can be integrated into workflows to automate complex processes that require decision-making and adaptability. This creates a dynamic and efficient system where AI agents handle tasks autonomously while humans focus on higher-level strategic initiatives.

Microsoft is at the forefront of agentic AI development with tools like Copilot Studio and Azure AI Foundry. Copilot Studio allows businesses to build and deploy custom AI agents with ease, while Azure AI Foundry provides a comprehensive platform for developing and managing AI solutions at scale. These tools empower businesses to create AI agents that can automate tasks, improve decision-making, and enhance customer experiences.

Copilot Studio is a low-code platform that allows you to build and deploy custom AI agents. You can use Copilot Studio to create agents that can answer questions, perform tasks, and automate workflows. Copilot Studio also offers generative AI features that can help you create agents more quickly and easily . With Copilot Studio, you can create agents from scratch or customise pre-built agents, design agents using a low-/no-code interface or natural language prompts, connect agents to data sources using pre-built or custom plugins, and publish agents for various environments, including Microsoft365 apps .

Microsoft Copilot Studio

Azure AI Foundry is a comprehensive platform for building and deploying AI solutions. It provides a wide range of tools and services for developing, customising, and managing AI solutions including agents. Azure AI Foundry also offers enterprise-grade security and compliance features and being an Azure service works seamlessly with other Azure services, making it suitable for businesses of all sizes. Azure AI Foundry offers:

A unified AI-native platform: This allows enterprises to design, test, deploy, and monitor AI models within a single environment.
Comprehensive platform: A rich catalog of over 1,800 models, including frontier, open-source, industry-specific, and task-based AI models.
A single API for AI development: This eliminates the need to integrate different tools and simplifies the development process.
A built-in observability and monitoring framework: This ensures real-time performance tracking and allows for proactive optimisation.
Enterprise-grade security, compliance, and governance: This makes it suitable for regulated industries and ensures data privacy.

Azure AI Foundry

These two powerful services from Microsoft provide a comprehensive suite of tools for building, deploying, and managing agentic AI solutions. Whether you're looking to create a customer service agent, develop an AI-powered workflow, or build a smart assistant, Copilot Studio and Azure AI Foundry offer the capabilities you need to succeed.

Microsoft is also introducing several of their own AI-powered agents to enhance workplace productivity and collaboration5. The Interpreter agent in Teams provides real-time speech-to-speech translation during meetings, even replicating the speaker’s voice for a more natural experience, with a public preview expected in early 2025. The Employee Self-Service Agent in Business Chat streamlines HR and IT processes by answering policy-related questions and facilitating key tasks like benefits inquiries or laptop requests, with customisation options in Copilot Studio, currently in private preview. Additionally, the Facilitator agent automates note-taking in Teams meetings and chats, while the Project Manager agent assists with task automation and plan creation in Microsoft Planner and both are available in public preview.

Agentic AI is still in its early stages, but its potential is undeniable. As AI agents become more sophisticated, they will play an increasingly important role in our lives, transforming industries and redefining the future of work. Gartner predicts that by 2028, 33% of enterprise software applications will incorporate agentic AI, a significant increase from less than 1% in 20246. At Versent, we're excited to be part of this transformative journey, helping businesses harness the power of agentic AI to achieve their goals and unlock new possibilities.

However, it's important to acknowledge the potential challenges associated with agentic AI. As with any emerging technology, there are risks and considerations that need to be addressed. One concern is the potential for job displacement as AI agents automate tasks previously performed by humans. This requires proactive strategies for reskilling and upskilling the workforce to adapt to the changing job market. Another challenge is ensuring the responsible and ethical use of agentic AI. As AI agents become more autonomous, it's crucial to establish clear guidelines and governance frameworks to prevent unintended consequences and ensure alignment with human values.

Despite these challenges, the potential benefits of agentic AI are too significant to ignore. By embracing this technology responsibly and strategically, businesses can unlock new levels of efficiency, productivity, and innovation. At, we're committed to helping our clients navigate the complexities of agentic AI and harness its power to drive positive change.

Unlocking Developer Productivity with Microsoft Dev Box

Sun, 20 Oct 2024 21:43:58 GMT

In today's fast-paced development environment, where speed, flexibility, and security are paramount, having a reliable, ready-to-use development workspace is crucial. Enter Microsoft Dev Box, a cloud-based developer workstation designed to streamline development workflows and boost productivity for developers and engineers. Let’s explore how this platform transforms the way developers work.

What is Microsoft Dev Box?

Microsoft Dev Box is a cloud-based, self-service developer workstation that allows developers to instantly access fully configured environments tailored to their specific needs. By providing pre-configured setups for projects, developers can begin working immediately, skipping the lengthy setup processes typically associated with traditional workstations. The platform leverages the power of Azure to ensure that every aspect of the developer workspace—tools, source code, applications, and security configurations—is ready from the start, significantly reducing the time it takes to onboard new projects.

Designed to be both flexible and secure, Microsoft Dev Box enables teams to scale easily, supporting multiple configurations for different projects and teams. By allowing developers to manage their own workstations, the platform reduces reliance on IT support, giving developers the freedom to control their environments without compromising security.

Why Microsoft Dev Box is a Game-Changer

The platform is built on several pillars that make it stand out from other cloud-based workspace solutions:

Efficiency and Consistency: With Microsoft Dev Box, developers no longer need to wait for IT teams to provision environments or manually configure workstations. Pre-configured templates and images are readily available, which drastically reduces setup time and ensures consistency across development environments.
Accessibility from Anywhere: Microsoft Dev Box supports remote work by allowing developers to access their workstations from anywhere using any device. Whether they’re working from a Windows, macOS, iOS, or Android device, developers can log into their cloud-based environment using remote desktop clients or a browser. This accessibility is a key advantage for distributed teams or developers who switch between devices.
Enhanced Security: In an era where security is paramount, Microsoft Dev Box integrates seamlessly with existing security frameworks. Using tools like Microsoft Intune, Azure Firewall, and Entra ID, organisations can manage security across all devices with features like multi-factor authentication (MFA), conditional access, and zero-day patch deployment. Moreover, because the development environment resides entirely in the cloud, sensitive intellectual property and code are never stored on local devices, reducing the risk of data breaches.
Scalability and Flexibility: The platform’s ability to support multiple configurations means that teams can scale efficiently. Whether a team is working on one large project or multiple smaller ones, developers can easily spin up different environments tailored to specific tasks without having to go through a complex provisioning process.

Key Benefits for Developers, Admins, and Security Teams

The advantages of Microsoft Dev Box extend beyond just the developers. It benefits various stakeholders across the organization:

For Developers and Engineers:

Improved Productivity: Pre-configured workstations come loaded with all the tools, libraries, and assets needed for specific projects. This means that developers spend less time configuring their environments and more time writing code.
Powerful Resources: Dev Box offers configurations starting at 8 vCPUs and 32 GB of RAM, with options going up to 32 vCPUs and 128 GB of RAM. This ensures that even the most resource-intensive tasks can be handled smoothly.
Self-Service and Autonomy: Developers can create and manage multiple Dev Boxes without needing IT intervention, giving them the freedom to work on multiple projects or tasks simultaneously.

For Platform Administrators:

Simplified Management: With centralized management through the Dev Center, administrators can manage Dev Box configurations, network connections, and security protocols for multiple projects. This reduces administrative overhead and ensures that environments are consistent and compliant.
Streamlined Operations: Dev Box integrates with familiar Microsoft tools like Intune, making it easy for administrators to manage security, patches, and compliance across all Dev Boxes alongside other company devices.

For Security Teams:

Integrated Security Tools: Microsoft Dev Box offers multi-layered security by leveraging tools like Azure Firewall, Entra ID, and Microsoft Intune to provide endpoint, network, and identity security. Dev Boxes are also enrolled in Intune automatically upon provisioning, ensuring they are managed securely from day one.
Confidence in Compliance: Compliance policies can be applied to ensure that all Dev Boxes are up-to-date with the latest patches and security configurations, reducing the risk of vulnerabilities.
Supports existing tools: Because Dev Box is a Windows virtual machine you can install all the same tools you would install on other corporate Windows devices and they'll work seamlessly.

Pre-Configured Workstations: A Boost to Productivity

One of the most significant time-saving features of Microsoft Dev Box is the ability to pre-configure workstations. Organizations can create Dev Box images that include all the tools, source code, applications, and settings required for specific projects. Once the image is ready, developers can start working on code immediately, skipping the need for manual configuration. This capability significantly reduces the time spent on setup and onboarding, especially for new hires or new project teams.

Dev Boxes can also be customized for different roles or tasks within a project. For example, a front-end developer might have a Dev Box with specific UI design tools, while a back-end engineer might have a configuration tailored for database management or API development. This level of customization ensures that each developer has the right tools for the right job, boosting both individual and team productivity.

Self-Service and Flexibility for Developers

Developers often need to work on multiple projects at once, and with traditional IT-managed workstations, switching between projects can be cumbersome and time-consuming. Microsoft Dev Box offers a solution to this with its self-service model, allowing developers to create, manage, and use multiple workstations as needed. This flexibility means that a developer can spin up a new Dev Box in minutes, without waiting for IT provisioning.

The self-service nature of Microsoft Dev Box doesn’t compromise on security. Azure's Role-Based Access Control (RBAC) ensures that developers only have access to the resources they need, and network security is managed through Azure Virtual Networks. This ensures that even though developers have the freedom to manage their environments, they are doing so within a secure framework.

Scalability and Cost Management

As organizations grow, so do their development needs. Microsoft Dev Box is designed to scale effortlessly, supporting distributed teams and multiple configurations across different regions. Whether your team is centralized or spread across the globe, Dev Boxes can be deployed closer to where developers are working, ensuring high performance and low latency.

One of the key benefits of Dev Box is its cost efficiency. With auto-stop schedules and hibernation features, organizations can ensure they are not overpaying for unused resources. Dev Boxes can be provisioned only when needed, and a monthly cost limit ensures that you won’t go beyond a set budget, regardless of how much the service is used. This combination of scalability and cost control makes Microsoft Dev Box an attractive option for organizations of all sizes.

Security and Compliance with Microsoft Intune

Microsoft Dev Box integrates closely with Microsoft Intune to provide enterprise-grade security and management features. Upon provisioning, Dev Boxes are automatically enrolled in Intune, which simplifies device management. Administrators can apply conditional access policies, ensuring that only compliant devices have access to sensitive environments.

Security features include endpoint protection with built-in antivirus, threat detection, and firewalls to ensure that Dev Boxes are protected against potential vulnerabilities. Additionally, organisations can deploy configuration profiles for automatic patching and schedule expedited updates to ensure all Dev Boxes are secure and up-to-date.

The Future of Developer Workstations

Microsoft Dev Box is not just a cloud-based desktop—it’s a fully optimised development platform designed to enhance productivity, streamline operations, and fortify security. Its integration with existing tools like Azure, Intune, and Entra ID makes it easy to adopt, while its flexible configuration options meet the diverse needs of modern development teams.

For developers and organizations looking to innovate faster, cut operational overhead, and ensure secure, consistent work environments, Microsoft Dev Box represents a powerful solution. Whether you’re managing a small team or a large, distributed workforce, this cloud-based developer workstation platform is a game-changer for modern software development. As the industry continues to evolve, solutions like Microsoft Dev Box are poised to play a pivotal role in shaping the future of work.

Why You Should Care About GitHub Copilot Enterprise

Mon, 20 May 2024 06:51:30 GMT

Even though the name says enterprise, the reality is that the features this version of GitHub Copilot (just Copilot from here on in) brings are so impactful for developers and engineers that the name is almost misleading. You’d be forgiven for thinking Copilot Enterprise was all about additional security controls and options, some different licensing and support for SSO at first glance of the name. The reality is very different. The real value in the Enterprise edition of Copilot is how it impacts the time when developers are not writing code.

Developers only spend roughly 25% of their time writing code1,2. If you're a developer, I imagine this isn't a surprise to you. If you're a manager or business person this might come as a bit of a shock. Much like their business counterparts, technologists spend a lot of their time in meetings doing things like planning, gathering requirements, Agile ceremonies, designing, discussing potential solutions, helping others, responding to incidents and so on.

Copilot has revolutionised the 25% of their time that developers spend writing code, making them more productive3 and more fulfilled4. Now, with the features in the Enterprise edition, Copilot is taking aim at the other 75% of a developer's time, and helping bring increased productivity and fulfilment to more of their role.

How is it doing this? With these new features, only available in the Enterprise edition of Copilot:

Chat assistance integrated into GitHub.com
Search through your organisation's knowledge bases
Access the web's expertise with internet integration
Making pull requests easier with summaries and explanations

Chat assistance integrated into GitHub.com

The headline feature of Copilot Enterprise is the integrated chat on the GitHub.com website. This lets you interact with your repositories, projects, files, issues and more through chat window. It takes context from your organisation as well as the repository and file you have open, but doesn't always get it right. For example, when asking about a pull request (PR) I had to provide the PR number.

Regardless, the utility of this incredible. Think about when you're starting a new job or project, or trying to understand a new repository. Being able to have a conversation and ask questions just as you would speaking to a fellow engineer or developer is intuitive, and saves both your time and others. It also lets you ask whatever question you want without judgement. You can ask about tests, have the code explained to you in natural language, or ask for some recommendations on what to do next or how to improve parts of the code.

In this example, we're finding out about the terraform-azure-virtual-machine module, its tests and pipelines all through the chat interface. You'll notice it's unfiltered, and I had to reword my query about pipelines to get an answer. As with all things generative AI, it's not perfect, but it is still very useful!

💡

You'll notice this video is raw and unfiltered and includes an example of Copilot having a bit of trouble. I had to reword my query about pipelines to get an answer. As with all things generative AI, it's not perfect, but it is still very useful!

Search through your organisation's knowledge bases

Copilot Enterprise lets you search through your organisation's knowledge across repositories. It does this with the concept of knowledge bases, which bring together markdown documentation across repositories. This can then be used as context in a chat with Copilot.

For example, you can create a knowledge base of all your infrastructure-as-code modules, thereby having a single, combined view of all the documentation across those modules. When you chat with Copilot using this as context, you can discover other modules that you can use in your code and how to use them. That does mean however, that the modules need to be documented in the repository with markdown files, though.

Being able to access knowledge like this across repositories helps enable inner-sourcing, reduces re-work and double-handling, and helps developers find and use resources that they otherwise wouldn't be aware of or would recreate themselves. The potential for efficiency gains from this in a large enterprise is massive, and the improvement to a a developers quality of life is significant.

Here's how to set up a knowledge base:

Here we're using the knowledge base to find out if we have a module that we can use to create some networking resources in Azure.

Access the web's coding expertise with Bing

What's a Copilot without internet access these days? Copilot for Enterprise is integrated with Bing to retrieve internet results when a user asks about recent events, new trends or technologies, specific subjects, or if they explicitly ask for a web search. It's convenient to have this built in, so you don't have to leave your existing window to run an internet-enabled prompt with ChatGPT or Microsoft Copilot. It lets you stay in the flow, which is incredibly important for developer productivity and one of the core tenets of Copilot. It also means that your query is informed by the context of your previous conversation, helping you get more accurate results. Those results also then include references, so that you can make sure the response Copilot is giving is accurate.

It's important to note that this obviously requires your prompt to be sent to the Bing API to execute the search, thereby leaving the GitHub environment. The feature is still in beta at the time of writing, but worked quite well when I experimented with it.

It's also good to know that this is only the first integration Copilot has, with GitHub promising that there will be more to come.

In this example I'm using it to find out about some of the differences in Terraform versions.

Making pull requests easier with summaries and explanations

Pull requests (PRs) are fundamental to productive collaboration on code. There's a lot of opinions on how to do PRs "right", ranging from naming standards, to the size and frequency of PRs.

One of the important things to get right with any PR is the summary. This short blurb is most reviewer's starting point and describes concisely what work has been done in the PR. Copilot can generate these summaries for you in natural language while reference the code blocks that have been changed. This saves so much time both when creating and reviewing a PR.

What would be great to see next is support for PR templates. Many teams use this to make their PR summaries consistent and to ensure their processes are followed. Being able to generate PR summaries that honour a template would immensely helpful for private repos, but even more so for open source.

In this video we use Copilot to create the summary of our pull request, with no input other than the changes that have been committed.

This is already very helpful, both for those raising pull requests who no longer have to spend time thinking of and crafting a good summary as well as those reviewing pull requests who can get a clear and concise summary of what they need to review. If they still need some help though, Copilot can help with reviewing the PR via the chat interface, as shown in this clip.

Again, this is unfiltered and shows how Copilot needs some help knowing which PR you are referring to.

Conclusion

GitHub Copilot Enterprise is a transformative tool that significantly enhances developer productivity and satisfaction. While Copilot has already revolutionized the 25% of a developer's time spent writing code, the Enterprise edition expands this impact to the remaining 75% of their activities.

With features like integrated chat assistance on GitHub.com and mobile, the ability to search through your organisation’s knowledge bases, internet access via Bing, and enhanced pull request handling, Copilot Enterprise streamlines numerous aspects of a developer's workflow. These innovations not only save time and reduce redundancy but also provide intuitive support, making onboarding and project comprehension much more straightforward.

Ultimately, GitHub Copilot Enterprise empowers developers by addressing the myriad tasks that occupy their time beyond just coding. By doing so, it fosters a more efficient, informed, and collaborative working environment, making it an invaluable asset for any development team. The future promises even more integrations and capabilities, further solidifying Copilot's role as an essential tool in modern software development.

1: Code Time Report
2: How much time do developers spend actually writing code?
3: Research: Quantifying GitHub Copilot's Impact on Developer Productivity and Happiness
4: Research: Quantifying GitHub Copilot’s impact in the enterprise with Accenture

Per Item RBAC in Azure Key Vault

Wed, 16 Nov 2022 22:00:00 GMT

TL;DR

If you just need a refresher or really need to know how to do this fast, here it is for secrets.

Assign the identity that needs to read one or more secrets the Key Vault Readerrole at the scope of the Key Vault.
Assign that same identity Key Vault Secrets Officer at the scope of the secret(s) you want it to be able to retrieve the value of.

⚠️

The identity you grant these permissions to will be able to see all the secrets in the Key Vault you grant them access to, but it will only be able to read the value of secrets that have the Key Vault Secrets Officer role assigned to that identity.

Why RBAC for Key Vault Instead of Access Policies?

Traditionally controls in Azure Key Vaults were applied using access policies. The problem with these was that the permissions we granted applied to all objects of that type within a given Key Vault.

For example, if we granted an identity List and Get permissions on secrets in a given Key Vault, that identity could list all of the secrets in that Key Vault and read the value of all those secrets. There was no way to limit the scope of an identity to read only some, not all, secrets. The same applies to keys and certificates.

RBAC for Key Vault gives us the benefit of these more fine-grained controls along with a few added benefits such as support for Privileged Identity Management (PIM) and custom roles. It also brings Key Vault access control in line with the RBAC implementation in place across Azure more broadly.

💡

An Azure subscription supports 4,000 role assignments, whereas access policies support up to 1,024 policies per Key Vault. Something to keep in mind when using RBAC for Key Vault at a large scale.

How to Set up RBAC for Key Vault

The first thing we’ll need to do is either create a new Key Vault or change an existing one to use RBAC for Key Vault instead of access policies. Note that RBAC for Key Vault and access policies cannot co-exist, we have to pick one or the other.

To change an existing Key Vault to use RBAC instead of access policies open the Key Vault in the Azure portal and open up Access configuration and change the selection from Vault access policy to Azure role-based access control then click the Apply button.

⚠️

Make sure you have the necessary role assignments in place before switching to from access policies to RBAC or things will lose access to your Key Vault. There's a big warning about this on the page when you go to make the change as well.

If setting up a new Key Vault we just need to make sure we choose Allow role-based access control at the access policy stage of the wizard. If using the Azure CLI set the --enable-rbac-authorization flag to true.

Granting Access with RBAC for Key Vault

We’re going to focus on using the built-in roles in this post, but if those aren’t suitable for your needs you can always create a custom role with only the actions you need and apply it in the same way.

There are several built-in roles for working with RBAC for Azure but we’ll be working with the Key Vault Reader and Key Vault Secrets Officer roles. They’re highlighted in the list below.

The Key Vault Reader role is described in the portal as:
“Read metadata of key vaults and its certificates, keys, and secrets. Cannot read sensitive values such as secret contents or key material.”

And the Key Vault Secrets Officer as:
“Perform any action on the secrets of a key vault, except maange permissions.”

You can find the detailed permissions each role grants in the Azure Portal or using PowerShell or the Azure CLI. More on how to do that can be found here.

The first step is to apply the Key Vault Reader for our identity at the scope of the Key Vault. This will grant the identity we’re assigning the role to the ability to see the Key Vault if they can’t already, as well as all the secrets within the Key Vault. They won’t be able to read or copy the actual values of the secrets, but they will be able to see what secrets exist in the entire Key Vault. If this is going to be a problem in your case you’ll need to set up a separate Key Vault or look into custom roles.

With that in place we can now apply the Key Vault Secrets Officer for our identity at the scope of each secret we want that identity to be able to read the value of. To apply the role at the scope of a given secret, open that secret up in the Azure portal and on the left-hand side there will be an Access control (IAM) menu item, the same as there would be for the Key Vault itself and other Azure resource. You can then apply the Key Vault Secrets Officer role at the scope of the secret (which the portal will refer to as “this resource” in this context) in the exact same way you would apply roles in any other part of Azure.

You can also do this with the Azure CLI like so:

az role assignment create --role "Key Vault Secrets Officer" --assignee user@domain.com --scope /subscriptions/<subscription id>/resourcegroups/<resource group name>/providers/Microsoft.KeyVault/vaults/<key vault name>/secrets/<secret name>

Our identity can now only use the secrets that we have granted it access to with the Key Vault Secrets Officer role. It can see the other secrets exist, but cannot read the value of those secrets.

Deploy AMPLS with Terraform

Sun, 25 Jul 2021 23:00:00 GMT

Azure Monitor Private Link Scope (AMPLS) is a very convenient Azure service that amalgamates the various services required for Azure Monitor and makes them available via private endpoints within your virtual network (VNet). This enables completely private interactions with all aspects of the Azure Monitor service, from sending logs to workspaces and storage accounts to running queries. It also allows you to connect on-premises infrastructure to Azure Monitor over ExpressRoute. These features make AMPLS particularly attractive to enterprise and regulated consumers of Azure.

💡

AMPLS is not the focus of this article. To find out more about AMPLS and some of the potential gotchas, have a look at the Microsoft documentation.

If you try setting up AMPLS through the Azure portal, say for a proof of concept or to become familiar with the service, you’ll find the experience quite straightforward. It’s simple to set up the private link service and then add the private endpoints and DNS configuration required to use it. However, if you then start looking in to deploying this solution with Terraform, things become a bit more difficult. This article offers simple solutions to overcome the challenges you’ll face.

Hurdle Number One: Provider Support

The first issue is that at the time of writing (July 2021) the Terraform Azure provider does not support AMPLS. The main reason for this is that the feature is not yet supported by the Azure Go SDK. Once the feature exists in the Go SDK it can then be added to the Terraform provider.

💡

You can follow the GitHub issue for AMPLS support in the azurerm provider here.

This means we’ll need to use good old ARM templates to deploy AMPLS, the tried and true fall-back when something is not yet supported by the azurerm provider. If you haven’t run in to this scenario before then you’ll be pleased to know that the provider does offer a mechanism to deploy ARM templates, so you can still execute the deployment as part of your Terraform infrastructure code. Here’s the code to do this.

resource "azurerm_resource_group_template_deployment" "this" {
  name                = "ampls"
  resource_group_name = azurerm_resource_group.example.name
  deployment_mode     = "Incremental"
  parameters_content  = jsonencode({
    "private_link_scope_name" = {
      value = "ampls_private_scope"
    }
    "workspace_name" = {
      value = azurerm_log_analytics_workspace.this.name
    }
  })
  template_content = <<-EOT
    {
      "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
      "contentVersion": "1.0.0.0",
      "parameters": {
        "private_link_scope_name": {
          "defaultValue": "my-scope",
          "type": "String"
        },
        "workspace_name": {
          "defaultValue": "my-workspace",
          "type": "String"
        }
      },
      "variables": {},
      "resources": [
        {
          "type": "microsoft.insights/privatelinkscopes",
          "apiVersion": "2019-10-17-preview",
          "name": "[parameters('private_link_scope_name')]",
          "location": "global",
          "properties": {}
        },
        {
          "type": "microsoft.insights/privatelinkscopes/scopedresources",
          "apiVersion": "2019-10-17-preview",
          "name": "[concat(parameters('private_link_scope_name'), '/', concat(parameters('workspace_name'), '-connection'))]",
          "dependsOn": [
            "[resourceId('microsoft.insights/privatelinkscopes', parameters('private_link_scope_name'))]"
          ],
          "properties": {
            "linkedResourceId": "[resourceId('microsoft.operationalinsights/workspaces', parameters('workspace_name'))]"
          }
        }
      ],
      "outputs": {
        "resourceID": {
          "type": "String",
          "value": "[resourceId('microsoft.insights/privatelinkscopes', parameters('private_link_scope_name'))]"
        }
      }
    }
  EOT
}

Hurdle Number Two: Azure DNS Records

The second hurdle to be considered is DNS. There are two parts to this, the architectural problem and the infrastructure code problem. We’re going to focus on the latter here, but just be aware the you should put careful consideration in to how you configure DNS for AMPLS in the context of your environment. The Microsoft documentation on AMPLS covers these potential gotchas well.

The infrastructure code problem we face when trying to deploy AMPLS with Terraform is around replicating the outcome of deploying the service from the Azure portal. If you use the portal and create a private endpoint from the private link scope resource, the portal will offer an option to deploy the Azure Private DNS zones for you.

When you are creating the private endpoint and linking it to the private link scope in Terraform (or any infrastructure as code) this won’t be done automatically for you. You’ll need to create the DNS zones and A records yourself. The important thing to note here is that for the private endpoint to work properly you will need to assign particular IPs for each FQDN. The private endpoint will list the IP to be assigned to each FQDN if you take a look in the portal.

What’s happening here is that IPs from the private endpoint’s subnet are being allocated to each FQDN in a certain order. If we replicate that order we’ll be able to dynamically assign IPs from any give subnet in the same way the service would have if deployed via the portal. That means our AMPLS will work as expected and we’ll be able to deploy using automation and infrastructure as code.

We’ll need to use the Terraform cidrhost function which returns the IP address for a given host number within a subnet. So for example if we want the first host IP in the subnet 192.168.0.0/24:

cidrhost("192.168.0.0/24", 1)

From here we just need to know what order the IPs need to be allocated in to replicate the outcome of deploying via the portal so that each IP is assigned to the correct FQDN. Keep in mind that the first four IP addresses from every subnet are reserved in Azure. You’ll also need the workspace ID of the log analytics workspace you are sending your logs to. Note that this is the workspace ID which is different to the resource ID. With that in mind, this is the order in which the IPs need to be allocated.

Number	FQDN
4	<workspace id>.privatelink.oms.opinsights.azure.com
5	<workspace id>.privatelink.ods.opinsights.azure.com
6	<workspace id>.privatelink.agentsvc.azure-automation.net
7	api.privatelink.monitor.azure.com
8	global.in.ai.privatelink.monitor.azure.com
9	profiler.privatelink.monitor.azure.com
10	live.privatelink.monitor.azure.com
11	snapshot.privatelink.monitor.azure.com
12	scadvisorcontentpl.privatelink.blob.core.windows.net

With that resolved you’ll either need to assign the private DNS zones to any VNets that need to consume them, or update your DNS resolvers to forward requests for these zones to Azure DNS depending on your environment.

💡

The full Terraform code to deploy a demo of the AMPLS solution including AMPLS, private endpoints and DNS is available here. Note that this code is purely for demonstration only and is not production ready. You'll need to apply the solution to the context of your environment and extend it to meet your needs.

Conclusion

Now you’re ready to leverage the benefits of keeping all your monitoring traffic on private links and deploy it all using Terraform infrastructure as code. We’ve dealt with the lack of provider support with an ARM template deployment and resolved the DNS allocation issue as well. With the technical issues dealt with hopefully this frees you up to put more thought in to how AMPLS will fit in to your environment.

App Registrations, Enterprise Apps and Service Principals

Thu, 29 Apr 2021 23:00:00 GMT

It’s one of the confusing parts of Azure Active Directory (Azure AD) and something that many find difficult to understand properly. Often the terms are used interchangeably which only exacerbates the confusion. So what is the difference between an app registration, enterprise application and service principal in Azure AD?

Let’s start with the easy part - an enterprise application is a service principal. When you go to Azure AD in the Azure Portal and open up the list of Enterprise applications you’re looking at a list of service principals in your tenant.

I believe most of the confusion with enterprise apps and service principals comes from the term “enterprise apps” actually referring to applications published in the Azure AD gallery that you can use within your organization. This gallery is presented if you click the new application button in the enterprise applications blade of Azure AD. You can also create custom enterprise apps from this blade.

When you create a new enterprise application, either from the gallery or a custom app or service principal, you’ll find that an app registration is also created. So what’s this app registration?

An app registration represents a globally unique instance of your application, allowing you to integrate your app with Azure AD and use its features. You manage things like secrets and certificates, API permissions, token claims, branding and more with your app registration.

This app registration requires a service principal to represent it within an Azure AD tenant so that the application can access resources secured by Azure AD. The service principal is where access policies and permissions are assigned for the application. The app registration is only ever created once in the app’s home tenant, however a service principal is created in each tenant where the application is used (including the home tenant), referencing the app registration object. Think of it like a class in object-oriented programming - the service principal inherits some of the properties of the app registration much like an instance of an object inherits some of the properties of its class.

How do I create an identity for my infrastructure as code deployments?

Given infrastructure as code is one of the key things I talk about on this site, I thought it would be useful to include the process here. This example focuses on Terraform.

Create an application in app registrations and note the name and application ID.
Generate a client secret or certificate from the Certificates & secrets blade of the application. If creating a secret, store it securely.
Head to the enterprise applications blade of Azure AD and find your app. You can search using the app’s name or application ID. Note the object ID of the enterprise application/service principal object.
To assign RBAC permissions to the resources you need, use the object ID of the enterprise application/service principal.
Use the application ID and secret or certificate from the app registration to authenticate.

💡

The Azure AD overview blade has a search bar that can be used to search your entire tenant for anything including users, groups, apps, devices etc.

Azure Policy for AKS

Sun, 06 Sep 2020 23:00:00 GMT

Microsoft recently announced that pod security policy has been deprecated in favour of Azure Policy for AKS. As a result you’ll have to transition to Azure Policy for AKS and disable pod security policy by October 15th, 2020. If you don’t disable the pod security policy feature you’ll be unable to perform cluster upgrades and will lose Azure support, however given the feature is in preview this really isn’t a big deal.

With that in mind, what is Azure Policy for AKS and how does it work? The name pretty much says it all, it allows you to apply controls to, and audit the pods running in your AKS cluster using the familiar Azure Policy constructs of definitions and initiatives. This is implemented using the Gatekeeper admission controller.

Enabling Azure Policy for AKS

Azure Policy for AKS is an add-on that can be enabled using the az CLI, through the Azure Portal, or with Terraform. Prior to enabling the add-on the following pre-requisites must be satisfied:

Your AKS cluster must be running version 1.15 or later.
If you’re going to enable the add-on using az CLI the aks-preview extension must be installed and must be version 0.4.53 or later.

The AKS-AzurePolicyAutoApprove preview feature must be enabled. This can be done using the az feature register command.

az feature register --namespace Microsoft.ContainerService --name AKS-AzurePolicyAutoApprove
# After registering the feature you'll need to run the following to propogate the change.
az provider register -n Microsoft.ContainerService

The Microsoft.ContainerService and Microsoft.PolicyInsights resource providers must be registered. This can be done using the az provider registercommand.

az provider register --namespace Microsoft.ContainerService
az provider register --namespace Microsoft.PolicyInsights

Once these pre-requisites are met you can enable the add-on using any of the methods below.

Terraform

Add the following addon_profile block to your azurerm_kubernetes_cluster resource. If you have other add-ons enabled and already have an addon_profile block then simply add the azure_policyblock to it.

addon_profile {
  azure_policy {
    enabled = true
  }
}

AZ CLI

Simply execute the following command replacing democluster with your cluster name and aks-demo with the resource group your cluster belongs to.

az aks enable-addons \
--addons azure-policy \
--name democluster \
--resource-group aks-demo

Azure Portal

Navigate to your AKS cluster in the portal.
Click on Policies (Preview).

Click on Enable add-on

Policies

By default, Azure Policy for AKS comes with two initiatives - baseline and restricted. These initiatives are the quickest way to start applying some controls to your pods. The table below summarises the controls applied with each initiative.

Control	Baseline	Restricted
Disallow running of privileged containers	✔️	✔️
Disallow shared usage of host namespaces	✔️	✔️
Restrict all usage of host networking and ports	✔️	✔️
Restrict any usage of the host filesystem	✔️	✔️
Restrict Linux capabilities to the default set	✔️	✔️
Restrict usage of defined volume types		✔️
Privilege escalation to root		✔️
Restrict the user and group IDs of the container		✔️
Restrict allocating an FSGroup that owns the pod’s volumes		✔️
Requires seccomp profile		✔️

Of course, you are likely to want to develop an initiative that makes more sense for your use case. This may be to satisfy organisational policies, compliance frameworks, or security requirements. You can do this by reviewing the full list of available policy definitions and combining them in to an initiative. Instructions for creating Azure Policy initiatives can be found here.

Demo

To demonstrate Azure Policy for AKS I’m going to enable the add-on in a demo cluster, apply the baseline initiative, and then attempt to create a privileged container.

Enable the Add-On

The steps to enable the Azure Policy for AKS add-on are outlined at the start of this post, so I won’t repeat them here. Instead I’ll show you how to confirm that the add-on has been enabled for your cluster. The first thing to check is the Policies (preview) section of the AKS menu in the Azure portal which will look like this if the add-on is enabled:

You’ll also notice a few new pods running in your cluster for Gatekeeper and Azure Policy:

Apply the baseline policy

Head to Azure Policy in the Azure Portal.
Click on Assign Initiative.
Set your scope to either the entire subscription or a resource group containing your policy-enabled AKS cluster.
Open the initiative declaration search by clicking on the ellipsis button next to the Initiative definition box. Search for “kubernetes” to find the AKS policy initiatives and select the baseline initiative.

Open the Parameters tab and set the effect to “deny”. This will prevent a pod being deployed in in the event that it does not comply with policy, rather than only being audited.
Click on Review + Create to create the assignment.

Test a control

To confirm that the policies are being enforced, I’m going to attempt to create a privileged container. Referring to the table above you’ll notice that this is one of the controls implemented by the baseline policy. This is the manifest we are going to deploy:

apiVersion: v1
kind: Pod
metadata:
  name: alpine-privileged
spec:
  containers:
    - name: alpine-privileged
      image: alpine
      securityContext:
        privileged: true

Below is the error message given after attempting to deploy the privileged container, clearly showing that the deployment was denied by policy. Note that this gives immediate, clear feedback which is important in enabling developers to quickly identify and resolve the reason their pod is being denied.

If we then remove the privileged security context from the pod so that our manifest now looks like this:

apiVersion: v1
kind: Pod
metadata:
  name: alpine-standard
spec:
  containers:
    - name: alpine-standard
      image: alpine

We see that the pod is successfully deployed as expected as it complies with our policies.