Menu Close

Tag: #copilotstudio

{Do you know} Analyze quality of responses that use generative AI in Copilot Studio

uk365guy

Hello Everyone,

Today I am going to share my thoughts on Copilot Studio’s new feature, the quality of responses that use generative AI.

Let’s get started.

Microsoft’s Copilot Studio introduced a new feature on June 17, 2025, enabling administrators, makers, marketers, and analysts to automatically analyse the quality of responses generated by AI copilots. This feature aims to provide actionable insights to enhance agent performance.

The evaluation framework for generative AI (GenAI) systems has been evolving to address the limitations of traditional lab-based assessments.

A comprehensive evaluation framework was proposed to assess GenAI systems in real-world scenarios, emphasising the need for dynamic and ongoing assessments that consider user intent, social dynamics, and emergent behaviours.

Additionally, Google Cloud introduced the GenAI evaluation service within Vertex AI, allowing users to evaluate generative models against predefined or custom criteria. This service supports various metrics, including accuracy, relevance, and user satisfaction, facilitating a more nuanced understanding of AI performance.

These developments reflect a broader trend towards more sophisticated and context-aware evaluations of generative AI systems, moving beyond traditional benchmarks to emcompass real-world performance and user experience.

That’s it for today.

I hope this helps.

Malla Reddy Gurram(@aka UK365GUY)

Share this:

{How to} Use your own model when generating responses in copilot studio.

uk365guy

Hello Everyone,

Today i am going to share my thoughts on how to use your own model when generating responses in copilot studio.

Let’s get’s started.

In Copilot Studio, you can enhance AI behavior and responses by integrating your own custom model instead of relying on the default GPT based assistant. This is particularly useful for organizations that have domain specific data, proprietary language models, or unique use cases that require tailored AI responses. By using own model, you gain greater control over responses logic, accuracy and compliance with general standards or regulations.

To use your own model in Copilot Studio, you typically configure it through plugin actions or APIs.

This involves settings up an external endpoint where your model is hosted(e.g., Azure ML, AWS SageMaker, or a self-hosted REST API). Then you create a plugin in Copilot Studio that calls your model’s API, passing in user input and returning the model’s responses.

You can then use this plugin as a custom action in topics, ensuring your bot routes user queries through your specific model when needed.

+———————+
| User Message |
+———————+
|
v
+————————–+
| Copilot Studio Topic |
+————————–+
|
v
+————————–+
| Custom Plugin Action | <-- Calls your model +--------------------------+ | v +--------------------------+ | Your Custom Model (API) | +--------------------------+ | v +--------------------------+ | Response to Copilot | +--------------------------+ | v +--------------------------+ | User Receives | | Model's Response | +--------------------------+ This setup allows you to combine Copilot's powerful orchestration tools with the unique reasoning and training of your own AI models. That's it for today. Malla Reddy Gurram(@aka UK365GUY) #365blogpostsin365days

Share this: