Skip to content

Update AI Toolkit with Foundry Local support and UI changes #9141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ms-johnalex wants to merge 1 commit into
base: main
Choose a base branch
from
+18 −12
Open

Update AI Toolkit with Foundry Local support and UI changes #9141

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/intelligentapps/images/overview/get-started.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions docs/intelligentapps/images/overview/initial-view.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 7 additions & 6 deletions docs/intelligentapps/models.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
ContentId: 52ad40fe-f352-4e16-a075-7a9606c5df3b
DateApproved: 10/03/2025
DateApproved: 12/05/2025
MetaDescription: Find a popular generative AI model by publisher and source. Bring your own model that is hosted with a URL, or select an Ollama model.
---
# Explore models in AI Toolkit
Expand All @@ -12,7 +12,7 @@ Within the model catalog, you can explore and utilize models from multiple hosti
- Models hosted on GitHub, such as Llama3, Phi-3, and Mistral, including pay-as-you-go options.
- Models provided directly by publishers, including OpenAI's ChatGPT, Anthropic's Claude, and Google's Gemini.
- Models hosted on Microsoft Foundry.
- Models downloaded locally from repositories like Ollama and ONNX.
- Models downloaded locally from repositories like Foundry Local, Ollama and ONNX.
- Custom self-hosted or externally deployed models accessible via Bring-Your-Own-Model (BYOM) integration.

Deploy models directly to Foundry from within the model catalog, streamlining your workflow.
Expand All @@ -30,7 +30,7 @@ To find a model in the model catalog:
1. Select **MODELS** > **Catalog** to open the model catalog
1. Use the filters to reduce the list of available models

- **Hosted by**: AI Toolkit supports GitHub, ONNX, OpenAI, Anthropic, Google as model hosting sources.
- **Hosted by**: AI Toolkit supports Microsoft Foundry, Foundry Local, GitHub, ONNX, OpenAI, Ollama, Anthropic, Google, NVIDIA NIM, and Windows AI API as model hosting sources.
- **Publisher**: The publisher for AI models, such as Microsoft, Meta, Google, OpenAI, Anthropic, Mistral AI, and more.
- **Feature**: Supported features of the model, such as `Text Attachment`, `Image Attachment`, `Web Search`, `Structured Outputs`, and more.
- **Model type**: Filter models that can run remotely or locally on CPU, GPU, or NPU. This filter depends on the local availability.
Expand All @@ -48,6 +48,7 @@ To add a model from the model catalog:
1. Select the **Add** on the model card
1. The flow for adding models will be slightly different based on the providers:

- **Foundry Local**: Foundry Local downloads and runs the model, which may take a few minutes depending on your internet speed. The model is available on a localhost page and added to AI Toolkit. Learn more in [What is Foundry Local?](https://learn.microsoft.com/azure/ai-foundry/foundry-local/what-is-foundry-local?view=foundry-classic&preserve-view=true).
- **GitHub**: AI Toolkit asks for your GitHub credentials to access the model repository. Once authenticated, the model is added directly into AI Toolkit.
> [!NOTE]
> AI Toolkit now [supports GitHub pay-as-you-go models](/docs/intelligentapps/playground.md#_github-payasyougo-model-support), so you can keep working after passing free tier limits.
Expand All @@ -70,7 +71,7 @@ You can also add your own models that are hosted externally or run locally. Ther
- Add custom ONNX models, such as those from Hugging Face, using AI Toolkit's [model conversion tool](/docs/intelligentapps/modelconversion.md).

There are several entrypoints to add models to AI Toolkit:
- From **MY MODELS** in the tree view, hover over it and select the `+` icon.
- From **MY RESOURCES** section in the tree view, hover over **Models** and select the `+` icon.
![Screenshot of the AI Toolkit interface showing the Model Catalog toolbar with the + Add model button highlighted, indicating where users can click to add a new custom model.](./images/models/custom-1.png)

- From the **Model Catalog**, select the **+ Add model** button from the tool bar.
Expand Down Expand Up @@ -176,13 +177,13 @@ Some models require a publisher or hosting-service license and account to sign-i
In this article, you learned how to:

- Explore and manage generative AI models in AI Toolkit.
- Find models from various sources, including GitHub, ONNX, OpenAI, Anthropic, Google, Ollama, and custom endpoints.
- Find models from various sources, including Microsoft Foundry, Foundry Local, GitHub, ONNX, OpenAI, Anthropic, Google, Ollama, and custom endpoints.
- Add models to your toolkit and deploy them to Microsoft Foundry.
- Add custom models, including Ollama and OpenAI compatible models, and test them in the playground or agent builder.
- Use the model catalog to view available models and select the best fit for your AI application needs.
- Use filters and search to find models quickly.
- Browse models by category, such as Popular, GitHub, ONNX, and Ollama.
- Convert and add custom ONNX models using the model conversion tool.
- Manage models in MY MODELS, including editing, deleting, refreshing, and viewing details.
- Manage models in MY RESOURCES/Models, including editing, deleting, refreshing, and viewing details.
- Start and stop the ONNX server and copy endpoints for local models.
- Handle license and sign-in requirements for some models before testing them.
9 changes: 7 additions & 2 deletions docs/intelligentapps/overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
ContentId: 164299e8-d27d-40b9-8b8d-a6e05df8ac69
DateApproved: 10/03/2025
DateApproved: 12/05/2025
MetaDescription: Build, test, and deploy AI applications with AI Toolkit for Visual Studio Code. Features model playground, prompt engineering, batch evaluation, fine-tuning, and multi-modal support for LLMs and SLMs.
---
# AI Toolkit for Visual Studio Code
Expand All @@ -13,7 +13,7 @@ AI Toolkit offers seamless integration with popular AI models from providers lik

| Feature | Description | Screenshot |
|---------|-------------|------------|
| [Model Catalog](/docs/intelligentapps/models.md) | Discover and access AI models from multiple sources including GitHub, ONNX, Ollama, OpenAI, Anthropic, and Google. Compare models side-by-side and find the perfect fit for your use case. | ![Screenshot showing the AI Toolkit Model Catalog interface with various AI model options](./images/overview/catalog.png) |
| [Model Catalog](/docs/intelligentapps/models.md) | Discover and access AI models from multiple sources including Microsoft Foundry, Foundry Local, GitHub, ONNX, Ollama, OpenAI, Anthropic, and Google. Compare models side-by-side and find the perfect fit for your use case. | ![Screenshot showing the AI Toolkit Model Catalog interface with various AI model options](./images/overview/catalog.png) |
| [Playground](/docs/intelligentapps/playground.md) | Interactive chat environment for real-time model testing. Experiment with different prompts, parameters, and multi-modal inputs including images and attachments. | ![Screenshot showing the AI Toolkit Playground interface with chat messaging and model parameter controls](./images/overview/playground.png) |
| [Agent Builder](/docs/intelligentapps/agentbuilder) | Streamlined prompt engineering and agent development workflow. Create sophisticated prompts, integrate MCP tools, and generate production-ready code with structured outputs. | ![Screenshot showing the Agent Builder interface for creating and managing AI agents](./images/overview/agent-builder.png) |
| [Bulk Run](/docs/intelligentapps/bulkrun) | Execute batch prompt testing across multiple models simultaneously. Ideal for comparing model performance and testing at scale with various input scenarios. | ![Screenshot showing the Bulk Run interface for batch testing prompts across multiple AI models](./images/overview/bulk-run.png) |
Expand Down Expand Up @@ -95,6 +95,11 @@ AI Toolkit opens in its own view, with the AI Toolkit icon now displayed on the
- **Evaluation**: Evaluate models, prompts, and agents by comparing their outputs to ground truth data and computing evaluation metrics.
- **Tracing**: Trace capabilities to help you monitor and analyze the performance of your AI applications.

- **Build Agent with GitHub Copilot**: This section enables you to use GitHub Copilot to help you build AI agents faster with AI Toolkit. It contains the following subsections:
- **Create Agent**: Opens the Chat pane and creates a prompt to build an AI agent with a Console application using GitHub Copilot.
Copy link
Contributor

@ntrogh ntrogh Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Create Agent**: Opens the Chat pane and creates a prompt to build an AI agent with a Console application using GitHub Copilot.
- **Create Agent**: Opens the Chat view and creates a prompt to build an AI agent with a Console application using GitHub Copilot.

- **Enable Tracing**: Opens the Chat pane and creates a prompt to add tracing to the current workspace using GitHub Copilot.
Copy link
Contributor

@ntrogh ntrogh Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Enable Tracing**: Opens the Chat pane and creates a prompt to add tracing to the current workspace using GitHub Copilot.
- **Enable Tracing**: Opens the Chat view and creates a prompt to add tracing to the current workspace using GitHub Copilot.

- **Add Evaluation Framework**: Opens the Chat pane and creates a prompt to add the evaluation framework to the current workspace using GitHub Copilot.
Copy link
Contributor

@ntrogh ntrogh Dec 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Add Evaluation Framework**: Opens the Chat pane and creates a prompt to add the evaluation framework to the current workspace using GitHub Copilot.
- **Add Evaluation Framework**: Opens the Chat view and creates a prompt to add the evaluation framework to the current workspace using GitHub Copilot.


- **MCP Workflow**: This section contains tools you use to add an existing MCP server or to create a new one. It contains the following subsections:
- **Add MCP Server**: The link for adding and working with an existing MCP server.
- **Create new MCP Server**: The link for creating and deploying new MCP servers in AI Toolkit.
Expand Down