Invoke an App Service web app from Azure AI Foundry Agent Service

Azure AI Foundry Agent Service allows you to create AI agents tailored to your needs through custom instructions and augmented by advanced tools like code interpreter, and custom functions. You can now connect your Azure AI Agent to an external API using an OpenAPI 3.0 specified tool, allowing for scalable interoperability with various applications.

Azure App Service is a fully managed platform for building, deploying, and scaling web apps and APIs. If your API is hosted on Azure App Service, you can connect your AI Agent to the API using the OpenAPI specification. The OpenAPI specification defines the API and how to interact with it. You can then use natural language to invoke the API through your AI Agent. This tool is powerful because it allows you to add AI agent capabilities to your existing apps with minimal code changes. Also, this agent has the ability to directly interact with your app. There's no need for significant code changes or implementation work other thank minor updates to interact with the agent using the available SDKs.

In this tutorial, you're using an Azure AI Foundry Agent to invoke an existing API hosted on Azure App Service. By the end of this tutorial, you have a fashion assistant chat application running in App Service using an AI agent from the Azure AI Foundry Agent Service.

Prerequisites

1. A GitHub account. You can also get one for free.
2. An Azure account with an active subscription. If you don't have an Azure account, you can create one for free.

1. Inspect the sample in GitHub Codespaces

1. Sign in to your GitHub account and navigate to https://github.com/Azure-Samples/ai-agent-openai-web-app/fork.
2. Select Create fork.
3. Select Code > Create codespace on main. The codespace takes a few minutes to set up.

The sample repository has the following content:

2. Deploy the Azure infrastructure and application

1. Sign into your Azure account by using the azd auth login command and following the prompt:

   azd auth login
   

2. Create the App Service app and deploy the code using the azd up command:

   azd up
   

   The azd up command might take a few minutes to complete. azd up uses the Bicep files in your projects to create an App Service app in the P0v3 pricing tier and deploys the .NET app in src/webapp. The command also creates the Azure AI Foundry and supporting resources for the agent.

3. Create the AI agent

1. In the Azure portal, navigate to the resource group that the azd template creates. The name of the resource group is in the output of the azd command you run.

2. Select the Azure AI project resource. The name should be in the format ai-aiproject-<identifier>. Ensure you select the project resource and not the hub or AI services resources. Agents are created from the Azure AI project resource.

3. Select Launch studio to open the Azure AI Foundry studio.

4. From the left menu under Build and customize, select Agents.

5. When the page loads, in the dropdown, select the autogenerated Azure OpenAI Service resource that is created for you and then select Let's go.

6. Select + New agent to create a new agent or use the default one if one is already created for you.

7. When the agent is created, add the following instructions in the right menu. These instructions ensure your agent only answers questions and completes tasks related to the fashion store app.

    You are an agent for a fashion store that sells clothing. You have the ability to view inventory, update the customer's shopping cart, and answer questions about the clothing items that are in the inventory. You should not answer questions about topics that are unrelated to the fashion store. If a user asks an unrelated question, please respond by telling them that you can only talk about things that are related to the fashion store.
    

4. Add the OpenAPI Specified Tool to the AI agent

For detailed guidance with screenshots and additional information, see Add OpenAPI spec tool in the Azure AI Foundry portal. The steps are summarized in the following instructions.

1. Select + Add next to Action.

2. Select OpenAPI 3.0 specified tool.

3. Give your tool a name and the following description. The description is used by the model to decide when and how to use the tool.

    This tool is used to interact with and manage an online fashion store. The tool can add or remove items from a shopping cart as well as view inventory.
    

4. Leave the authentication method as anonymous. There's no authentication on the web app. If the app required an API key or managed identity to access it, this ___location is where you would specify this information.

5. Copy and paste the OpenAPI specification in the text box. The OpenAPI specification is provided in the Codespace in the src/webapp directory and is called swagger.json.

6. Before you complete creating the tool, you need to copy and paste your app's URL into the OpenAPI specification. Replace the placeholder <APP-SERVICE-URL> in line 10 of the OpenAPI specification with your app's URL. It's in the format https://<app-name>.azurewebsites.net.

   • To find your app's URL, go back to your resource group and navigate to your App Service. The URL is on the Overview page of your App Service.

7. Select Next, review the details you provided, and then select Create Tool.

5. Connect your App Service to the AI Agent

After setting up the AI Agent and adding the OpenAPI Specified Tool, you need to configure your App Service with the appropriate environment variables so the app knows which agent to connect to. The app already has a managed identity assigned that gives it access to the AI Agent Service. This managed identity is required for the app to reach the agent and is created with the azd template.

1. From the Agents dashboard where you added the OpenAPI tool, note the agent ID. It's in the format asst_<unique-identifier>.
2. Select Overview in the menu note the project's connection string. It's in the format <region>.api.azureml.ms;<subscription-id>;<resource-group-name>;<project-name>.
3. Navigate to your App Service in the Azure portal.
4. Select Environment variables in the left menu.
5. In the App settings tab, select + Add and add the following setting:
   • Name: AzureAIAgent__ConnectionString
   • Value: The connection string you noted from your AI Agent Service
6. Add another app setting:
   • Name: AzureAIAgent__AgentId
   • Value: The Agent ID you noted when creating your agent
7. Select Apply at the bottom of the page and confirm when prompted. The app restarts with the new settings applied.

6. Verify the running app

In the azd output, find the URL of your app and navigate to it in the browser. The URL looks like this in the azd output:

Deploying services (azd deploy)

    (✓) Done: Deploying service web
    - Endpoint: https://<app-name>.azurewebsites.net/

In the chat window, ask the agent questions such as:

• What's in my cart?
• Add a small denim jacket to my cart
• Do we have any blazers in stock? You can also ask general questions about the items and the agent should be able to provide information.
• Tell me about Red Slim Fit Checked Casual Shirt
• Is the blazer warm?

Clean-up

When you're done with this app, run the following to delete the resource group with all of the resources created during this tutorial.

azd down

Troubleshooting

Common Issues

1. Chat Not Working
   • Verify that the environment variables (AzureAIAgent__ConnectionString and AzureAIAgent__AgentId) are correctly set in the App Service environment variables.
   • Check that the AI Agent is properly created and configured with the correct OpenAPI tool.
   • Ensure the OpenAPI specification URL is accessible from the Azure AI Foundry Agent Service.
   • Ensure the App Service URL is updated in the swagger.json provided to the OpenAPI Specified Tool.
2. Permission Issues
   • If you encounter authentication errors, ensure that your App Service's managed identity has proper permissions to access the Azure AI Foundry Agent Service. The managed identity needs at least the Microsoft.MachineLearningServices/workspaces/agents/action permission to interact with the Agent. The provided "Azure AI Developer role" has this permission and should be sufficient. If you decide to change this role, be sure it has the necessary permission.
3. API Issues
   • If the agent is unable to perform actions on the inventory or cart, check the API routes in the OpenAPI specification.
   • Verify that the API endpoints are responding correctly by testing them directly in the Swagger UI at /api/docs.

Viewing Logs

To view logs for your App Service:

1. Navigate to your App Service in the Azure portal.
2. In the left menu, select Monitoring > Log stream to view real-time logs.
3. These logs reveal any application issues that you might need to address.

Understanding the API Capabilities

The OpenAPI specification provides the AI agent with information about available endpoints:

Cart API Endpoints

• GET /api/Cart: Retrieves the current shopping cart contents and total cost
• DELETE /api/Cart: Clears all items from the cart
• POST /api/Cart/add: Adds an item to the shopping cart
• PUT /api/Cart/{productId}/size/{size}: Updates the quantity of a specific item
• DELETE /api/Cart/{productId}/size/{size}: Removes a specific item from the cart

Inventory API Endpoints

• GET /api/Inventory: Lists all available inventory items
• GET /api/Inventory/{id}: Gets details about a specific product
• GET /api/Inventory/{id}/size/{size}: Checks inventory for a specific product size
• GET /api/Inventory/sizes: Gets all available sizes in the inventory

Advanced Agent Interactions

Beyond basic interactions, the AI agent can handle more complex scenarios:

• Personalized Recommendations: "I need a business casual outfit for a meeting"
• Size Guidance: "What size blazer would fit someone who's 6'2" and 180 pounds?"
• Outfit Coordination: "What would go well with the black denim jacket?"
• Shopping Cart Management: "Remove the large shirt and add a medium instead"
• Inventory Checks: "Do you have any red shirts in medium?"
• Price Inquiries: "What's the price range for blazers?"

Security Considerations

• The application uses Azure managed identities for secure authentication to Azure AI Foundry Agent Service in production environments.
• You can further secure your app and agent using any of the standard practices and Azure resources. For secure Azure AI Agent infrastructure templates, see the azureai-samples repo.

Next steps

Now that you learned how to connect your AI Agent to an API on Azure App Service, you can explore the other AI integrations available with App Service: