Edit

Share via

Tutorial: Build a chatbot with Azure App Service and Azure OpenAI (Spring Boot)

In this tutorial, you'll build an intelligent AI application by integrating Azure OpenAI with a Java Spring Boot application and deploying it to Azure App Service. You'll create a Spring Boot controller that sends a query to Azure OpenAI and sends the response to the browser.

Tip

While this tutorial uses Spring Boot, the core concepts of building a chat application with Azure OpenAI apply to any Java web application. If you're using a different hosting option on App Service, such as Tomcat or JBoss EAP, you can adapt the authentication patterns and Azure SDK usage shown here to your preferred framework.

Screenshot showing a chatbot running in Azure App Service.

In this tutorial, you learn how to:

  • Create an Azure OpenAI resource and deploy a language model.
  • Build a Spring Boot application that connects to Azure OpenAI.
  • Use dependency injection to configure the Azure OpenAI client.
  • Deploy the application to Azure App Service.
  • Implement passwordless secure authentication both in the development environment and in Azure.

Prerequisites

1. Create an Azure OpenAI resource

In this section, you'll use GitHub Codespaces to create an Azure OpenAI resource with the Azure CLI.

  1. Go to GitHub Codespaces and sign in with your GitHub account.

  2. Find the Blank template by GitHub and select Use this template to create a new blank Codespace.

  3. In the Codespace terminal, install the Azure CLI:

    curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

  4. Sign in to your Azure account:

    az login

    Follow the instructions in the terminal to authenticate.

  5. Set environment variables for your resource group name, Azure OpenAI service name, and ___location:

    export RESOURCE_GROUP="<group-name>"
export OPENAI_SERVICE_NAME="<azure-openai-name>"
export APPSERVICE_NAME="<app-name>"
export LOCATION="eastus2"

    Important

    The region is critical as it's tied to the regional availability of the chosen model. Model availability and deployment type availability vary from region to region. This tutorial uses gpt-4o-mini, which is available in eastus2 under the Standard deployment type. If you deploy to a different region, this model might not be available or might require a different tier. Before changing regions, consult the Model summary table and region availability to verify model support in your preferred region.

  6. Create a resource group and an Azure OpenAI resource with a custom ___domain, then add a gpt-4o-mini model:

    # Resource group
az group create --name $RESOURCE_GROUP --___location $LOCATION
# Azure OpenAI resource
az cognitiveservices account create \
  --name $OPENAI_SERVICE_NAME \
  --resource-group $RESOURCE_GROUP \
  --___location $LOCATION \
  --custom-___domain $OPENAI_SERVICE_NAME \
  --kind OpenAI \
  --sku s0
# gpt-4o-mini model
az cognitiveservices account deployment create \
  --name $OPENAI_SERVICE_NAME \
  --resource-group $RESOURCE_GROUP \
  --deployment-name gpt-4o-mini \
  --model-name gpt-4o-mini \
  --model-version 2024-07-18 \
  --model-format OpenAI \
  --sku-name Standard \
  --sku-capacity 1
# Cognitive Services OpenAI User role that lets the signed in Azure user to read models from Azure OpenAI
az role assignment create \
  --assignee $(az ad signed-in-user show --query id -o tsv) \
  --role "Cognitive Services OpenAI User" \
  --scope /subscriptions/$(az account show --query id -o tsv)/resourceGroups/$RESOURCE_GROUP/providers/Microsoft.CognitiveServices/accounts/$OPENAI_SERVICE_NAME

Now that you have an Azure OpenAI resource, you'll create a web application to interact with it.

2. Create and set up a Spring Boot web app

  1. In your Codespace terminal, clone the Spring Boot REST sample to the workspace and try running it the first time.

    git clone https://github.com/rd-1-2022/rest-service .
mvn spring-boot:run

    You should see a notification in GitHub Codespaces indicating that the app is available at a specific port. Select Open in browser to launch the app in a new browser tab. When you see the white label error page, the Spring Boot app is working.

  2. Back in the Codespace terminal, stop the app with Ctrl+C.

  3. Open pom.xml and add the following dependencies:

    <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>
<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-ai-openai</artifactId>
    <version>1.0.0-beta.16</version>
</dependency>
<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-core</artifactId>
    <version>1.55.3</version>
</dependency>
<dependency>
    <groupId>com.azure</groupId>
    <artifactId>azure-identity</artifactId>
    <version>1.16.0</version>
    <scope>compile</scope>
</dependency>

  4. In the same directory as Application.java (src/main/java/com/example/restservice) add a Java file called ChatController.java and copy the following content into it:

    package com.example.restservice;

import java.util.ArrayList;
import java.util.List;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;

import com.azure.ai.openai.OpenAIAsyncClient;
import com.azure.ai.openai.models.ChatChoice;
import com.azure.ai.openai.models.ChatCompletionsOptions;
import com.azure.ai.openai.models.ChatRequestMessage;
import com.azure.ai.openai.models.ChatRequestUserMessage;
import com.azure.ai.openai.models.ChatResponseMessage;
import com.azure.core.credential.TokenCredential;
import com.azure.identity.DefaultAzureCredentialBuilder;

@Configuration
class AzureConfig {
    // Reads the endpoint from environment variable AZURE_OPENAI_ENDPOINT
    @Value("${azure.openai.endpoint}")
    private String openAiEndpoint;

    // Provides a credential for local dev and production
    @Bean
    public TokenCredential tokenCredential() {
        return new DefaultAzureCredentialBuilder().build();
    }

    // Configures the OpenAIAsyncClient bean
    @Bean
    public OpenAIAsyncClient openAIClient(TokenCredential tokenCredential) {
        return new com.azure.ai.openai.OpenAIClientBuilder()
                .endpoint(openAiEndpoint)
                .credential(tokenCredential)
                .buildAsyncClient();
    }
}

@Controller
public class ChatController {
    private final OpenAIAsyncClient openAIClient;

    // Inject the OpenAIAsyncClient bean
    public ChatController(OpenAIAsyncClient openAIClient) {
        this.openAIClient = openAIClient;
    }

    @RequestMapping(value = "/", method = RequestMethod.GET)
    public String chatFormOrWithMessage(Model model, @RequestParam(value = "userMessage", required = false) String userMessage) {
        String aiResponse = null;
        if (userMessage != null && !userMessage.isBlank()) {

            // Create a list of chat messages
            List<ChatRequestMessage> chatMessages = new ArrayList<>();
            chatMessages.add(new ChatRequestUserMessage(userMessage));

            // Send the chat completion request
            String deploymentName = "gpt-4o-mini";
            StringBuilder serverResponse = new StringBuilder();
            var chatCompletions = openAIClient.getChatCompletions(
                deploymentName, 
                new ChatCompletionsOptions(chatMessages)
            ).block();
            if (chatCompletions != null) {
                for (ChatChoice choice : chatCompletions.getChoices()) {
                    ChatResponseMessage message = choice.getMessage();
                    serverResponse.append(message.getContent());
                }
            }
            aiResponse = serverResponse.toString();
        }
        model.addAttribute("aiResponse", aiResponse);
        return "chat";
    }
}

    Tip

    To minimize the files in this tutorial, the code combines the Spring @Configuration and @Controller classes in one file. In production, you would normally separate configuration and business logic for maintainability.

  5. Under src/main/resources, create a templates directory, and add a chat.html with the following content for the chat interface:

    <!DOCTYPE html>
<html xmlns:th="http://www.thymeleaf.org">
<head>
    <meta charset="UTF-8">
    <title>Azure OpenAI Chat</title>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/css/bootstrap.min.css" rel="stylesheet">
</head>
<body>
<div class="container py-4">
    <h2 class="mb-4">Azure OpenAI Chat</h2>
    <form action="/" method="get" class="d-flex mb-3">
        <input name="userMessage" class="form-control me-2" type="text" placeholder="Type your message..." autocomplete="off" required />
        <button class="btn btn-primary" type="submit">Send</button>
    </form>
    <div class="mb-3">
        <div th:if="${aiResponse}" class="alert alert-info">AI: <span th:text="${aiResponse}"></span></div>
    </div>
</div>
</body>
</html>

  6. In the terminal, retrieve your OpenAI endpoint:

    az cognitiveservices account show \
  --name $OPENAI_SERVICE_NAME \
  --resource-group $RESOURCE_GROUP \
  --query properties.endpoint \
  --output tsv

  7. Run the app again by adding AZURE_OPENAI_ENDPOINT with its value from the CLI output:

    AZURE_OPENAI_ENDPOINT=<output-from-previous-cli-command> mvn spring-boot:run

  8. Select Open in browser to launch the app in a new browser tab.

  9. Type a message in the textbox and select "Send, and give the app a few seconds to reply with the message from Azure OpenAI.

The application uses DefaultAzureCredential, which automatically uses your Azure CLI signed in user for token authentication. Later in this tutorial, you'll deploy your web app to Azure App Service and configure it to securely connect to your Azure OpenAI resource using managed identity. The same DefaultAzureCredential in your code can detect the managed identity and use it for authentication. No extra code is needed.

3. Deploy to Azure App Service and configure OpenAI connection

Now that your app works locally, let's deploy it to Azure App Service and set up a service connection to Azure OpenAI using managed identity.

  1. Create a deployment package with Maven.

    mvn clean package

  2. First, deploy your app to Azure App Service using the Azure CLI command az webapp up. This command creates a new web app and deploys your code to it:

    az webapp up \
  --resource-group $RESOURCE_GROUP \
  --___location $LOCATION \
  --name $APPSERVICE_NAME \
  --plan $APPSERVICE_NAME \
  --sku B1 \
  --runtime "JAVA:21" \
  --os-type Linux \
  --track-status false

    This command might take a few minutes to complete. It creates a new web app in the same resource group as your OpenAI resource.

  3. After the app is deployed, create a service connection between your web app and the Azure OpenAI resource using managed identity:

    az webapp connection create cognitiveservices \
  --resource-group $RESOURCE_GROUP \
  --name $APPSERVICE_NAME \
  --target-resource-group $RESOURCE_GROUP \
  --account $OPENAI_SERVICE_NAME \
  --system-identity

    This command creates a connection between your web app and the Azure OpenAI resource by:

    • Generating system-assigned managed identity for the web app.
    • Adding the Cognitive Services OpenAI Contributor role to the managed identity for the Azure OpenAI resource.
    • Adding the AZURE_OPENAI_ENDPOINT app setting to your web app.

  4. Open the deployed web app in the browser.

    az webapp browse

  5. Type a message in the textbox and select "Send, and give the app a few seconds to reply with the message from Azure OpenAI.

    Screenshot showing a chatbot running in Azure App Service.

Your app is now deployed and connected to Azure OpenAI with managed identity. Note that is it accessing the AZURE_OPENAI_ENDPOINT app setting through the @Configuration injection.

Frequently asked questions

Why does the sample use @Configuration and Spring beans for the OpenAI client?

Using a Spring bean for the OpenAIAsyncClient ensures that:

  • All configuration properties (like the endpoint) are loaded and injected by Spring.
  • The credential and client are created after the application context is fully initialized.
  • Dependency injection is used, which is the standard and most robust pattern in Spring applications.

The asynchronous client is more robust, especially when using DefaultAzureCredential with Azure CLI authentication. The synchronous OpenAIClient can encounter issues with token acquisition in some local development scenarios. Using the asynchronous client avoids these issues and is the recommended approach.

What if I want to connect to OpenAI instead of Azure OpenAI?

To connect to OpenAI instead, use the following code:

OpenAIClient client = new OpenAIClientBuilder()
    .credential(new KeyCredential(<openai-api-key>))
    .buildClient();

For more information, see OpenAI API authentication.

When working with connection secrets in App Service, you should use Key Vault references instead of storing secrets directly in your codebase. This ensures that sensitive information remains secure and is managed centrally.

Can I connect to Azure OpenAI with an API key instead?

Yes, you can connect to Azure OpenAI using an API key instead of managed identity. This approach is supported by the Azure OpenAI SDKs and Semantic Kernel.

When working with connection secrets in App Service, you should use Key Vault references instead of storing secrets directly in your codebase. This ensures that sensitive information remains secure and is managed centrally.

How does DefaultAzureCredential work in this tutorial?

The DefaultAzureCredential simplifies authentication by automatically selecting the best available authentication method:

  • During local development: After you run az login, it uses your local Azure CLI credentials.
  • When deployed to Azure App Service: It uses the app's managed identity for secure, passwordless authentication.

This approach lets your code run securely and seamlessly in both local and cloud environments without modification.

Next steps