Tutorial: Build a retrieval augmented generation app in Azure App Service with Azure OpenAI and Azure AI Search (.NET)

In this tutorial, you'll create a .NET retrieval augmented generation (RAG) application using .NET Blazor, Azure OpenAI, and Azure AI Search and deploy it to Azure App Service. This application demonstrates how to implement a chat interface that retrieves information from your own documents and leverages Azure AI services to provide accurate, contextually aware answers with proper citations. The solution uses managed identities for passwordless authentication between services.

In this tutorial, you learn how to:

Architecture overview

Before you begin deployment, it's helpful to understand the architecture of the application you'll build. The following diagram is from Custom RAG pattern for Azure AI Search:

In this tutorial, the Blazer application in App Service takes care of both the app UX and the app server. However, it doesn't make a separate knowledge query to Azure AI Search. Instead, it tells Azure OpenAI to do the knowledge querying specifying Azure AI Search as a data source. This architecture offers several key advantages:

• Integrated Vectorization: Azure AI Search's integrated vectorization capabilities make it easy and quick to ingest all your documents for searching, without requiring more code for generating embeddings.
• Simplified API Access: By using the Azure OpenAI On Your Data pattern with Azure AI Search as a data source for Azure OpenAI completions, there's no need to implement complex vector search or embedding generation. It's just one API call and Azure OpenAI handles everything, including prompt engineering and query optimization.
• Advanced Search Capabilities: The integrated vectorization provides everything needed for advanced hybrid search with semantic reranking, which combines the strengths of keyword matching, vector similarity, and AI-powered ranking.
• Complete Citation Support: Responses automatically include citations to source documents, making information verifiable and traceable.

Prerequisites

• An Azure account with an active subscription - Create an account for free.
• GitHub account to use GitHub Codespaces - Learn more about GitHub Codespaces.

1. Open the sample with Codespaces

The easiest way to get started is by using GitHub Codespaces, which provides a complete development environment with all required tools preinstalled.

1. Navigate to the GitHub repository at https://github.com/Azure-Samples/app-service-rag-openai-ai-search-dotnet.

2. Select the Code button, select the Codespaces tab, and click Create codespace on main.

3. Wait a few moments for your Codespace to initialize. When ready, you'll see a fully configured VS Code environment in your browser.

2. Deploy the sample architecture

1. In the terminal, log into Azure using Azure Developer CLI:

   azd auth login
   

   Follow the instructions to complete the authentication process.

2. Provision the Azure resources with the AZD template:

   azd provision
   

3. When prompted, give the following answers:

   Question	Answer
   Enter a new environment name:	Type a unique name.
   Select an Azure Subscription to use:	Select the subscription.
   Pick a resource group to use:	Select Create a new resource group.
   Select a ___location to create the resource group in:	Select any region. The resources will actually be created in East US 2.
   Enter a name for the new resource group:	Type Enter.

4. Wait for the deployment to complete. This process will:

   • Create all required Azure resources.
   • Deploy the Blazor application to Azure App Service.
   • Configure secure service-to-service authentication using managed identities.
   • Set up the necessary role assignments for secure access between services.

   Note

   To learn more about how managed identities work, see What are managed identities for Azure resources? and How to use managed identities with App Service.

5. After successful deployment, you'll see a URL for your deployed application. Make note of this URL, but don't access it yet because you still need to set up the search index.

3. Upload documents and create a search index

Now that the infrastructure is deployed, you need to upload documents and create a search index that the application will use:

1. In the Azure portal, navigate to the storage account that was created by the deployment. The name will start with the environment name you provided earlier.

2. Select Containers from the left navigation menu and open the documents container.

3. Upload sample documents by clicking Upload. You can use the sample documents from the sample-docs folder in the repository, or your own PDF, Word, or text files.

   

4. Navigate to your Azure AI Search service in the Azure portal.

5. Select Import and vectorize data to start the process of creating a search index.

   

6. In the Connect to your data step:

   • Select Azure Blob Storage as the Data Source.
   • Select RAG.
   • Choose your storage account and the documents container.
   • Make sure Authenticate using managed identity is selected.
   • Select Next.

7. In the Vectorize your text step:

   • Select your Azure OpenAI service.
   • Choose text-embedding-ada-002 as the embedding model. The AZD template already deployed this model for you.
   • Select System assigned identity for authentication.
   • Check the acknowledgment checkbox for additional costs.
   • Select Next.

   Tip

   Learn more about Vector search in Azure AI Search and Text embeddings in Azure OpenAI.

8. In the Vectorize and enrich your images step:

   • Keep the default settings.
   • Select Next.

9. In the Advanced settings step:

   • Ensure Enable semantic ranker is selected.
   • (Optional) Select an indexing schedule. This is useful if you want to refresh your index regularly with the latest file changes.
   • Select Next.

10. In the Review and create step:

    • Copy the Objects name prefix value. It is your search index name.
    • Select Create to start the indexing process.

11. Wait for the indexing process to complete. This might take a few minutes depending on the size and number of your documents.

12. To test the data import, select Start searching and try a search query like "Tell me about your company."

13. Back in your Codespace terminal, set the search index name as an AZD environment variable:

    azd env set SEARCH_INDEX_NAME <your-search-index-name>
    

    Replace <your-search-index-name> with the index name you copied previously. AZD uses this variable in subsequent deployments to set the App Service app setting.

4. Test the application and deploy

If you prefer to test the application locally before or after deployment, you can run it directly from your Codespace:

1. In your Codespace terminal, get the AZD environment values:

   azd env get-values
   

2. Open appsettings.Development.json. Using the terminal output, update the values of OpenAIEndpoint, SearchServiceUrl, and SearchIndexName.

3. Sign in to Azure with the Azure CLI:

   az login
   

   This allows the Azure Identity client library in the sample code to receive an authentication token for the logged in user.

4. Run the application locally:

   dotnet run
   

5. When you see Your application running on port 5017 is available, select Open in Browser.

6. Try asking a few questions in the chat interface. If you get a response, your application is connecting successfully to the Azure OpenAI resource.

7. Apply the new SEARCH_INDEX_NAME configuration in Azure and deploy the sample application code:

   azd up
   

5. Test the deployed RAG application

With the application fully deployed and configured, you can now test the RAG functionality:

1. Open the application URL provided at the end of the deployment.

2. You see a chat interface where you can enter questions about the content of your uploaded documents.

   

3. Try asking questions that are specific to the content of your documents. For example, if you uploaded the documents in the sample-docs folder, you can try out these questions:

   • How does Contoso use my personal data?
   • How do you file a warranty claim?

4. Notice how the responses include citations that reference the source documents. These citations help users verify the accuracy of the information and find more details in the source material.

   

5. Test the hybrid search capabilities by asking questions that might benefit from different search approaches:

   • Questions with specific terminology (good for keyword search).
   • Questions about concepts that might be described using different terms (good for vector search).
   • Complex questions that require understanding context (good for semantic ranking).

Clean up resources

When you're done with the application, you can delete all the resources to avoid incurring further costs:

azd down --purge

This command deletes all resources associated with your application.

Frequently asked questions

• How does the sample code retrieve citations from Azure OpenAI chat completions?
• What's the advantage of using managed identities in this solution?
• How is the system-assigned managed identity used in this architecture and sample application?
• How is hybrid search with semantic ranker implemented in the sample application?
• Why are all resources created in East US 2?
• Can I use my own OpenAI models instead of the ones provided by Azure?
• How can I improve the quality of responses?

How does the sample code retrieve citations from Azure OpenAI chat completions?

The sample retrieves citations by using the AzureSearchChatDataSource() as the data source for the chat client. When a chat completion is requested, the response includes a Citations object within the message context. The code extracts these citations as follows:

var result = await _chatClient.CompleteChatAsync(messages, options);

var ctx = result.Value.GetMessageContext();

var response = new ChatResponse
{
    Content = result.Value.Content,
    Citations = ctx?.Citations
};

return response;

In the chat response, the content uses [doc#] notation to reference the corresponding citation in the list, allowing users to trace information back to the original source documents. For more information, see:

• Azure OpenAI On Your Data API Reference
• Data source - Azure AI Search

What's the advantage of using managed identities in this solution?

Managed identities eliminate the need to store credentials in your code or configuration. By using managed identities, the application can securely access Azure services like Azure OpenAI and Azure AI Search without managing secrets. This approach follows Zero Trust security principles and reduces the risk of credential exposure.

How is the system-assigned managed identity used in this architecture and sample application?

The AZD deployment creates system-assigned managed identities for Azure App Service, Azure OpenAI, and Azure AI Search. It also makes respective role assignments for each of them (see the main.bicep file). For information on the required role assignments, see Network and access configuration for Azure OpenAI On Your Data.

In the sample application, the Azure SDKs use this managed identity to authenticate requests securely, without storing credentials or secrets anywhere. For example, the AzureOpenAIClient is initialized with DefaultAzureCredential, which uses the managed identity when running in Azure:

_openAIClient = new AzureOpenAIClient(
    new Uri(_settings.OpenAIEndpoint),
    new DefaultAzureCredential()
);

Similarly, when configuring the data source for Azure AI Search, the managed identity is specified for authentication:

options.AddDataSource(new AzureSearchChatDataSource()
{
    Endpoint = new Uri(_settings.SearchServiceUrl ?? throw new ArgumentNullException(nameof(_settings.SearchServiceUrl))),
    IndexName = _settings.SearchIndexName,
    Authentication = DataSourceAuthentication.FromSystemManagedIdentity(), // Use system-assigned managed identity
    // ...
});

This enables secure, passwordless communication between the Blazor app and Azure services, following best practices for Zero Trust security. Learn more about DefaultAzureCredential and Azure Identity client library for .NET.

How is hybrid search with semantic ranker implemented in the sample application?

The sample application configures hybrid search with semantic ranking using the Azure.AI.Search.Documents SDK. In the backend, the data source is set up as follows:

options.AddDataSource(new AzureSearchChatDataSource()
{
    // ...
    QueryType = DataSourceQueryType.VectorSemanticHybrid, // Combines vector search with keyword matching and semantic ranking
    VectorizationSource = DataSourceVectorizer.FromDeploymentName(_settings.OpenAIEmbeddingDeployment),
    SemanticConfiguration = _settings.SearchIndexName + "-semantic-configuration", // Build semantic configuration name from index name
});

This configuration enables the application to combine vector search (semantic similarity), keyword matching, and semantic ranking in a single query. The semantic ranker reorders the results to return the most relevant and contextually appropriate answers, which are then used by Azure OpenAI for generating responses. The semantic configuration name is automatically defined by the integrated vectorization process. It uses the search index name as the prefix and appends -semantic-configuration as the suffix. This ensures that the semantic configuration is uniquely associated with the corresponding index and follows a consistent naming convention.

Why are all resources created in East US 2?

The sample uses the gpt-4o-mini and text-embedding-ada-002 models, both of which are available with the Standard deployment type in East US 2. These models are also chosen because they aren't scheduled for retirement soon, providing stability for the sample deployment. Model availability and deployment types can vary by region, so East US 2 is selected to ensure the sample works out of the box. If you want to use a different region or models, make sure to select models that are available for the same deployment type in the same region. When choosing your own models, check both their availability and retirement dates to avoid disruptions.

• Model availability: Azure OpenAI Service models
• Model retirement dates: Azure OpenAI Service model deprecations and retirements.

Can I use my own OpenAI models instead of the ones provided by Azure?

This solution is designed to work with Azure OpenAI Service. While you could modify the code to use other OpenAI models, you would lose the integrated security features, managed identity support, and the seamless integration with Azure AI Search that this solution provides.

How can I improve the quality of responses?

You can improve response quality by:

• Uploading higher quality, more relevant documents.
• Adjusting chunking strategies in the Azure AI Search indexing pipeline. However, you can't customize chunking with the integrated vectorization shown in this tutorial.
• Experimenting with different prompt templates in the application code.
• Fine-tuning the search with other properties in the AzureSearchChatDataSource class.
• Using more specialized Azure OpenAI models for your specific ___domain.

More resources

• Explore hybrid search capabilities in Azure AI Search
• Use Azure OpenAI On Your Data
• .NET Client SDK for Azure OpenAI Service