Build a Chat API with ASP.NET Core and an LLM

Build a Chat API with ASP.NET Core and an LLM Learn how to build a chat API with ASP.NET Core and an LLM using Microsoft.Extensions.AI: conversation history, ownership, failure handling, and testing. build-chat-api-aspnet-core-llm chat API with ASP.NET Core Introduction In the previous guide, we built the foundation of an AI application with ASP.NET Core and kept the LLM as a dependency inside the application, not the owner of authentication, business logic, and data access. Now we'll build something users can interact with: a chat API with ASP.NET Core. A chat interface is usually the first feature people associate with an AI application, but sending a message to a model is only one part of the workflow. The API has to accept input, maintain a conversation, call the model, handle failures, and return a response the frontend can consume. Once several users arrive, conversation ownership and persistence matter too. We'll use ASP.NET Core, C#, and Microsoft.Extensions.AI, starting with a basic endpoint and then adding conversation history, a separate service layer, and fixes for problems that appear when a prototype becomes a real application. The architecture stays small: no agent framework or vector database is needed to make a chat endpoint work. What We Are Building Imagine an AI assistant in a developer support platform. A user asks, "What is dependency injection in ASP.NET Core?", then "Explain scoped versus singleton services," then "Show me a small example using an interface." The assistant should understand these belong to one conversation. The API exposes two operations: create a conversation, and send a message to an existing one. Frontend → ASP.NET Core API → Chat Service → IChatClient → LLM One boundary doesn't change: the frontend never talks to the model provider directly. All model requests go through our backend. 1. Create the Project and Define the Contract dotnet new webapi -n FoundryChat cd FoundryChat Install Microsoft.Extensions.AI and the provider package for your model (Ollama, Azure OpenAI, or another supported integration). The examples use the IChatClient abstraction, so the application service stays independent of the provider, which helps if you later move from a local model to a hosted one. Decide what the API accepts and returns first: public sealed record SendMessageRequest(string Message); public sealed record ChatResponse(Guid ConversationId, string Message); Returning your own DTO instead of the model's raw response keeps the API contract stable and leaves room to add metadata later. 2. Register the AI Client The client belongs in dependency injection: builder.Services.AddSingleton(sp => { // Create the provider-specific chat client here. // Configure the model and endpoint. // Return the client as IChatClient. }); That's an architectural sketch, not a complete provider registration, because the constructor and package setup depend on your provider. The chat service receives IChatClient through its constructor, and a controller never needs to know which model runs or how the provider authenticates. 3. The First Chat Endpoint using Microsoft.Extensions.AI; public sealed class ChatService(IChatClient chatClient) { public async Task SendAsync(string message, CancellationToken cancellationToken) { var response = await chatClient.GetResponseAsync(message, cancellationToken: cancellationToken); return response.Text ?? string.Empty; } } builder.Services.AddScoped(); [ApiController] [Route("api/chat")] public class ChatController(ChatService chatService) : ControllerBase { [HttpPost] public async Task Send(SendMessageRequest request, CancellationToken cancellationToken) { var response = await chatService.SendAsync(request.Message, cancellationToken); return Ok(new { message = response }); } } The API now returns an LLM response, with an obvious limitation: every request is independent. The model has no idea what the user asked before. 4. Why a Chat API Needs Conversation History User: What is dependency injection? Assistant: A technique for supplying dependencies to a class instead of having it create them. User: Show me an example in C#. The last message means nothing on its own, and the model may produce an unrelated example. The application has to send the previous messages along with the new one. Microsoft.Extensions.AI represents them as ChatMessage: var messages = new List { new(ChatRole.User, "What is dependency injection?"), new(ChatRole.Assistant, "Dependency injection supplies dependencies to a class."), new(ChatRole.User, "Show me an example in C#.") }; var response = await _chatClient.GetResponseAsync(messages, cancellationToken: cancellationToken); This is the essential difference between a basic LLM endpoint and a conversational API: the application owns the history, and the model uses it to generate the next response. 5. Conversation IDs and Storage Sending the whole conversation from the frontend on every request creates trust and consistency problems, so the backend should own it. A new conversation gets an ID: public sealed record CreateConversationResponse(Guid ConversationId); [HttpPost("conversations")] public IActionResult CreateConversation() => Ok(new CreateConversationResponse(Guid.NewGuid())); The frontend then calls POST /api/chat/conversations/{id}/messages with {"message": "Explain dependency injection with an example."}. But generating an ID isn't storing a conversation, so we need somewhere to keep the messages. For a first implementation, an in-memory store is enough to understand the workflow: public sealed class ChatConversation { public Guid Id { get; init; } public List Messages { get; } = []; } public sealed class ConversationStore { private readonly ConcurrentDictionary _conversations = new(); public ChatConversation Create() { var conversation = new ChatConversation { Id = Guid.NewGuid() }; _conversations[conversation.Id] = conversation; return conversation; } public bool TryGet(Guid id, out ChatConversation conversation) => _conversations.TryGetValue(id, out conversation!); } builder.Services.AddSingleton(); That's for a prototype or single-process demo only. An in-memory store loses data on restart and doesn't make concurrent updates to one conversation safe. For a real application, persist conversations and messages in a database such as SQL Server. 6. Connect the Store to the Chat Service public sealed class ChatService(IChatClient chatClient, ConversationStore store) { public async Task SendAsync(Guid conversationId, string message, CancellationToken cancellationToken) { if (!store.TryGet(conversationId, out var conversation)) throw new KeyNotFoundException("Conversation was not found."); conversation.Messages.Add(new ChatMessage(ChatRole.User, message)); var response = await chatClient.GetResponseAsync( conversation.Messages, cancellationToken: cancellationToken); var assistantMessage = response.Text ?? string.Empty; conversation.Messages.Add(new ChatMessage(ChatRole.Assistant, assistantMessage)); return assistantMessage; } } The flow: find the conversation, append the user's message, send the history to the model, append the assistant's reply, and return it. The model stores nothing. The application controls the history and decides what goes into each request. The controller endpoint: [HttpPost("conversations/{conversationId:guid}/messages")] public async Task SendMessage( Guid conversationId, SendMessageRequest request, CancellationToken cancellationToken) { if (string.IsNullOrWhiteSpace(request.Message)) return BadRequest("Message cannot be empty."); try { var response = await _chatService.SendAsync(conversationId, request.Message, cancellationToken); return Ok(new ChatResponse(conversationId, response)); } catch (KeyNotFoundException) { return NotFound("Conversation was not found."); } } 7. Validate Requests and Protect Conversations An AI endpoint is still an HTTP endpoint. Validate empty messages, maximum length, conversation identifiers, request rate, and authentication: public sealed record SendMessageRequest( [Required] [StringLength(4000, MinimumLength = 1)] string Message); The right limit depends on your application and the model's context window. Treat user messages as untrusted input: they should never override authorization or expose secrets from the system prompt. A conversation ID is not an authorization mechanism. If one user can reach another's conversation by guessing or obtaining its ID, you have a serious data exposure problem. Associate each conversation with an authenticated user (ConversationId, UserId, CreatedAt, UpdatedAt), and verify ownership on every message request: var userId = User.FindFirst(ClaimTypes.NameIdentifier)?.Value; Query by both the conversation ID and the user ID. Never fetch by ID alone and assume the caller is allowed to use it. This matters most when conversations hold private customer data or internal business information. 8. Handle Timeouts and Failures The model call is a remote dependency. The provider can be unavailable, overloaded, rate-limiting you, or slow, and the connection can drop. Pass the cancellation token through the whole request path, as in SendAsync above, and distinguish client cancellation, provider failure, and application errors. Don't return raw provider exception details to the frontend: return a meaningful error and log the technical detail internally. Retry deliberately, because retrying every failed model request adds latency and cost without fixing the cause. 9. Don't Send Unlimited History Right now every request sends the entire conversation. That's fine for a short chat, but with hundreds of messages it raises token use, latency, and context-window pressure, and earlier information gets lost among newer messages. Introduce a context strategy: Keep the most recent messages. Keep a compact summary of older ones. Retrieve relevant earlier messages when needed. Apply a token budget before sending. Stored history and model context don't have to be identical: the database can keep the full conversation while the model receives only a subset. 10. Test Without a Live LLM Unit tests shouldn't need a real model. Because the service depends on IChatClient, you can substitute a fake that returns a predictable response, and verify that a conversation is created, user messages are stored, the assistant reply is appended, missing conversations return an error, empty messages are rejected, and cancellation propagates. Answer quality needs a separate evaluation process: a successful HTTP response doesn't prove the assistant answered correctly, so use representative questions and expected behavior. 11. What to Add Before Production I wouldn't deploy the in-memory version for a multi-user application. A production chat API needs more around the same core design: Component Responsibility Authentication Identify the requesting user Authorization Enforce conversation ownership SQL Server Persist conversations and messages Chat Service Coordinate the conversational workflow IChatClient Communicate with the selected model Context Manager Control message history and token limits Rate Limiting Protect the API and model budget Logging Diagnose requests without exposing sensitive content Error Handling Return controlled failures Evaluation Measure answer quality and regressions None of this requires turning the chat endpoint into an autonomous agent. The final request path is: Frontend → ASP.NET Core API → Authentication → Conversation Service → Context Preparation → IChatClient → LLM The response comes back through the application, where the assistant message is stored before it's returned. SQL Server can replace the in-memory store without changing the chat service's responsibility. That's the advantage of keeping the model integration behind an application boundary. Common Mistakes Sending only the latest message, so follow-up questions lose context. Keeping conversations in memory, so restarts destroy history and multiple instances can't share it. Trusting conversation IDs as proof of ownership. Passing unlimited history, which gets expensive and slow. Calling the model from the controller, mixing AI orchestration with HTTP handling. Retrying every provider error. Testing only against live models, which is slow, expensive, and unpredictable. Logging every prompt and response, which puts sensitive user data in your logs. Conclusion A chat API with ASP.NET Core is relatively simple once the model integration is separated from the application. The interesting decisions come later: maintaining conversations, protecting user data, managing context, handling model failures, and supporting multiple users. Start with a small chat service and a clear conversation model, persist history when the application needs it, and add context management before conversations get large. The same foundation can support a knowledge assistant, a customer-support chatbot, or an AI feature inside an existing business application. A natural next step is streaming, so users see the answer as it's generated instead of waiting for the whole response. References Microsoft Learn: AI apps for .NET developers Microsoft Learn: Microsoft.Extensions.AI Microsoft Learn: ASP.NET Core Web API Microsoft Learn: Authentication and authorization in ASP.NET Core More practical AI and .NET engineering guides: AI Engineering Foundry