ASP.NET Core Background Services Prevent AI Processing Timeouts

A founder building an AI meeting assistant used background services to prevent 504 gateway timeouts when processing 20-minute audio files, offloading heavy AI tasks from API controllers.

A founder developing an AI Meeting Assistant for a university project encountered a critical architectural challenge: processing 20-minute audio recordings for transcription and summarization consistently triggered 504 Gateway Timeouts after 90 seconds. The core task involved downloading audio from Backblaze B2, sending it to a self-hosted transcription service, then passing the resulting text to Google Gemini for minute generation, and finally emailing the output. This entire sequence, requiring over a minute, could not execute within a standard API controller's request lifecycle.

The solution implemented was ASP.NET Core Background Services. This approach decoupled the long-running AI operations from the immediate HTTP request, allowing the user's browser to receive an immediate response while processing continued asynchronously. The strategy addresses a common bottleneck for applications integrating intensive AI workloads.

Offloading Long-Running AI Tasks

The fundamental problem stemmed from the nature of HTTP requests and server timeouts. A user uploading a 20-minute audio file initiated a sequence of operations: download, transcription, AI summarization, and email delivery. This entire chain took "over a minute" to complete. Standard API controllers are designed for rapid request-response cycles. When a process exceeds typical server or proxy timeouts, often around 90 seconds, the connection is severed, resulting in a 504 Gateway Timeout error for the client.

The founder explicitly compared the API Controller to a "Cashier" in a fast-food restaurant, taking an order but not cooking the food. If the cashier (controller) attempts to cook (process the AI task), the customer line (subsequent requests) backs up and eventually leaves. This analogy highlights the blocking nature of synchronous processing within a controller. The user's browser would display a loading spinner for the entire duration, only to fail.

Implementing BackgroundService for Processing

The architectural shift involved introducing an ASP.NET Core BackgroundService. Unlike controllers, which are short-lived and instantiated per request, a BackgroundService is long-lived. It starts once with the application and runs continuously until the application shuts down. This design allows it to operate independently of incoming HTTP requests.

For the AI Meeting Assistant, a TranscriptionWorker class was created, inheriting from BackgroundService. The core logic resides within its ExecuteAsync method. This method runs in a continuous loop, checking for new tasks. When a user uploads an audio file, the API controller's role is reduced to simply accepting the file and perhaps enqueueing a task for the TranscriptionWorker to pick up. The controller then immediately returns a response to the user, indicating that processing has begun.

The TranscriptionWorker then handles the heavy lifting: fetching the audio from Backblaze B2, sending it to the self-hosted transcription service, passing the transcribed text to Google Gemini for summarization, and finally dispatching the meeting minutes via email. This asynchronous execution ensures the web server remains responsive, preventing timeouts and improving user experience. The source provides a skeleton of the TranscriptionWorker class, demonstrating the basic structure for implementing this pattern in .NET.

The Workflow: From Upload to Email

The complete workflow, as described, begins with a user uploading an audio recording. This audio is stored on Backblaze B2. The API controller receives the upload request, likely stores metadata about the job, and then hands off the actual processing to the background service. The background service, running its ExecuteAsync loop, detects the new job. It retrieves the audio from Backblaze B2, initiating the transcription process. This transcription, performed by a self-hosted service, converts speech to text.

Once transcription is complete, the text is forwarded to Google Gemini. Gemini then generates structured meeting minutes based on the transcribed content. The final step involves emailing these generated minutes to the user. This multi-stage process, with distinct external dependencies and computational requirements, is precisely what necessitated the background service pattern. It ensures that each computationally intensive step can run to completion without blocking the main application thread or exceeding HTTP request limits.

What We'd Change

While the use of ASP.NET Core Background Services effectively solves the immediate 504 timeout problem, several considerations arise for a production-grade application beyond a university project. The source mentions encountering "3 massive 'Gotchas'" but does not detail them. Without this context, a full assessment of potential pitfalls and their solutions is limited. However, common challenges with this pattern include managing worker state, handling transient errors, and ensuring idempotent processing.

For instance, the ExecuteAsync method's while (!stoppingToken.IsCancellationRequested) loop implies a polling mechanism. In a high-throughput system, a more robust approach would involve a dedicated message queue (e.g., RabbitMQ, Azure Service Bus, AWS SQS) to decouple the API controller from the background worker. The controller would publish a message to the queue, and the background worker would consume messages, providing better scalability, reliability, and retry mechanisms. This also simplifies horizontal scaling of workers.

The choice of a "self-hosted transcription service" introduces operational overhead. While potentially cost-effective for a project, a commercial product would need to weigh the maintenance burden, scalability, and reliability of a self-hosted solution against managed services (e.g., AWS Transcribe, Google Cloud Speech-to-Text). These managed services often offer higher accuracy, lower latency, and built-in scalability, albeit at a potentially higher direct cost.

Finally, the user experience for long-running tasks can be further enhanced. Instead of merely returning an immediate response, the API could provide a job ID. The client could then poll a status endpoint with this ID or receive real-time updates via WebSockets or Server-Sent Events, offering transparency into the processing progress. This moves beyond simply avoiding timeouts to actively informing the user about their request's status.

Offloading long-running processes to background services is a fundamental pattern for building responsive and scalable applications, particularly when integrating intensive AI workloads. The founder's implementation in ASP.NET Core demonstrates a direct solution to a common timeout issue. This approach ensures that user-facing interactions remain fluid while complex, time-consuming tasks execute reliably behind the scenes. The core principle of decoupling synchronous requests from asynchronous processing applies across various technology stacks and scales, serving as a foundational strategy for modern application architecture.

Pull quote: “The controller then immediately returns a response to the user, indicating that processing has begun.”