SharpAI

SharpAI is a local, OpenAI-compatible AI inference server built for the .NET world. It lets applications use large language models and speech-to-text through the exact API they already know — while every token is processed on infrastructure the owner controls.

The challenge

Two forces pull against each other in most AI adoption. Teams want the ergonomics and ecosystem of the OpenAI API, but many cannot send their data to a third-party cloud for privacy, compliance or cost reasons. On top of that, .NET shops are under-served: the local-inference tooling ecosystem is overwhelmingly Python-first. The goal was a server that is a genuine drop-in for the OpenAI API, runs models locally, and feels native to a C#/.NET stack.

The approach

I built the server on .NET 9, implementing the OpenAI wire format for chat completions, audio transcription and model management so existing clients need no changes. Local LLM inference runs through LLamaSharp; speech-to-text through Whisper.NET. Because the API is compatible at the field level, it behaves as a true replacement — and I extended it where local hosting can offer more than the cloud:

• Standard OpenAI fields (content, role, tool_calls…) plus streaming.
• Extended channels — a reasoning stream for chain-of-thought and a commentary stream for meta-insight — giving applications transparency into how an answer was formed.
• Model management so the same server hosts and swaps between local models.

The outcome

SharpAI makes local AI a first-class option for .NET teams: existing OpenAI clients point at it unchanged, no data leaves the building, and developers gain visibility into model behaviour that a closed cloud API cannot provide.