AWS Bedrock Healthcare Data Assistant

Overview

Client: Healthcare data company

How we helped a United States healthcare data company leverage AWS Bedrock and gpt-oss-120b to simplify access to complex healthcare data through a single AI-powered conversational interface. Healthcare data questions that once required navigating multiple systems could now be answered through a single conversational interface, available 24/7 within the platform.

Business challenges

• Reference information was fragmented: Information on billing codes (CPT, HCPCS, MS-DRG), diagnosis codes (ICD-10), modifiers, and general healthcare terminology, coding conventions, and regulatory concepts was spread across numerous systems, sources, and formats. Much of it, particularly on CMS and related government websites, is difficult to locate even though it is publicly available.
• Not everyone had the same starting point: The client’s existing platform gave users powerful filtering and query capabilities, but effective use required already knowing the right terms and classifications.
• Users had no natural-language way in: Reference information could be found through the platform’s existing interfaces and workflows, but there was no way to simply ask a question in plain language and get a direct answer.

Technical challenges

• Staying tightly scoped by controlling model behaviour: Healthcare terminology, codes, and classifications are complex, and a general-purpose language model can easily wander outside the intended use case. The assistant needed clear boundaries around what it should and shouldn’t answer, functioning more like an intelligent help layer than an analysis or query engine.
• Enforcing those boundaries without limiting capability: The assistant needed to behave predictably and stay within scope, while still giving users useful, well-scoped answers drawn from the model’s own trained knowledge.
• Enterprise privacy and access control: The solution needed to operate within the client’s existing platform, using AWS’s identity and permission tools to manage access appropriately, and avoid exposing user queries or data to the underlying model provider.

The solution

We designed and built a conversational AI assistant embedded directly into the client’s platform using AWS Bedrock and gpt-oss-120b (openai.gpt-oss-120b-1:0), extending a platform we had already developed. Instead of piecing together answers from multiple documentation sources and reference sites, users can simply ask a question in plain language and receive a direct answer within the same interface they already use to query the underlying data.

We chose gpt-oss-120b, an open-weight mixture-of-experts model, for its strong reasoning performance, its large context window for multi-turn healthcare queries, and its cost efficiency at scale. Rather than using all of its parameters on every query, the model activates only a small portion at a time. This keeps inference costs low, while still handling complex reasoning well. The assistant’s scope is controlled through structured prompt engineering: a system-level prompt defines its domain and restricts the answers the model is able to provide. Out-of-scope questions get a defined decline-and-redirect response instead of an open-ended answer. Built on AWS Bedrock, the solution also benefits from enterprise-grade privacy and control. Queries and data are not used to train the underlying models, and AWS’s identity and permission management tools help ensure the solution operates securely within the client’s platform. There is also room to add future capabilities such as RAG, knowledge base integrations, and governance controls should those needs arise.

What we built

• Conversational AI interface: Extended the client’s platform, which we had also built, with a natural-language interface, allowing users to ask common healthcare data questions without searching across separate systems or documentation sources.
• AWS Bedrock integration: Integrated the application with AWS Bedrock to provide managed access to the underlying language model.
• gpt-oss-120b implementation: Configured openai.gpt-oss-120b-1:0 to process user queries and generate responses within the application’s defined scope.
• Prompt-based controls: Developed structured system instructions to define the assistant’s domain, response behaviour, and handling of out-of-scope questions.
• Custom context handling: Built a custom solution to carry relevant context across follow-up questions and to cap the number of exchanges per session, controlling cost and discouraging off-scope use, since we called the Bedrock model API directly rather than using Bedrock Agents, which add cost. The model API is stateless, so conversational memory and per-session limits had to be built in.
• Access controls: Integrated the assistant into the client’s existing application environment, with access managed through AWS identity and permission controls.
• Foundation for future expansion: Designed the implementation so that RAG, knowledge bases, and additional governance mechanisms can be introduced as the client’s requirements expand.

Results

• Faster answers: Common reference questions are typically answered in under 10 seconds, directly within the platform.
• 24/7 access and availability: Users can ask healthcare reference questions at any time, without leaving the application.
• Plain-language access: Complex healthcare data terminology and classifications can be explained in response to a simple, natural-language question.
• Consistent and predictable behaviour: Structured prompt engineering keeps responses consistent and within scope.
• Scalable foundation with room to grow: The implementation provides a starting point for future capabilities including RAG and knowledge base integrations, if and when the client needs them.