Grounding specialty-pharmacy answers in approved content with hybrid retrieval, citations, privacy controls, and human oversight.

By Reshma Thakkallapelly · 8–10 minute read

Specialty-pharmacy teams do not need another chatbot that always produces an answer. They need a knowledge assistant that can locate the current approved source, enforce access rules, cite the evidence, and stop when the available evidence is insufficient.

This article presents a practical, public-safe account of designing a retrieval-augmented generation assistant for specialty-pharmacy teams. The architecture separates approved knowledge retrieval from patient-specific transactions and emphasizes permission-aware search, source citations, abstention, evaluation, and human escalation.

The Problem Was Not a Lack of Information

Specialty-pharmacy teams work with information that is detailed, time-sensitive, and distributed across many sources.

BioPlus publicly describes services that support patients and providers through specialty-medication referrals, fills, refills, financial-assistance resources, and therapy coordination. Behind those experiences, team members may need to navigate:

• Standard operating procedures
• Job aids and training materials
• Referral forms
• Drug-specific guides
• Payer and program requirements
• Frequently asked questions
• Escalation procedures
• Operational policies

The operational question is rarely:

“Do we have a document about this?”

The harder questions are:

• Which document is approved?
• Is it still effective?
• Does this user have permission to view it?
• Which section actually answers the question?
• What should happen when the sources are incomplete, outdated, or conflicting?

A generic large language model is a poor fit for that problem. It may generate a fluent answer without knowing the current approved procedure—and fluency is not evidence.

I therefore designed the assistant around retrieval-augmented generation, or RAG: retrieve relevant passages from an approved knowledge base at question time, then ask the model to answer only from that evidence and show the supporting sources.

The Operating Rule

What I Designed the Assistant to Do

The assistant was designed to:

• Find and summarize approved SOPs, job aids, referral guidance, and training content.
• Return the source title, section or page, owner, effective date, and link with each answer.
• Apply user-role and document-level permissions before content reaches the language model.
• Recognize unsupported, restricted, or patient-specific questions.
• Route questions outside its scope to an authorized workflow or human expert.
• Capture feedback and evaluation signals without copying sensitive conversation content into unrestricted logs.

What It Was Not Designed to Do

The assistant was not positioned as a diagnostician, prescriber, pharmacist, clinical reviewer, or final prior-authorization or coverage decision-maker.

It could explain approved operational knowledge and direct users to relevant source material. It could not:

• Invent missing policy
• Override clinical judgment
• Make independent prescribing or coverage decisions
• Expose patient information merely because a user requested it
• Treat an unsupported answer as established fact
• Replace the applicable system of record or qualified human reviewer

The Late-2024 Technology Baseline

The following implementation pattern reflects technologies available in late 2024. The exact services can be substituted without changing the architectural controls.

Experience and Identity

• React or Next.js web application, or Microsoft Teams
• Microsoft Entra ID
• Single sign-on
• Azure API Management
• Managed identities

Orchestration and Model

• Java 21
• Spring Boot 3.2
• Azure software development kits
• Azure OpenAI GPT-4o or another approved GPT-4-class deployment

Retrieval Layer

• Azure OpenAI embeddings
• Azure AI Search
• Keyword retrieval
• Vector retrieval
• Metadata filtering
• Semantic ranking

Ingestion and State

• Azure Blob Storage
• Event Grid or Azure Functions
• Azure AI Document Intelligence
• PostgreSQL
• Optional short-lived Redis caching

Security Controls

• Azure Key Vault
• Private endpoints
• Role-based access control
• Network segmentation
• Data-loss-prevention scanning
• Encryption
• Audit controls

Observability and Delivery

• OpenTelemetry
• Application Insights
• Log Analytics
• Curated evaluation sets
• Red-team test suites
• CI/CD
• Terraform or Bicep
• Policy gates

Reference Architecture

The design separates the knowledge plane from the patient-transaction plane.

Approved documents can be indexed for search. Patient and referral records remain in their systems of record and are accessed only through authorized APIs when a permitted workflow requires current data.

The architecture contains five primary layers:

1. Approved knowledge sources
2. Ingestion, classification, and lifecycle governance
3. Permission-aware knowledge retrieval
4. Authenticated query and RAG orchestration
5. Grounded generation and human oversight

Patient-specific transactions remain outside the shared knowledge index, while managed identity, private networking, access controls, audit logging, observability, evaluation, and red-team testing operate across the platform.

1. Register Approved Sources

Every document enters through a governed source registry rather than an uncontrolled file drop.

Required metadata includes:

• Source owner
• Document type
• Intended audience
• Therapy or business area
• Effective date
• Review date
• Confidentiality level
• Permitted access groups
• Lifecycle status

A document without an accountable owner or clear lifecycle status is not eligible to become trusted grounding content.

This is important because a RAG assistant cannot determine whether a document is authoritative merely because the document exists.

2. Extract Structure, Not Just Text

The ingestion pipeline preserves:

• Headings
• Lists
• Tables
• Page numbers
• Section boundaries
• Document versions
• Source anchors

Azure AI Document Intelligence can extract text and structured elements from forms and documents. That is more useful for RAG than flattening every file into one large text block.

I used section-aware chunks so that a procedure, warning, exception, table, and related instructions did not become disconnected fragments.

The objective was not simply to create chunks of equal size. The objective was to preserve the meaning and operational context of each section.

3. Build a Permission-Aware Hybrid Index

Each indexed chunk contains:

• Searchable text
• An embedding
• Document metadata
• Access-control information
• Effective and review dates
• Page or section anchors
• Source ownership information

Hybrid retrieval matters because specialty-pharmacy questions combine natural-language intent with exact terminology such as:

• Drug names
• Program names
• Form titles
• Identifiers
• Abbreviations
• Payer terminology
• Operational codes

Keyword search helps locate exact terms. Vector search helps locate semantically similar passages and paraphrases. Semantic reranking improves the selection of passages that ultimately reach the model.

4. Ground the Answer and Validate the Citations

The model receives only the highest-ranked authorized passages, together with explicit instructions not to rely on unsupported knowledge.

Every material answer should identify the relevant source.

A post-generation validator verifies that:

• Every cited source identifier exists in the retrieved evidence.
• The cited document version is current.
• Page or section references resolve correctly.
• The answer does not claim more than the retrieved passages support.
• Restricted content has not entered the response.
• The response remains within the assistant’s approved purpose.

A citation is not decorative. It must lead the user to the correct source, document version, and relevant section.

5. Keep Patient-Specific Operations Outside the Shared Index

This was one of the most important architectural boundaries.

A general knowledge index should not become a shadow patient database.

When an authorized user needs a current referral status or another patient-specific fact, the orchestrator may call an approved transactional API using per-request authorization.

Those results are used only for the authorized interaction. They are not silently embedded into the shared knowledge corpus.

This separation makes the security model easier to understand:

• Approved operational knowledge belongs in the governed knowledge plane.
• Current patient and referral information remains in its system of record.
• Patient-specific access occurs only when required.
• Authorization is evaluated for every transactional request.
• Transaction results are not automatically retained as reusable grounding content.

The Query Pipeline in Practice

A typical request moves through:

1. Authentication
2. Query classification
3. Permission filtering
4. Hybrid retrieval
5. Semantic reranking
6. Grounded generation
7. Citation validation
8. Abstention or escalation
9. Feedback capture

The orchestration logic should remain simple enough to test.

def answer(user, question):
    claims = identity.authorize(user)

    hits = search.hybrid(
        question,
        filter=acl_filter(claims) & current_documents(),
        top=20,
    )

    evidence = rerank(hits)[:5]

    if evidence_is_weak(evidence):
        return abstain_or_escalate()

    draft = model.generate(
        question,
        evidence,
        citations=True,
    )

    return validate_or_abstain(draft, evidence)

This pseudocode is intentionally simple. It illustrates the responsibility boundaries rather than a complete production implementation.

Illustrative Answer

Question

What information is normally required before benefits investigation can begin for a new specialty referral?

Illustrative response

This example is synthetic and is not an actual BioPlus policy or instruction.

Healthcare Guardrails That Mattered

Hallucinated Procedure

Control: Answer only from retrieved, approved passages. Require citations, apply evidence thresholds, and abstain when the source support is insufficient.

Outdated Content

Control: Store effective dates and review dates, prefer the current approved version, and suppress or clearly label superseded documents.

Unauthorized Disclosure

Control: Enforce identity and document access controls before retrieval. Test for cross-role leakage and use least-privilege service identities.

Protected Health Information in Prompts or Logs

Control: Avoid bulk patient indexing. Redact or tokenize sensitive information where appropriate and restrict telemetry content, access, and retention.

Prompt Injection in Retrieved Documents

Control: Scan ingested content, separate instructions from evidence, prevent retrieved text from independently invoking tools, and allowlist tool actions.

Clinical or Coverage Overreach

Control: Define the assistant as a knowledge-navigation system. Route decisions, exceptions, and patient-specific determinations to qualified humans and authoritative systems of record.

Unverifiable Responses

Control: Return the source title, page or section, effective date, document version, and resolvable link. Fail closed when citation validation or source support is weak.

The HIPAA Security Rule addresses appropriate access controls, audit controls, integrity protections, authentication, and transmission security for electronic protected health information. The NIST Generative AI Profile provides additional guidance for structured generative-AI risk management, testing, and governance.

How I Evaluated the Assistant

I did not treat a fluent demonstration as evidence of readiness.

The evaluation set needed:

• Questions written or reviewed by subject-matter experts
• Expected source passages
• Expected citations
• Permission scenarios
• Unsupported and ambiguous questions
• Restricted-content tests
• Explicit rules for acceptable abstention
• Regression cases covering previously accepted answers

Retrieval Quality

• Does the correct source appear among the top results?
• Are current-document filters applied correctly?
• Are permission filters applied correctly?
• Does hybrid retrieval outperform keyword-only or vector-only retrieval for the relevant question types?

Groundedness

• Can each material statement in the answer be traced to retrieved text?
• Does the response avoid adding unsupported details?
• Does it distinguish source-supported information from uncertainty?

Citation Accuracy

• Do source links resolve?
• Do page and section references point to the relevant passage?
• Is the correct document version identified?
• Are cited sources actually present in the retrieved evidence?

Abstention Behavior

• Does the assistant refuse or escalate unsupported questions?
• Does it recognize ambiguous questions?
• Does it avoid answering restricted questions?
• Does it route clinical, prescribing, prior-authorization, or coverage decisions appropriately?

Security

• Can one role retrieve content assigned only to another role?
• Are logs and feedback stores appropriately restricted?
• Can retrieved document text manipulate the system into ignoring its controls?
• Are patient-specific transactions kept outside the shared index?

Operational Performance

• Is latency within the agreed service objective?
• Are token usage and search load controlled?
• Are retrieval errors, model errors, and citation-validation failures observable?
• Is user feedback captured without exposing sensitive content?

Regression Testing

• Do model changes break previously accepted answers?
• Do prompt changes alter abstention behavior?
• Do index changes reduce retrieval quality?
• Do document updates introduce stale or conflicting answers?
• Do permission changes take effect consistently?

Any public claim about time savings, answer accuracy, adoption, or reduced manual effort should be supported by production dashboards, controlled evaluations, or other objective records.

I have intentionally omitted numerical performance results from this public article.

What I Learned

1. Metadata Was as Important as the Model

Ownership, effective dates, page anchors, document versions, and access groups determined whether an answer was usable and safe.

A highly capable model cannot compensate for ungoverned source material.

2. Hybrid Retrieval Was More Dependable Than Vector Search Alone

Specialty-pharmacy questions frequently contain exact drug names, forms, identifiers, abbreviations, and program terminology.

Keyword retrieval helped with exact matches. Vector retrieval helped with paraphrases. Semantic reranking improved the final evidence selection.

3. Source Quality Set the Ceiling

RAG cannot repair contradictory, outdated, or ownerless documentation.

It may help identify those problems, but it cannot transform an unapproved source into authoritative policy.

4. Citations Had to Be Operational

A decorative list of document names was not enough.

Users needed links that opened the correct document version and, wherever possible, the relevant page or section.

5. Abstention Was a Product Feature

In healthcare, the response:

“I could not find an approved source that supports an answer.”

can be safer and more useful than a polished guess.

The assistant should be evaluated not only on how often it answers, but also on whether it stops appropriately.

6. Knowledge Retrieval and Patient Transactions Required Different Controls

Keeping general knowledge retrieval separate from patient-specific transactions reduced privacy risk and made authorization easier to reason about.

The knowledge assistant could explain approved procedures without becoming an uncontrolled repository of patient information.

Final Thoughts

The purpose of this assistant was not to replace specialty-pharmacy expertise.

It was to reduce the friction between a question and the current approved knowledge needed to address that question consistently.

The language model was only one component. The more important engineering work involved:

• Content ownership
• Identity and access management
• Permission filtering
• Retrieval quality
• Document lifecycle governance
• Citation validation
• Auditability
• Evaluation
• Abstention
• Human escalation

Technology and product names reflect a late-2024 reference baseline. Service names, capabilities, and model availability may change over time.