Company Knowledge Assistant

What Are We Building?

Build an internal AI assistant that answers employee questions using company documents, policies, product information, and knowledge bases. The assistant should retrieve relevant company knowledge before generating an answer.

Understand the Architecture

A practical architecture separates ingestion from question answering: documents are collected and indexed once, while employee questions trigger retrieval and generation.

Connect Company Knowledge Sources

Typical sources include PDFs, DOCX files, web pages, help-center articles, Notion pages, wikis, tickets, and structured databases. Each source should have an owner and update strategy.

Ingest Documents

The ingestion pipeline loads source content, extracts text, preserves metadata, cleans noise, chunks the content, creates embeddings, and stores searchable records.

Preserve Metadata

Keep document ID, title, department, source URL, access group, section, timestamps, and version information. Metadata becomes important for filtering, citations, and permissions.

Chunk Company Knowledge

Good chunks preserve a coherent policy, procedure, product explanation, or section. Avoid chunks that mix unrelated departments or topics.

Create Embeddings

Embeddings turn chunks into vectors so a question such as “How many days can I work remotely?” can retrieve semantically related policy text even when the wording differs.

from dataclasses import dataclass

@dataclass
class CompanyChunk:
    text: str
    document_id: str
    department: str
    access_group: str

chunk = CompanyChunk(
    text="Employees receive 20 paid vacation days.",
    document_id="hr-policy-2026",
    department="hr",
    access_group="employees",
)

print(chunk.document_id, chunk.department)

Store the Knowledge Index

Use a vector database or search index to store embeddings, text, and metadata. Keep the original source reference so answers can show where information came from.

Retrieve Relevant Knowledge

At query time, embed the employee question, retrieve candidate chunks, apply metadata and permission filters, and optionally rerank the candidates before generation.

Apply Access Control

Retrieval must respect the employee’s permissions. A user should never receive retrieved text from a private HR, finance, legal, or executive document they are not authorized to access.

Build a Grounded Prompt

Give the model only the selected company context and clear instructions to answer from that context. If the evidence is insufficient, the assistant should say that it cannot verify the answer.

def build_grounded_prompt(question: str, contexts: list[str]) -> str:
    evidence = "\n\n".join(contexts)
    return f"""Answer using only the company evidence below.
If the evidence is insufficient, say you cannot verify the answer.

Evidence:
{evidence}

Question: {question}"""

prompt = build_grounded_prompt(
    "How many vacation days do employees receive?",
    ["HR policy: Employees receive 20 paid vacation days."],
)
print(prompt)

Return Sources and Citations

Show document titles, sections, page numbers, or links where possible. Citations let employees verify important answers and help administrators debug retrieval.

Handle Unknown Questions

Do not force an answer. If retrieval confidence is low or the context does not support the question, route to a human, ask for clarification, or state that the knowledge base does not contain enough information.

Keep Knowledge Fresh

Company information changes. Track document versions, update timestamps, ingestion status, and deletion events. Re-index changed content instead of blindly creating duplicates.

• Track source version
• Record updated timestamp
• Remove deleted content
• Re-index changed documents

Combine RAG with Tools

Use RAG for knowledge and tools for live actions. For example, retrieve the leave policy with RAG, then call an HR system to check an employee’s remaining leave balance.

Monitor the Assistant

Track retrieval quality, answer quality, latency, token usage, citations, failed searches, permission denials, and user feedback.

def can_access(user_groups: set[str], required_group: str) -> bool:
    return required_group in user_groups

user_groups = {"employees", "engineering"}
print(can_access(user_groups, "employees"))
print(can_access(user_groups, "executives"))

Evaluate with Real Questions

Create a representative evaluation set from real employee questions. Measure retrieval relevance, groundedness, citation correctness, refusal behavior, and answer usefulness.

Secure the Knowledge Base

Protect source documents, embeddings, credentials, logs, and prompts. Treat retrieved company data as sensitive application data and apply least-privilege access.

Build the Company Knowledge Assistant

A useful first version can support one department, a small document set, citations, access filters, feedback buttons, and an escalation path. Expand after measuring quality.

Common Mistakes

Indexing everything without ownership, ignoring permissions, returning uncited answers, using stale documents, retrieving too much context, and treating the LLM as the source of truth are common production mistakes.

Mini Project: Internal Support Assistant

Create an assistant for HR or IT. Index approved documents, attach metadata, filter by employee access, retrieve the top candidates, generate a grounded answer, and return citations plus a “contact support” fallback.

30-Second Recap

Company Knowledge Assistant = trusted sources + ingestion + chunking + embeddings + retrieval + permission filtering + grounded generation + citations + monitoring.

Key Takeaways

Treat company knowledge as a governed data system. Retrieval should be permission-aware, answers should be grounded and cited, and freshness and evaluation must be part of the design.