When ChatGPT Helps Write a Service Note, Who Is Accountable?

During routine remote support sessions, I increasingly see ChatGPT or another AI tool open beside email, Microsoft Word, and the systems employees use to complete their daily work.

That does not mean someone is doing something wrong. Employees are under pressure. They want to write more clearly, finish documentation faster, and keep up with growing workloads. AI gives them quick help.

But it also tells me that many organizations are not adopting AI through a formal rollout. AI is entering one task, one browser tab, and one employee at a time.

In a human-services organization, that matters. A paragraph may become a service note, progress note, incident report, program summary, intake record, or another official document.

The key question is no longer just:

"Are employees using AI?"

The more important question is:

"What happens when AI-generated language moves from a private browser window into an official record?"

A service note is not just another piece of writing

AI can correct grammar, organize rough notes, or help someone get past a blank page. Those uses may appear harmless.

Service documentation, however, can carry much more weight than an ordinary email. Depending on the program and funding source, it may help show:

• What service was delivered
• When and where it happened
• Who provided it
• What the employee observed
• How the person receiving services responded
• Whether the service matched an approved plan
• Why the organization had the right to bill for it

New York Medicaid providers must prepare and maintain contemporaneous records. In plain language, records should be created at or near the time of service and must support the provider's right to receive payment. Information connected to a claim must also be true, accurate, and complete.

OPWDD also publishes service-specific documentation requirements that support provider claims for reimbursement. The exact rules differ by service, but the larger point is the same: the record helps show what was delivered and why the claim is valid.

Once information enters an official record, someone is responsible for its accuracy.

AI does not accept that responsibility.

The employee, supervisor, and organization do.

The risk is not always an obvious false statement

Most people know that AI can make mistakes. The harder problem is that it can produce language that sounds professional, complete, and believable.

Consider a fictional example. James is not a real person, and this note is illustrative only.

"James looked upset and did not join the group."

The employee asks an AI tool to make the note sound more professional.

The tool responds:

"James displayed signs of emotional distress and declined to participate in the scheduled group activity despite staff encouragement."

The second version sounds better.

It may also contain details the employee never provided.

Did James show emotional distress, or did he simply look upset?

Was the activity scheduled?

Did he decline, or did he remain quiet?

Did staff encourage him?

The AI is not recalling what happened. It is generating language that commonly fits the situation.

"Plausible is not the same as observed."

Once that paragraph is copied into an official system, a future reader may assume every detail came from the employee's direct knowledge.

The note looks more polished, but it may be less accurate.

The Prompt-to-Record Chain

Human-services leaders need to look at the full path between what happened and what was finally recorded.

I call this the Prompt-to-Record Chain:

Observation → Prompt → AI Draft → Human Verification → Official Record → Attestation → Downstream Use

Each stage creates a different question.

1. Observation

What did the employee personally see, hear, or do?

This is the factual starting point. If the original observation is vague, incomplete, or recorded much later, AI cannot rebuild what actually happened.

2. Prompt

What information did the employee enter into the AI tool?

A prompt may contain names, diagnoses, behaviors, medications, addresses, family details, employee information, or internal procedures.

Removing a name may not be enough. A combination of details can still point to a specific person.

3. AI draft

Did the tool improve the wording, or did it add meaning?

There is a major difference between:

"Correct the grammar without adding or changing any facts."

and:

"Turn this into a complete professional service note."

The second prompt invites the tool to fill gaps.

4. Human verification

Did the employee compare every statement with what actually occurred?

Thinking that a paragraph "looks right" is not the same as checking it line by line.

The employee should be able to answer:

• Did AI add a fact?
• Did it change the meaning?
• Did it turn an observation into a conclusion?
• Did it overstate what occurred?
• Does the final wording still reflect my own knowledge?

5. Official record

What part of the AI draft entered the organization's official system?

Once the language is copied into a case-management, documentation, or billing platform, it may become part of the permanent record.

The organization may no longer be able to tell which words came from the employee and which came from AI.

6. Attestation

Who signed, approved, or accepted responsibility for the record?

A signature is more than a click.

AHRC NYC's published AI-use policy requires employees to review AI-assisted documentation for accuracy and compare it with source material. It also says a supervisor's signature serves as a formal attestation that the content was reviewed and validated.

"Can a supervisor meaningfully approve a record without knowing how AI contributed to it?"

7. Downstream use

Where might the information go next?

A service note may later affect:

• Billing
• Service planning
• Quality reviews
• Incident investigations
• Audits
• Legal matters
• Communication with a family
• Reports to government agencies or funders

A small wording change near the start of the chain can travel much further than the employee expected.

An approved tool does not mean every use is approved

Organizations often ask whether a specific AI product is secure or approved.

That is important, but it is only one decision.

Leadership must separate five questions:

1. Is the application approved?
2. Is the employee using an approved organizational account?
3. Is the information allowed in that environment?
4. Is the specific task an approved use of AI?
5. Is there a clear way to verify the output?

An organization may allow AI for drafting a public event announcement but prohibit it for service notes.

It may allow brainstorming but block the entry of participant, employee, financial, or other sensitive information.

AHRC NYC offers a useful real-world example. Its policy names Microsoft Copilot as its authorized generative AI platform, yet it still places limits around service documentation, privacy, approved use, and human review. It prohibits other generative AI platforms for service documentation and organizational data.

The U.S. Department of Health and Human Services took a similar approach in its August 2025 ChatGPT Privacy Impact Assessment. HHS lists productive uses such as drafting and summarizing, but its rules prohibit users from entering protected health information, personally identifiable information, sensitive financial information, internal pre-decision content, sensitive operational information, and nonpublic workforce information.

"Buying or approving an AI tool does not automatically approve every account, every type of data, or every workflow."

A written policy helps, but staff also need examples that match their real jobs.

A rule in a handbook will not guide someone who is behind on documentation near the end of a busy shift.

Three steps leaders can take now

Organizations do not need to ban AI. They also do not need a perfect 50-page strategy before acting.

They need to understand what is already happening and place practical controls around it.

1. Ask how AI is being used

Start with discovery, not punishment.

Ask employees which tools they use, what tasks they use them for, and where AI saves time.

People are more likely to be honest when the goal is to understand the workflow rather than catch someone breaking a rule.

2. Separate low-risk work from official records

Not every use carries the same risk.

Brainstorming a staff-event theme with no confidential information is very different from rewriting a service note, incident report, personnel record, or billing-related document.

Give employees clear examples of:

• What is allowed
• What needs approval
• What is not allowed

3. Set a real verification standard

"Review the output" is too vague.

Employees and supervisors should know what must be checked before AI-assisted language enters an official record.

That includes names, dates, services, direct observations, interpretations, and any wording AI may have added.

The New York Alliance for Inclusion & Innovation is already helping I/DD providers examine these issues through an AI Learning Community focused on policies, staff training, risk, oversight, vendor review, and regulatory alignment.

Trace one record before buying another AI tool

Before purchasing more licenses or announcing a large AI initiative, choose one real documentation workflow and trace it from beginning to end.

Ask:

• What happened?
• Who observed it?
• Where were the original facts recorded?
• Was AI used at any point?
• What information was entered?
• Did AI add or change anything?
• Who checked the final language?
• Who signed it?
• Where might that record be used later?

This exercise will reveal more than a general discussion about whether AI is good or bad.

AI can help human-services organizations reduce some administrative work, improve access to information, and help employees communicate more clearly.

But the goal should not be the most polished note in the shortest amount of time.

The goal is a record that is accurate, timely, secure, and tied to what actually happened.

AI can assist with that process.

It cannot own it.

Can you still explain how every word got there?

AI is already inside many organizations. The question is whether their records can still show what a person observed and what a tool added.

Through Bitralynx Solutions, I help nonprofit and human-services leaders trace this chain, from browser tab to signed record, and put practical controls around it.

If AI is already showing up in your service documentation or other important workflows, let's talk.