How to Train AI on Transcripts and Sales Scripts
Training AI on your transcripts rarely means rebuilding a model. It means giving an existing model four things: clear instructions, a knowledge base, example conversations, and test scenarios. Programming Insider’s guide to training an AI phone agent describes this setup, and it is the practical path for an agent, contractor, adviser or broker who wants instant lead replies that sound like them. The work is curation: picking the right conversations, cleaning them, turning them into rules and approved answers, and testing before a real lead sees anything.
This guide walks through that process step by step, with the privacy and consent rules a US business should check first.
AstreauxTurn Your Business Voice Into Faster RepliesAstreaux learns your business’s unique voice and uses conversational AI to engage new leads with personalized responses.Book a call
TL;DR
“Training” for most service businesses means configuring instructions, a knowledge base, example conversations and test scenarios. It does not mean retraining the underlying model.
Choose varied transcripts (routine questions, quote requests, complaints, calls that went wrong), remove personal details, and rewrite them as clean examples and short behavioral rules.
Grounding only works when the content is current and approved. Wrong or outdated documents still produce wrong answers.
Test quality separately from speed, and keep a named human owner for handoffs.
In the US, AI-generated voices fall under the TCPA, and the FTC says there is no AI exemption from existing law. Check consent and data handling before you upload anything.
What it means to train AI on transcripts and scripts
People use “train” loosely. A large language model has already been trained on a vast body of text. What you do is adapt its behavior to your business, which is a different and much cheaper job.
In practice that adaptation has four layers:
Instructions. Who the assistant is, what it may and may not say, how it greets a lead, and when it hands off to you.
A knowledge base. Approved facts: service areas, pricing rules, availability, FAQs, policies.
Example conversations. Real exchanges, cleaned up, that show your tone and your way of moving a lead toward an appointment.
Test scenarios. A fixed set of tricky inputs you run every time you change something.
The research on phone agents says these layers can support accurate answers to routine questions and help the agent recognize when a call should go to a person. They are also the things you keep improving, rather than retraining the model itself.
The mechanics behind it are simple. Twilio’s documentation on generative custom operators explains that a prompt is sent together with a transcript to a large language model, and the resulting analysis goes back to downstream systems. Your transcripts either become material the model reads at run time, or the source for rules you write yourself. Either way, you control the quality of what goes in.
Transcripts versus sales scripts
They do different jobs, and you want both.
Transcripts show what leads actually ask and how they phrase it. A buyer says “is the house still available?” and not “please confirm listing status.” A homeowner says “how much to fix a leak under the sink?” and not “request for plumbing estimate.”
Sales scripts show what you want to happen: the qualifying questions, the order you ask them in, the objection responses you trust, and the booking call to action.
A script alone produces stiff replies. A transcript alone teaches the assistant your bad habits along with the good ones. Use scripts for structure and transcripts for language.
Which transcripts and scripts to use
Do not start with your best five calls. Start with variety. Programming Insider recommends collecting calls that cover routine questions, transfers, repeat callers, complaints, and calls that went wrong. The failures matter because they show the assistant where confusion starts.
Papo’s help-center guide on training with real conversation examples suggests a similar spread: product questions, complaints, quote requests and appointments. Storylane’s training documentation lists websites, documents, FAQs, demos, best call recordings and sales transcripts as usable onboarding material, and notes that FAQs are the right place for wording that must be exact or approved.
Here is how that translates by profession.
Professional | Transcripts worth collecting | Scripts worth including |
|---|---|---|
Real estate agent | First calls from portal inquiries, showing requests, “just browsing” conversations, price-reduction questions | Buyer qualifying questions, showing-booking script, open house follow-up |
Contractor | Estimate requests, emergency calls, questions about timelines and permits, calls where the lead asked for a ballpark price | Site-visit scheduling script, scope-clarifying questions, no-show reminder |
Insurance agent | Quote requests, renewal questions, “why did my rate change” calls | Quote follow-up sequence, information-gathering questions, referral ask |
Financial adviser or mortgage broker | Discovery calls, document-request exchanges, calls from leads who went quiet | Discovery agenda, nurture messages, approved compliance language |
If you also run campaigns, our guide to insurance text templates with consent logging shows how approved message wording can double as source material.
How many is enough?
There is no magic count, and no source we trust gives one. Aim for coverage rather than volume: every common lead type, every common objection, and at least a few examples of a conversation ending in a booked appointment and one ending in a polite refusal. Twenty well-chosen, cleaned examples will usually teach more than two hundred raw ones.
Preparing transcripts before the AI sees them
Raw transcripts are noisy. They contain filler words, crosstalk, mistranscribed names and, often, details you should never feed into a system. Programming Insider advises against loading raw transcripts unchanged. Its steps are to transcribe the recordings, mark confusion points, clarifying questions and resolutions, and then rewrite the patterns as concise behavioral rules.
A workable routine:
Transcribe and proofread. Automated transcription makes errors with names, street addresses and trade terms. Papo’s guide specifically calls for reviewing transcription errors before upload.
Remove personal information. Names, phone numbers, addresses, account numbers, health or financial details. Replace them with placeholders such as {{firstName}} or {{propertyAddress}}.
Mark the turning points. Highlight where the lead was confused, where you asked the question that moved things forward, and where the appointment was set.
Rewrite as clean examples. Tidy each exchange into a short, ideal version in your voice.
Extract rules. Convert patterns into one-line instructions, such as “Ask for the property address before quoting a time window.”
Save organized text documents. One topic per document, titled clearly, so the right section is easy to retrieve.
If you want a model for turning messy conversations into something reusable, a customer success call prep routine shows the same discipline from the other direction: decide what you need from a conversation before you try to capture it.
Turn recurring questions into FAQs
Papo notes that questions that recur across transcripts can become FAQs, while the vocabulary and expressions your customers use can inform the agent’s prompt. This is the highest-return step for most small businesses. If twelve of twenty transcripts include “do you charge for estimates?”, that answer should exist once, in approved wording, and not be improvised.
Keep your source material trustworthy
Salesforce’s Trailhead module on unstructured data for reliable AI responses lists what reliability depends on: a trusted source, the current approved version, good indexing and retrieval, participant and timestamp metadata, and traceable redaction or summarization. For a small team, that translates into habits:
Date every document and note who said what (you, the lead, a colleague).
Delete old price lists instead of leaving them beside new ones.
Keep a record of what you redacted and why.
Prompts, knowledge base or fine-tuning?
For a service professional, start with the first two and keep fine-tuning as a distant option. The phone-agent research says to use instructions, a knowledge base, example conversations and test scenarios, and usually improve those rather than retrain the model.
The reasons are practical:
You can change them in minutes. A new service area or a price change is an edit, not a project.
You can see what the assistant is working from. That makes errors traceable.
They suit small data. A solo agent does not have thousands of calls to learn from.
A knowledge base typically works through retrieval-augmented generation: the assistant looks up relevant passages and answers from them. Salesforce says this grounds answers in documents such as policies, emails and transcripts, but adds the caveat that wrong, outdated, poorly indexed or unapproved documents can still lead to unreliable answers. Grounding is not a guarantee. It is only as good as the library behind it.
Making replies sound like you, not like a script
The goal is a first reply that feels like a quick, attentive message from you. Three habits help.
Teach voice with a few strong samples. A short voice description and a handful of examples beat a long manual. We cover this in detail in our piece on brand voice AI with one page and three to five samples.
Write instructions about behavior, not just tone. “Be friendly” is vague. “Answer the question asked first, then ask one qualifying question, then offer two appointment times” is something you can test. Guidance on humanizing AI text with instructions makes a similar point: specific direction beats general requests for a natural tone.
Use placeholders and variation. An example like this gives the assistant a pattern without a fixed sentence:
Hi {{firstName}}, thanks for reaching out about {{service}}. I can usually get out to {{neighborhood}} this week. Would {{slot1}} or {{slot2}} work for a quick look?
If every reply starts with the same sentence, the AI is copying your example instead of learning your style. Vary your samples and watch for repeated phrasing in testing.
Stopping the AI from inventing answers
Language models can state false things confidently. NIST’s Generative AI Profile calls this confabulation: producing and confidently presenting erroneous or false content, including output that diverges from or contradicts the input. It follows from how these models predict likely words rather than verify facts. NIST’s point is that confident wording makes the problem more dangerous, which is why it pairs grounding with documented oversight, independent evaluation, red-teaming, feedback mechanisms and regularly updated risk measurement.
For a lead-reply assistant, that becomes a short list of guardrails:
Put approved facts in the knowledge base and tell the assistant to answer only from them.
Tell it to say it will check and follow up when it does not know, rather than guess.
List topics that always go to a human: exact quotes, coverage decisions, legal or tax advice, investment recommendations.
Never let it promise a price, a timeline or a coverage outcome that you have not approved in writing.
Test before real leads see it
A test set is a fixed list of inputs you rerun after every change. Build it from your own transcripts, including awkward ones.
Programming Insider recommends that voice tests include regional accents, non-native English, background noise, fast speakers, pauses, interruptions and weak audio. For text channels, the equivalent is typos, one-word replies, slang, and messages that bundle three questions together.
Judge quality separately from speed
Fast is not the same as good. Takeover Search’s guide to evaluating lead response says a reply should answer the actual question accurately and specifically, use plain language, ask a useful question where appropriate, and admit uncertainty rather than guess. Its view is that a good lead experience combines accurate answers, ownership, a concrete next step and follow-through. A quick but irrelevant reply can still lose the lead.
A simple scorecard for each test reply:
Check | Pass looks like |
|---|---|
Answers the question asked | The first sentence addresses what the lead wrote |
Accurate | Every fact matches your knowledge base |
Plain language | No jargon the lead did not use |
Useful next step | One clear question or two appointment options |
Honest about limits | Says it will check rather than guessing |
Right handoff | Escalates the topics on your human-only list |
Run it for each lead type you serve. A contractor might test an emergency leak, a vague remodel idea and a price-shopper. An insurance agent might test a renewal question, a competitor-quote comparison and a lead who says “stop texting me.”
For more on what happens after the first reply, see our guides on AI lead qualification and turning missed calls into booked jobs with fast text replies.
Privacy, consent and US rules to check first
Transcripts are customer data, and sometimes sensitive data. This is general information, not legal advice. For anything specific to your state or profession, talk to an attorney, and for regulated advice work such as insurance or financial services, to your compliance officer or licensing body.
AI voices and the TCPA. The FCC’s February 2024 declaratory ruling confirms that AI-generated human voices fall within the TCPA’s restrictions on artificial or prerecorded voices. Those calls generally require prior express consent, and telemarketing calls require prior express written consent under the cited rules. The FCC’s approach is technology-neutral: a voice that resembles a human does not escape the rules because it was generated. If your assistant places calls or sends marketing messages, confirm and log consent first. Our article on inbound lead automation with US compliance covers the practical side.
No AI exemption. The FTC states in its comment to the Copyright Office that there is no AI exemption from existing laws, and that deceptive or unfair practices involving AI remain prohibited. Do not claim your assistant is a person, and do not claim data protections you do not have.
Transcripts can hold a lot. In the FTC’s case against a medical transcript services provider, transcript files could contain driver’s-license numbers, tax information, medical histories, medications and psychiatric notes. Weak vendor safeguards left files publicly accessible. The lesson is that a privacy promise is not enough: you stay responsible for reasonable security and for overseeing the vendors and subcontractors that handle your files. Therapists and anyone handling health information should be especially careful and speak to a HIPAA-qualified adviser before uploading any session content.
Do not keep voice data forever. In its 2023 Amazon matter, the FTC emphasized that companies should not retain voice data indefinitely and should limit access to people with a business need. Set a retention period for recordings, delete originals once cleaned versions exist, and restrict who can open them.
Recording consent also varies by state, so check the rules where you and your leads are located before you record calls for training. A short notice at the start of a call is a common practice, but your attorney should confirm what your state requires.
Perspective: what we have learned about voice training
We built Astreaux around one observation: leads go cold in the gap between inquiry and reply, and a reply that sounds like a stranger is almost as bad as no reply. That is why we put so much weight on learning each business’s voice, so the first message a new lead receives sounds like the professional they contacted.
Our view is that most people over-engineer this. You do not need a data science project. You need a one-page description of how you talk, a few real samples, approved answers to your most common questions, and a clear line for when a human takes over. After that, the gains come from reviewing real replies each week and fixing what you see.
We also think speed and quality have to be earned together. Instant engagement matters, but only when the answer is right and the next step is clear. That is why we pair automated replies with appointment booking, nurturing and integrations with over 7,000 apps, so the conversation continues into your CRM and calendar and not just into an inbox. For revenue-focused teams, our guide to conversational AI for sales goes deeper on that.
— Jamaal
Put it to work with Astreaux
If you would rather not assemble prompts, knowledge bases and test sets by hand, Astreaux is built for this job. It learns the voice of your business, replies to new leads instantly, nurtures prospects, books appointments, and connects to your CRM and ad platforms. It serves real estate agents, contractors, insurance agents, financial advisers, mortgage brokers and therapists. Start with our walkthrough on setting up brand voice AI, then see how it fits your lead flow.
Astreaux is a subscription product that begins with a free trial or demo.
Measure results and refresh your material
A trained assistant is never finished. Set a review rhythm and track what matters.
EAGLE BS’s follow-up checklist for US businesses recommends tracking valid unique inquiries, measuring first human response time separately from automated acknowledgement, using cohort-based booking and sales conversion, tracking attendance, showing counts alongside rates, and counting unresolved inquiries. It also recommends a named owner, an escalation point and a monitored human handoff.
For scorecard design, Revelir’s QA scorecard guidance says a useful metric should predict a customer outcome or lead to an actionable coaching decision. It suggests a fixed quarterly review and retiring or recalibrating criteria that show no meaningful variance.
A practical cadence for a small business:
Weekly: read ten random AI replies and ten escalations. Fix wording and add missing FAQs.
Monthly: compare booking and attendance rates against the month before. Look at unresolved inquiries.
Quarterly: retire outdated documents, rerun your full test set, and drop any metric that never changes.
Whenever something changes: a new price, service area, license or policy means updating the knowledge base the same day.
One caution on interpreting results: a small business may see only a few dozen leads a month, so rates swing widely. Watch the counts beside the percentages, as the checklist advises, before you decide a change worked.
FAQ
Do I need to fine-tune a model to use my transcripts?
Usually not. The practical approach is to give an existing model instructions, a knowledge base, cleaned example conversations and test scenarios, then improve those over time. Fine-tuning is slower to change and harder to inspect, which is a poor fit for a business whose prices and availability shift.
Can I train AI on sales scripts alone?
You can, but replies will tend to sound stiff, because scripts show your intended structure and not how leads actually talk. Add a handful of cleaned transcripts so the assistant learns real phrasing, objections and vocabulary. Keep scripts for the order of questions and the booking step.
What should I remove from transcripts first?
Remove names, phone numbers, addresses, account and policy numbers, and any financial or health details, replacing them with placeholders. Keep a note of what you redacted. If your work involves medical or therapy information, get advice from a HIPAA-qualified professional before uploading anything.
How do I stop the AI from making things up?
Ground it in approved, current documents, tell it to say it will check when it does not know, and list the topics that always go to a person, such as exact quotes or coverage decisions. Then test with tricky questions after every change. Grounding reduces the risk but does not remove it, so keep reviewing real replies.
Do US rules apply if the AI only sends texts?
They can. The FCC’s 2024 ruling addresses AI-generated voices under the TCPA, and marketing messages carry their own consent requirements. The FTC also says existing laws apply to AI. Confirm consent practices and your state’s rules with an attorney before you automate outreach.
Sources
How to train Prompt and AI with real conversation examples (Papo)
Understanding Unstructured Data for Reliable AI Responses (Salesforce Trailhead)
NIST Artificial Intelligence Risk Management Framework: Generative AI Profile
How to evaluate lead response, not just response time (Takeover Search)
FTC: Provider of Medical Transcript Services Settles FTC Charges
Website and inquiry follow-up checklist for U.S. businesses (EAGLE BS)
