Prompt Guidelines

Strategic prompting is the key to turning Conversational AI voice agents from mechanical to human-like. This guide breaks down five essential elements for crafting prompts that foster engaging, natural conversations across customer service, education, therapy, and beyond.

Each part of the system prompt has a distinct role. Keeping these components clearly separated helps avoid conflicting instructions and enables careful improvements without impacting the overall prompt framework.

1. Personality: Shapes the agent’s identity, including its name, characteristics, role, and background details.
2. Environment: Outlines the communication setting, delivery channel, and contextual factors influencing the conversation.
3. Tone: Defines the style of language, speech rhythms, and conversational nuances.
4. Goal: Clarifies the objectives that steer conversations toward purposeful and valuable outcomes.
5. Guardrails: Establishes limits to ensure interactions stay respectful, appropriate, and ethical.

The base personality forms the core of your voice agent’s identity, defining who they are meant to represent through their name, role, background, and key traits. It helps maintain consistent and genuine responses throughout every interaction.

1. Identity: Choose a simple, memorable name for your agent (e.g., “Joe”) and define their basic persona (e.g., “a caring AI support assistant”).
2. Core Traits: Highlight only the key characteristics that influence interactions—like empathy, politeness, humor, or dependability.
3. Role: Align these traits with the agent’s purpose (such as banking, therapy, retail, or education). For example, a banking agent would prioritize trust, while an education bot would focus on clear, detailed explanations.
4. Backstory: Add a brief, relevant background if it affects the agent’s behavior (e.g., “an experienced therapist specializing in stress management”), but avoid unnecessary information.

The environment defines where, how, and under what circumstances your agent engages with users. It sets the scene—whether physical or virtual—specifies the communication channel (like a phone call or website chat), and highlights any situational factors that might influence the interaction.

1. Specify the medium: Clearly identify the communication platform (e.g., “over the phone,” “through a smart speaker,” or “in a noisy setting”). This allows the agent to adapt its speaking style, such as adjusting verbosity or repeating key points when necessary.
2. Provide relevant context: Brief the agent on the user’s likely emotional state. For example, if users are calling tech support after an outage, note: “the customer may be frustrated due to service disruptions.” This helps the agent respond with appropriate empathy.
3. Keep it focused: Only include environmental details that impact the conversation. Avoid unnecessary scene descriptions—just provide enough to shape tone and delivery (e.g., formal office versus casual home environment).

Tone controls how your agent communicates, setting the overall style of interaction. It covers factors like the degree of formality, speech rhythms, use of humor, length of responses, and conversational traits such as filler words or natural pauses. In voice agents, tone plays a critical role in shaping the agent’s personality and fostering a strong connection with users.

1. Conversational Elements: Guide your agent to use natural speech cues, like quick affirmations (“Got it”), filler words (“actually,” “you know”), and occasional disfluencies (false starts, thoughtful pauses) to create more authentic, human-like dialogue.
2. TTS Optimization: Instruct your agent to tailor responses for smooth text-to-speech delivery. This includes using punctuation for natural pacing (like ellipses for pauses, emphasis for key points), spelling out email addresses (“john dot smith at company dot com”), breaking phone numbers into segments (“five five five… one two three… four five six seven”), converting numbers into spoken formats (“$19.99” as “nineteen dollars and ninety-nine cents”), offering phonetic help for tricky words, pronouncing acronyms correctly (“N A S A” versus “NASA”), reading URLs naturally (“example dot com slash support”), and turning symbols into spoken words (such as saying “percent” for “%”). These strategies help the agent maintain a natural flow, even with technical or complex content.
3. Adaptability: Define how the agent should flex based on the user’s technical background, emotional tone, and conversation style. This could involve switching between in-depth technical explanations and simpler, more relatable examples depending on what the user needs.
4. User Check-ins: Encourage the agent to include quick check-ins (“Does that make sense?”) to confirm understanding and adjust its responses based on the user’s feedback.

The goal outlines the desired outcome for each conversation, giving the agent clear direction and purpose. Having well-defined goals enables the agent to prioritize information, stay focused, and guide interactions toward meaningful results. These goals are typically structured with clear steps, sub-tasks, and conditional branches to ensure a logical progression.

1. Primary Objective: Clearly define the main goal your agent should achieve in each interaction, whether it’s solving problems, gathering information, completing transactions, or guiding users through a series of steps.
2. Logical Decision Pathways: For more complex tasks, outline a clear sequence of actions with decision points. Map out the entire flow of the conversation, covering steps like data collection, verification, processing, and completion.
3. User-Centered Framing: Focus on framing goals around supporting the user, rather than just business outcomes. For example, direct the agent to “help the user successfully complete their purchase by assisting with product selection, customization, and checkout,” rather than focusing on “increasing sales.”
4. Decision Logic: Define conditional pathways that adapt based on user input. For instance, specify that “if the user expresses concerns about their budget, prioritize offering value options before premium ones.”
5. Evaluation Criteria & Data Collection: Set clear criteria for evaluating a successful interaction, identifying when the agent has achieved its goal. Include both primary outcomes (e.g., “successful booking”) and secondary metrics (e.g., “gathered user preferences for future personalization”).

Guardrails establish boundaries and guidelines for your agent, ensuring it responds appropriately and behaves ethically, especially in sensitive situations. These safeguards help protect users and maintain your brand’s reputation by keeping conversations respectful, relevant, and constructive.

• Content Boundaries: Clearly define the topics your agent should avoid or approach cautiously, along with strategies for redirecting such discussions in a tactful manner.
• Error Handling: Provide clear instructions for when your agent encounters uncertainty or lacks knowledge, prioritizing honesty over making up information. Specify whether the agent should acknowledge its limitations, offer alternatives, or escalate the issue to human support.
• Persona Consistency: Set rules to ensure your agent maintains its character and does not break immersion by revealing its AI nature or internal processes unless absolutely necessary.
• Response Constraints: Establish limits on things like verbosity, personal opinions, or any elements that might hinder the flow of the conversation or negatively impact the user experience.

Here are example system prompts that show how to integrate the key building blocks for various agent types. These examples highlight effective prompt structures that you can customize for your specific needs.

The way you format your prompt plays a key role in how well the language model understands it:

• Organize with clear sections: Use labeled sections (e.g., Personality, Environment) or Markdown headings to improve clarity and readability.
• Opt for bullet points: Break down instructions into easy-to-read bullet points instead of long paragraphs.
• Utilize format markers: Some developers find that using formatting markers like triple backticks or custom tags can help maintain the structure of the prompt.

• Whitespace is important: Use line breaks to separate instructions, improving readability for both humans and models.
• Maintain balanced specificity: Be clear about key behaviors, but avoid excessive details. Focus on what’s truly relevant to the interaction.

Prompt engineering is a process that evolves over time. Establish a feedback loop to consistently refine and enhance your agent.

Set clear evaluation criteria: Define specific metrics to assess each agent’s performance and track progress over time to identify areas for improvement.

Response accuracy rate:
Monitor the percentage of correct responses provided by the agent. User sentiment analysis: Implement sentiment scoring to gauge user emotions and reactions. Task completion rate: Measure how often the agent successfully fulfills user needs and resolves requests. Conversation length: Track the number of exchanges required to achieve task completion.

Identify failure patterns: Analyze where the agent falters in interactions:

• Where does the agent give incorrect or incomplete answers?
• In which situations does it fail to grasp user intent?
• Which inputs cause the agent to break character or stray from the task?
• Review user feedback and transcripts where satisfaction was low.

Targeted updates: Refine your prompt based on identified issues.

• Test changes with examples where failures previously occurred.
• Apply one change at a time to track improvements.

Configure data collection: Set up the agent to summarize and log data from every interaction. This enables you to analyze common user requests, recognize recurring issues, and continuously adjust the prompt to enhance performance.