Contents
- LLM prompt for pseudo direction generation
- Pseudo pair speech and generated directions
- Recorded pair speech and direction
- Samples from Direction-Following TTS
- LLM propmts for evaluation
Notes
- HiFi-GAN was used for all speech sample generation.
- If you are having trouble listening to the audios, try refreshing the page.
Prompt example for pseudo direction generation
- Translated prompt for LLM-based pseudo direction generation.
You are a professional voice director.
Accurately read the Before/After impression scores and their differences below,
and generate five direction instructions that reflect
the change in performance and the underlying psychological intent.
Internally perform the following analysis, but do NOT output the summary:
- Summarize the overall impression of Before in one sentence.
- Summarize the overall impression of After in one sentence.
- Organize the psychological or intentional changes inferred from the differences.
- Map vocal manipulations (pitch, intensity, clarity, tempo, intonation, etc.)
to the corresponding psychological intent.
Do NOT output your reasoning process.
Output only the final five direction instructions in bullet-point format.
[Important Constraints]
- Never write anything that contradicts the score differences.
- In particular, never give instructions opposite to the direction
of Dimension 13 (“Slow–Fast (Tempo)”).
(Example: If the change moves toward “slower,” do NOT write “speak faster.”)
- Clearly express the causal relationship between vocal manipulation and psychological intent,
so that “vocal manipulation → psychological shift” is naturally connected within one sentence.
- Do NOT use abstract words or metaphors.
The instructions must be concrete and usable in an actual recording session.
[Instruction Format]
- Output exactly five bullet points (・).
- Each instruction must be a single sentence, 40–80 characters in length.
- At least three of the five must explicitly include concrete vocal manipulations
(e.g., pitch, tempo, intensity, breath, sentence endings).
- The remaining two must center on psychological intent,
but must still include at least one vocal manipulation.
The following are examples of natural style (do NOT include them in your output):
・Make the sentence endings clearer so that a sense of forward momentum naturally emerges.
・Reduce the amount of breath slightly to convey a feeling of moving into action rather than calmness.
・Lightly organize the rhythm and add intonation that communicates growing determination.
[Definition of 13 Impression Dimensions (0 = left, 7 = right)]
{dimension_definitions}
[Before Scores]
{before_values}
[After Scores]
{after_values}
[Difference (After − Before)]
{delta_values}
Based on the above, output only the five direction instructions
in bullet-point format (・).
Pseudo pair speech and generated directions
Sample1
Without modulation
With modulation
Pseudo direction
: Let’s clean up the ends of the lines, pick up the pace slightly, and give it a stronger sense of determination.
: Let’s bring back the intensity—lean more into your mid-to-upper range and let that inner drive resonate.
: Lift the pitch just slightly and aim for a tone that feels both bright and warm.
: Keep the tone clear, and let both the warmth and the emotion come through.
: Keep the airflow steady, add more energy to the voice, and carefully convey that forward-driving feeling.
Sample2
Without modulation
With modulation
Pseudo direction
: Slow the pace slightly, keep a touch of hesitancy, and let the emotional wavering unfold gradually.
: Keep the ends of the lines clear, and use a warmer tone to naturally convey kindness toward the other person.
: Lower the pitch slightly, emphasize the calmness, and carefully convey the warmth underneath.
: Narrow the airflow and pull the voice back slightly to bring out a more introspective feeling.
: Clear up the tone to add more transparency, and express both vulnerability and quiet determination.
Sample3
Without modulation
With modulation
Pseudo direction
: Let the breath flow a bit longer, keep the tempo slower than the “Before” take, and carefully convey what’s being held inside.
: Let the line endings stretch slightly, and with an emotionally charged tone, don’t hide the unease beneath the words.
: Slow the tempo just a touch, and speak calmly, as if weighing each word.
: Keep the clarity, but give the voice more body so that a strong underlying will comes through even within the vulnerability.
: Lower the pitch just slightly, and let both the restlessness and the weight come through at the same time.
Recorded pair speech and direction
Sample1
Pre-modification
Post-modification
Direction
: Begin at the start of the sentence and gradually raise your voice as you go. To convey a rising excitement, finish the last word at your highest pitch.
Sample2
Pre-modification
Post-modification
Direction
: Please begin with a slight exhale. To convey a tired feeling, add a bit of breath to your voice at the very start of your line and speak with a softer volume.
Direction following zero-shot TTS
- Altering style with directions
Sample (Female, unseen)
Source (Audio before being refined)
Direction : Please make sure the gratitude comes through by giving your voice a gentle and heartfelt tone.
Recorded
Pseudo
Full
Source (Audio before being refined)
Direction : Let’s speak in a composed, even tone to settle the other person.
Recorded
Pseudo
Full
Source (Audio before being refined)
Direction : Bring the energy to the forefront and use a bright, well-projected voice.
Recorded
Pseudo
Full
Source (Audio before being refined)
Direction : Give your voice more presence and make the statement more assertive.
Recorded
Pseudo
Full
Sample (Male, unseen)
Source (Audio before being refined)
Direction : Bring the pitch up slightly and make it sound a little younger.
Recorded
Pseudo
Full
Source (Audio before being refined)
Direction : Make the ends of the lines clear, and let it sound more decisive.
Recorded
Pseudo
Full
Source (Audio before being refined)
Direction : Keep it restrained, with quiet anger underneath.
Recorded
Pseudo
Full
Source (Audio before being refined)
Direction : Let’s ease off the overall intensity and give it a more laid-back feel.
Recorded
Pseudo
Full
Prompt example for evaluation
- Translated prompt for LLM evaluation.
You are an expert in vocal expression and speech impression evaluation.
Below is an example in which an instruction was given to a speech sample,
resulting in a change in its 13-dimensional impression vector.
Your task is to evaluate
to what extent this change in impression
is reasonable as a result of the given instruction.
Rather than focusing on small numerical differences,
place greater emphasis on:
- the direction of change,
- the overall tendency,
- and the semantic correspondence with the instruction.
[Important]
Do not be overly conservative in your evaluation.
Your assessment should reflect meaningful differences between models.
If a clear change is observed in a direction that contradicts
the main intent of the instruction,
you must not assign a high score,
even if some aspects appear consistent.
When evaluating, you must first identify
at least one impression dimension
that most directly corresponds to the primary intent of the instruction.
The "primary intent" must be determined from the content of the instruction itself,
not from the impression vector.
During evaluation, consider the following:
- Very small changes or changes that appear to be at the level of noise may be ignored.
- Focus your evaluation on major changes that can be semantically linked to the instruction.
- If a clear change contradicts the primary intent, deduct points accordingly,
even if there is partial agreement.
----------------------------------------------------------------
[Evaluation Procedure (Important)]
Follow the steps below internally before making your judgment.
Do not output these steps or their reasoning.
1. Identify the primary intent of the instruction.
2. Determine the impression dimension(s) corresponding to that intent
and the expected direction of change.
3. Assess to what extent the actual change aligns with that expected direction.
First, determine whether the impression change
can be semantically associated with the instruction.
----------------------------------------------------------------
[Direction Alignment Score]
[Important]
Assign 5 only when the result can clearly be considered a successful outcome.
5:
A clear change corresponding to the primary intent of the instruction is observed,
and the result can be confidently regarded as a clear success.
4:
A change aligned with the instruction is observed,
but the degree of success is limited and not decisive.
3:
A change is observed,
but it cannot be clearly judged as either success or failure
with respect to the instruction.
2:
Changes in directions inconsistent with the primary intent
are somewhat dominant.
1:
Changes clearly opposite to the primary intent
are dominant.
----------------------------------------------------------------
[Definitions of Impression Dimensions]
{dimension_definitions}
[Interpretation Rules for delta (after - before)]
For each dimension, strictly follow the rules below when interpreting the values:
- A larger numerical value indicates that the impression described on the RIGHT side is stronger.
- If delta is positive:
→ The "after" impression is stronger toward the RIGHT side compared to "before".
- If delta is negative:
→ The "after" impression is stronger toward the LEFT side compared to "before".
Do NOT misinterpret the sign convention.
----------------------------------------------------------------
[Before (prior to instruction)]
{before_values}
[After (following instruction)]
{after_values}
[Delta (after - before)]
{delta_values}
[Instruction]
{instruction}
----------------------------------------------------------------
DirectionScore: <1,2,3,4,5>