1	Providing the Best Caller Experience with User Interface Design Harry E. Blanchard, PhD AT&T Shannon Labs Middletown, NJ Speech Tek New York, NY September 14, 2004
2	Topics for Today User Interface Design and Speech: A Long History A Classification of Technology from the Dialog Designer’s View Principles for Creating Natural Language Dialogs How may I help you?^SM intent determination Principles for Request Fulfillment Directed dialog design Transactions
3	Speech Technology and UI Design What is usability? Effectiveness Efficiency Satisfaction Where are you familiar with it? “User friendliness” Devices Easy to use controls, use your phone without a manual Computers Mac & Windows technology opens use with menus, forms Web Sites Can you find and accomplish what you need to do?
4	Speech Technology and UI Design In addition to Make it friendly and easy to use Make the voice pleasant and conversational (equivalent of color choice and graphics on the web) Allow more callers to accomplish their tasks and More callers to accomplish their tasks quickly and more efficiently User interface design in speech applications can change the apparent performance statistics of the technology itself And … realize the goal of many automation projects: To automate procedures and keep callers away from representatives except when required
5	Speech Technology and UI Design A long history … American Express (Network World 1990) Only 3 out of 3,000 callers spoke dollar amounts as required by service 30 sec explanation of rounding amount reduced to 2 sec Now Amex trials all new voice processing technologies before deploying them AT&T Voice Response Operator Service (1990s) Iterative user testing of prompt designs lowered transaction times by half a second, working out to savings of millions of dollars for AT&T and customers Design of prompts – and other dialog elements – can many times results in enormous improvements in success rates, eclipsing improvements from tuning grammars and algorithms.
6	Technology from the UI Point of View Classification by machine’s ability to respond Directed Dialog Command words and phrase-based responding Voice menus – key phrases and common synonyms Transactions using number inputs Yes/No Prompting for exact phrase Grammar designed for prompted phrases, variations, synonyms, and unprompted options Slot-filling E.g. ATIS, airline reservations Less constrained speech Pertinent information (dates, times, locations) captured by grammar design
7	Technology from the UI Point of View Classification by machine’s ability to respond Natural Language How May I Help You?^SM Intent Determination Unconstrained speech accepted by machine User asks questions or responds freely, in their own words No vocabulary or syntax constraints imposed upon caller There is no “out of grammar” response Callers may be too vague or respond not on task Data-driven Natural language is trained on natural utterances callers give to a “how may I help you?” prompt or attendant NLU classifies responses into a set of predefined categories Pay-Bill, Get-Copy-of-Bill, etc.
8	General Approach for All Dialogs Find out the optimal design How would humans do this? Identify tasks Model the dialog Utilize Core Dialog Design Principles The four core principles are: Minimize cognitive load Balance efficiency with clarity Assure high accuracy Ensure graceful error recovery
9	Simple Intent Determination Design “Receptionist”
10	The Process: Language Analysis and Category Identification
11	Mapping of User Intent to System Capabilities This stage of analysis can be rich with renewed understanding of caller needs and business procedures
12	Labeling Guide Drives NLU Training and Dialog Manager Design
13	Design Principles: Natural Language The Greeting Prompt Boyce (2000) “I am an automated assistant. You can speak naturally to me. How may I help you.” Several laboratory and on-line system studies: balance between performance (best language in responses) and customer preference Other strategies Provide examples of what to say -- “prime” callers Welcome to Clarion Wireless Customer Service. You can ask me about things like ‘minutes used’, ‘automatic payments’, and ‘calling plans’. So, how can I help you with your account? … You can say things like “I want to activate my phone’ and ‘I want to change my voicemail password” … (Sheeder & Baloguh, 2003) à This has not worked with AT&T VoiceTone
14	Design Principles: Greeting Prompt A effective strategy: Combine Boyce (2000) prompt … Natural, conversational Efficient & fast for majority of callers “I am an automated assistant. You can speak naturally to me. How may I help you.” … with “directed re-prompting” (Blanchard & Stewart, 2004) If fail on first try – give more direction on second try Give direction in general categories, as a person would, rather than dictate speech examples Okay, in order to direct your call, please tell me if you need to refill and existing medication, find out the status of your prescription order, or anything else … I’m not sure what you need, please tell me if you want your account balance, or whether you needed to know if a payment has been applied to your account Okay, in order to direct your call, please tell me if you need to refill an existing prescription, or what else you want to speak to the Pharmacist about.
15	Using the technology: Vague Responses BEFORE (a) Okay. What's your question? (FIRST TRY) (b) Okay, I'm going to connect you to a customer service representative. Please wait. (SECOND TRY) AFTER VAGUE REQUEST ABOUT AN ORDER: I'm sorry, do you need to refill an existing medication or get status on an order you've already sent in? Please tell me how I may help you. ALL OTHER VAGUE REQUESTS: (a) Okay. What's your question? (FIRST TRY) (b) I'm sorry, do you need to refill an existing medication or get status on an order you've already sent in? Please tell me how I may help you. (SECOND TRY)
16	Designing the Fulfillment Dialog Directed Dialog Menus: Prompt for options Keep options to manageable length Trade-off depth and breath of menus Use Prompt-Pause-Options with barge-in if expect frequent expert usage User caller’s language Provide for flexibility in caller responses Allow synonyms and variations Trade-off with grammar complexity
17	Designing the Fulfillment Dialog Error handling Do not blame user Direct user with more detailed information on re-prompting
18	Persona Identify tasks Develop prompts (no prompt naturalness or voice talent coaching) Insert standard error recovery
19	Persona Identify tasks Model the Dialog Derive prompts (natural, conversational, coaching) Include error recovery based on overall dialog structure
20	Yes/No Questions Are Your Friends Yes/No questions can be made very conversational Do not say if this is correct, say ‘yes’ now If you said ‘track a package’ say ‘yes’ otherwise say ‘no’ Ballentine & Morgan (2001) Use Is this correct? Did you say ‘track a package’? Direct user with more information on re-prompting: … Please say ‘yes’ or ‘no’ Many dialogs can be yes/no questions instead of menus Must trade off the efficiency vs. “twenty questions” effect If one option is extremely frequent – consider first prompting that one option with yes/no – then following with menu
21	Example: Voice Mail
22	Contact: Harry E. Blanchard AT&T Shannon Labs 200 Laurel Ave. S. Middletown, NJ 07760 hblanchard@research.att.com 732-420-6894