I always add a line to my prompts something to the effect of "Ask as many clarifying questions as need be before answeing."

Thanks and in your experience, does it work?

It works quite well. Try out MyChefAI.com. I am using this method there to get the LLM to ask several questions to the user.

My own experience is that when it misinterprets an ambiguity, it's immediately obvious to me what was ambiguous in my question and I can add clarification. Asking for a clarification always adds an extra step, compared to giving me a likely answer which only adds an extra step when it's wrong.

This is a problem I've seen too but I haven't dug into it. Could you try specifically asking what extra information the llm would want in order to give the best answer (or e.g. what information is missing) to push it to answer that question specifically?

Thats a good suggestion. Will try it and revert back if it works :)

Few days abck I chatted with stanford alpaca folks. Got the response that no clear solution as of now. All the transformer models are design to answer the questions in a given context.