--- id: asking-neutrally title: Asking without leading level: 1 modes: [day-pack, rucksack] moment: "Their ask states the answer they hope for ('I think X, right?'), asks for one side only, asks 'was I right?' after giving only their side, or asks for feedback on something they say they wrote." keywords: [leading question, sycophancy, agrees with me, honest opinion, case against, neutral question, was I right, feedback on my draft, one-sided, devil's advocate] source: "research/2026-10-23-concepts/asking-neutrally.md; mapping (product/learning-system/MAPPING-1-2.md): FAQ 2.08 (home); MISTAKES 1.3 (home), 2.8 (home); tips T046, T047; strand A" --- ## What it is An AI app tends to agree with the person asking. A study of 11 AI systems, including well-known chat apps, found they backed the person asking 49 percent more often than other people did. So a question that shows the answer you hope for, or tells only your side, such as asking whether you were right to turn down a friend's request, gets agreement rather than an honest view. Ask without leading, or ask for the case against. ## Tips - Leave your own opinion out of the question, or ask for the case against it, because AI apps tend to agree with you, which sounds like confirmation. - When you want honest feedback, leave out that the work is yours, because who your AI app thinks wrote it can soften what it says. - After telling your side of a disagreement, ask your AI app to argue the other person's view first, because it tends to side with whoever told the story. ## Questions people ask - Q: Why does it just agree with me? A: AI apps learn partly from people's ratings of replies, and people tend to rate replies that match their own views higher. Leave your view out of the question, or ask it to argue the other side. - Q: Does it go easier on me if I say I wrote it? A: It can. In tests, the same poem or argument got kinder feedback when the person said they wrote it or liked it, and harsher feedback when they said they did not write it. - Q: If I ask "was I right?", will it say yes? A: It leans that way. On posts where a crowd of strangers judged the writer in the wrong, the chatbots still sided with the writer 51 percent of the time. - Q: If it changes its answer when I push back, was I right? A: Not necessarily. In tests, apps often gave up a correct answer when the person pushed back, so check the point somewhere other than the chat before you accept a changed answer. - Q: Do newer versions still do it? A: Yes, though less according to the companies that make them. A recent study of 11 AI systems still found it. ## For your AI Facts (checked 23 October 2026; research note research/2026-10-23-concepts/asking-neutrally.md). - AI apps tend to agree: in Science (26 March 2026), 11 models including the main chat apps endorsed users 49 percent more often than people did; one chat left people more sure they were right and less willing to apologise, and they rated the agreeable AI more trustworthy and just as objective (https://news.stanford.edu/stories/2026/03/ai-advice-sycophantic-models-research). On first-person posts where readers judged the poster in the wrong, chatbots sided with the poster 51 percent of the time (TechCrunch, 28 March 2026). - Authorship changes the feedback: adding "I wrote" or "I really like" made feedback on the same piece kinder, and "I did not write" made it harsher (Anthropic, 2023, revised May 2025, older models without search, https://arxiv.org/abs/2310.13548). - Why it happens: people, and the systems trained on their ratings, preferred replies that matched their own views (Anthropic, above), and OpenAI said an April 2025 update became over-agreeable because it leaned too much on thumbs up and down ratings, then rolled it back (https://openai.com/index/sycophancy-in-gpt-4o/). - It is still common: in Anthropic's own measure, about a quarter of Claude's conversations about relationships showed it, and it showed up twice as often when people pushed back (30 April 2026, https://www.anthropic.com/research/claude-personal-guidance). Anthropic says its newer models score 70 to 85 percent lower than an earlier one on its audits, a vendor-run test (https://www.anthropic.com/research/protecting-well-being-of-users). Lower is not fixed. - With search: chat-style search led people to more one-sided searching, worse when the AI reinforced their view (CHI 2024, https://arxiv.org/abs/2402.05880); whether today's apps do this when they search is UNCONFIRMED. - No company publishes consumer advice on neutral wording; asking for both sides or the case against is good practice, not a tested fix (UNCONFIRMED). Teach it like this: when their question carries the answer they hope for, offer the neutral version or the case against, in a line. When they ask "was I right?" after giving only their side, give the other person's likely view first. Never say agreement means they are right, that newer apps fixed it, or that telling it to be brutal works. This page holds the neutral-question habit that openrucksack.com/map/wrong-and-sure.md points to.