Why Students Click Straight Down the Middle: Satisficing in Course Evaluations

In short: When a course-evaluation questionnaire gets long or tedious, many students stop optimising their answers and start satisficing — giving a good-enough response with the least effort: clicking straight down one column, picking the first plausible option, agreeing by default, or quitting early. Barge and Gehlbach (2012) found that 61% and 81% of students satisficed on at least one measure across two campus surveys, and — the counter-intuitive part — that satisficing inflated internal-consistency and inter-scale correlations, making the data look more reliable and valid than it was. Reducing satisficing is therefore not a cosmetic concern; it is a precondition for trusting any number your instrument produces.

The question this article answers

Every quality office worries about who doesn''t respond (non-response bias). Far fewer worry about how carelessly the people who do respond actually answer. Yet inattentive responding can corrupt your data in a more insidious way than non-response, because it doesn''t just lose information — it manufactures fake signal. This article explains the mechanism, the evidence, and what to do about it.

What the research says

The concept comes from Jon Krosnick''s (1991) theory of survey satisficing, published in Applied Cognitive Psychology. Krosnick borrowed Herbert Simon''s term to describe what respondents do when the cognitive work of answering well exceeds their motivation. Optimal answering requires four steps — comprehending the question, retrieving relevant information, integrating it into a judgement, and reporting it on the scale. Satisficing is short-circuiting that process. Krosnick distinguished weak satisficing (going through the motions but taking shortcuts) from strong satisficing (skipping retrieval and judgement almost entirely), and catalogued the tell-tale behaviours: choosing the first reasonable option (primacy), acquiescence (agreeing with whatever the item asserts), non-differentiation / straightlining (giving near-identical ratings across diverse items), defaulting to the midpoint or "don''t know", and effectively random selection. Satisficing rises with task difficulty, falls with respondent ability and motivation, and grows as a questionnaire drags on — late items and long surveys are where it concentrates.

Steven Barge and Hunter Gehlbach (2012), in Research in Higher Education, turned this theory into measurement. They operationalised satisficing as a set of observable behaviours — straightlining/non-differentiation, item-skipping, speeding, and dropping out — and computed a satisficing index per respondent across two student surveys. Two findings matter most:

1. Satisficing is the norm, not the exception. Most students satisficed on at least one indicator — 61% in one survey and 81% in the other. This is not a fringe of careless respondents; it is a majority behaviour that any instrument must assume is present.
2. Satisficing biases your quality statistics upward. This is the crucial, under-appreciated result. Straightlining and non-differentiation artificially inflated internal-consistency estimates (Cronbach''s alpha) and the correlations between scales. In other words, the very statistics analysts use to argue an instrument is reliable and that its constructs hang together can be artefacts of inattentive responding. Bad data can look psychometrically excellent.

Christine Vriesema and Hunter Gehlbach (2021), in Educational Researcher, extended this line, showing that unmotivated questionnaire responding systematically degrades data quality and that researchers routinely overestimate the trustworthiness of self-report data because they do not screen for it.

Why it matters for course evaluation in practice

The Barge–Gehlbach result reframes a common reassurance. When a vendor or QA report says "our instrument has high internal consistency and the subscales correlate sensibly", that is exactly the pattern satisficing produces. High alpha is not proof of careful responding; under straightlining it can be proof of the opposite.

Practical consequences:

• Long, repetitive Likert grids are self-defeating. The more items you stack, the more you push students from optimising into satisficing — and the worse (yet better-looking) your data becomes.
• End-of-survey items are the least trustworthy. Fatigue concentrates satisficing late, so the questions you put at the bottom get the weakest answers.
• You cannot detect the problem from the means alone. Satisficing hides inside healthy-looking aggregate statistics. You need response-level behavioural indicators (timing, non-differentiation, drop-off) to see it.
• Acquiescence contaminates positively worded item banks. If most items are phrased as agreements with flattering statements, default-agreers inflate scores in a direction that looks like satisfaction.

Limitations and honest caveats

The satisficing literature deserves the same scrutiny it applies to surveys.

• Indices are heuristics, not ground truth. A student who rates several items identically may be satisficing — or may genuinely hold consistent views. Non-differentiation is a probabilistic flag, not proof of carelessness, and over-aggressive screening can discard valid data.
• Speed is ambiguous. Fast responses can mean inattention or fluency. Timing thresholds are context-dependent and easy to mis-set.
• Self-report measurement of a self-report problem. Some operationalisations rely on the same instrument they critique, which introduces circularity.
• Generalisability. Barge and Gehlbach studied particular campus surveys; satisficing rates vary with stakes, length, mode, and culture (and interact with the cross-cultural response-style effects covered elsewhere in this knowledge base).

The honest takeaway is not "discard satisficed responses" but "design so that satisficing is harder to fall into, measure it when you can, and stop treating high internal consistency as automatic evidence of quality."

How Koji incorporates this

Satisficing is fundamentally a motivation-and-effort failure of static questionnaires, which is precisely the failure Koji''s conversational design targets.

• Conversational interviews raise the cost of a non-answer. Koji''s AI-moderated interviews ask open-ended questions and follow up, so the path of least resistance is no longer "click the middle column". Producing a coherent spoken or typed answer requires the retrieval-and-judgement step that satisficing skips — engagement is built into the format rather than fought against it.
• Adaptive probing detects thin responses. When an answer is short or non-committal, Koji is designed to ask a clarifying follow-up, converting a would-be satisficed response into a substantive one rather than recording the shortcut.
• Shorter, dynamic instruments reduce fatigue. Because the interview adapts and skips irrelevant branches, students face fewer redundant items than a fixed Likert grid, lowering the fatigue that drives late-survey satisficing.
• Mixed question types break straightlining. Combining open_ended, scale, single_choice, multiple_choice, ranking, and yes_no items interrupts the down-the-column pattern that produces non-differentiation.
• Quality scoring flags low-effort responses. Koji applies response quality scoring so that thin, contradictory, or speeded answers can be down-weighted or surfaced — addressing the Barge–Gehlbach warning that inattentive responses otherwise inflate reliability statistics.

Koji is designed to mitigate satisficing, not to claim immunity from it — a sufficiently disengaged respondent can give thin answers in any medium. The point is to make the careful answer the easy answer. The same conversational, follow-up-driven engine powers Koji''s core research platform at koji.so, where satisficing equally distorts customer and product surveys.

How to spot satisficing in your own data

You cannot fix what you cannot see, and the Barge–Gehlbach warning is precisely that satisficing hides inside healthy-looking aggregate statistics. Four response-level diagnostics make it visible. Non-differentiation (straightlining): flag respondents whose answers across a battery of differently worded — and ideally oppositely worded — items show near-zero variance. Genuine consistency exists, but a long run of identical clicks across items that should diverge is the clearest signature of effort-conservation. Completion speed: record per-respondent timing and inspect the fast tail; responses completed far below the time it physically takes to read the items deserve scrutiny (though fluency, not just inattention, can produce speed, so treat it as a flag, not a verdict). Item non-response and drop-off: rising skip rates and abandonment toward the end of an instrument reveal where fatigue tips students into strong satisficing — and tell you your questionnaire is too long. Acquiescence checks: include reverse-worded items; a respondent who "agrees" with both a statement and its opposite is defaulting to agreement rather than reporting a view.

The constructive response is not mass deletion of flagged responses — that risks discarding valid data and introduces its own bias — but a combination of design and weighting. On the design side, shorten instruments, vary item formats, and avoid long homogeneous Likert grids. On the analysis side, treat internal-consistency statistics with suspicion when satisficing indicators are high, report them alongside a satisficing audit rather than in isolation, and consider down-weighting clearly low-effort responses rather than letting them silently inflate your reliability estimates. The goal is a standing habit: never report an alpha or a scale correlation without first asking how much of it the straightliners built.

Related Resources

• How Many Scale Points Should a Course-Evaluation Question Have?
• Response Styles and Likert Scales: Why Cross-Cultural Evaluation Needs More Than Numbers
• Survey Fatigue: Why Over-Surveying Students Wrecks Response Rates
• What Can Open-Text Student Comments Tell You That Likert Scores Cannot?
• How Many Responses Do You Need for a Reliable Course Evaluation?

References

• Barge, S., & Gehlbach, H. (2012). Using the theory of satisficing to evaluate the quality of survey data. Research in Higher Education, 53(2), 182–200. https://doi.org/10.1007/s11162-011-9251-2
• Krosnick, J. A. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5(3), 213–236. https://doi.org/10.1002/acp.2350050305
• Vriesema, C. C., & Gehlbach, H. (2021). Assessing survey satisficing: The impact of unmotivated questionnaire responding on data quality. Educational Researcher, 50(9), 618–627. https://doi.org/10.3102/0013189X211040054