SIFT: Search Conjoint

\[ \newcommand{\iid}{\stackrel{iid}{\sim}} \newcommand{\bfbeta}{\boldsymbol{\beta}} \newcommand{\bfgamma}{\boldsymbol{\gamma}} \newcommand{\bfkappa}{\boldsymbol{\kappa}} \newcommand{\bfeta}{\boldsymbol{\eta}} \newcommand{\bftheta}{\boldsymbol{\theta}} \newcommand{\bfOmega}{\boldsymbol{\Omega}} \newcommand{\bfX}{\mathbf{X}} \newcommand{\bfL}{\mathbf{L}} \newcommand{\bfZ}{\mathbf{Z}} \newcommand{\Jset}{\mathcal{J}} \newcommand{\Sset}{\mathcal{S}} \newcommand{\Sbar}{\bar{\mathcal{S}}} \newcommand{\resv}{z} \newcommand{\cost}{c} \]

1 Introduction

Conjoint analysis answers a question that begins after the shopping has already happened. A respondent is shown a handful of alternatives, described on the attributes the analyst chose, and asked which one they would buy. From repeated choices we recover preferences, and from preferences we simulate share. The method is forty years old, industrially standardized, and very good at what it does.

What it does not do is ask what the respondent would have looked at. Every alternative in a conjoint task is fully described and costlessly examined; the respondent evaluates all of them because the instrument gives them no alternative. Real purchases rarely work that way. A shopper faces a list of options carrying a few visible attributes — a name, a price, an image — clicks into a handful to learn the rest, and buys from among those. Most of the category is never opened at all. Conjoint has no representation of that behaviour and therefore no way to distinguish an alternative that was rejected from one that was never seen.

1.1 What we propose

Conjoint analysis was, in essence, the discrete choice model brought into a survey. The econometric object already existed; the contribution was an instrument that manufactured the variation the model needed, by design rather than by luck, and a set of conventions for fielding and estimating it at commercial scale.

We do the same thing for the sequential search model. SIFT — Search Intensity and Feature Tradeoffs — is a survey instrument in which the respondent faces a landing page of alternatives showing a few attributes, clicks through to product pages to reveal the rest, and then buys one or none. Each respondent completes a sequence of such tasks. Position on the landing page, the content of the landing page, and the content of each product page are randomized in every task. The resulting click-and-choose data are fit with a sequential search model in the tradition of Weitzman (1979), and the estimates include both the preference parameters conjoint delivers and a search cost, for which conjoint has no analogue.

The econometric object again already exists. Structural search models have been estimated on clickstream and platform data for two decades (Kim et al. 2010; Chen and Yao 2017; Ursu 2018; Morozov et al. 2021). What has not existed is an instrument that designs the variation those models need instead of waiting for a platform to supply it. Ursu (2018) obtained randomized rankings once, from Expedia, as a rare gift. In a survey, that randomization is free and available in every task.

1.2 Conjoint is the zero-search-cost special case

The relationship between the two methods is nesting, not competition. Drive the search cost to zero in a sequential search model and every alternative is inspected, choice is made over fully realized utilities, and what remains is a standard choice-based conjoint model.

This matters for three reasons. It is non-adversarial: we are generalizing the industry workhorse rather than attacking it, and a client who has bought conjoint for a decade need not concede that they were wrong. It is roughly true rather than a rhetorical convenience. And most usefully, it makes the choice between methods an empirical question rather than a matter of taste. Estimate the search cost and test whether it is distinguishable from zero. If it is not, conjoint was well specified for that category and nothing has been lost. If it is, conjoint was misspecified and the evidence is in hand.

1.3 What SIFT buys

It separates non-purchase into two diagnoses that conjoint cannot tell apart. When a brand underperforms, conjoint can only report that it was not valued enough. SIFT distinguishes alternatives that were never opened from those that were opened and rejected. These have opposite remedies — distribution, placement, and media on one side; product and price on the other — and a method that cannot separate them will send a manager to the wrong lever roughly whenever consideration rather than preference is binding.

It represents consideration, which conjoint structurally cannot. The claim here is narrower than it is sometimes made, and the narrower version is the defensible one. A conjoint task shows every attribute of every alternative, so it measures preference under full information, and in simulation it does so accurately — recovering preference ratios to within 0.03, including for attributes that a shopper would only discover by inspection (Section 6). Conjoint does not mismeasure preference.

The difficulty is downstream. Share is the product of the probability that an alternative is considered and the probability that it is chosen once considered. Conjoint estimates the second term well and sets the first to one, so every question about what gets seen falls outside what the instrument can express. In the same simulation, purchase share falls by half again from the top of the listing page to the bottom; conjoint predicts a ratio of exactly one, because a conjoint task has no position and the fitted model has no parameter that could carry one. This is not a bias that a larger sample would reduce. The model cannot state the proposition that promoting a product changes its share.

It answers placement and merchandising questions that conjoint cannot pose. What happens to share if we move from position six to position two? What belongs on the listing tile versus the detail page? These are live questions for any marketplace, retail media network, app store, or streaming carousel, and they are questions about what gets seen, not about what is preferred once seen. Because SIFT randomizes position and page content by design, they are answered directly rather than assumed away.

It absorbs large attribute spaces. Conjoint degrades as attributes accumulate: respondent burden rises, number-of-levels effects intrude, and simplification heuristics contaminate the very tradeoffs the design was built to measure. SIFT distributes a large attribute space across two pages and lets respondents select what to examine. Non-attention stops being noise to be minimized and becomes data to be modelled.

It prices friction, and friction is a lever managers control. The search cost is estimated in utility units and convertible to money. Fewer clicks to a decision, better filters, better defaults, and better placement are all interventions on search cost, and SIFT is the only preference-measurement instrument that quantifies what they are worth. Conjoint offers nothing here, because in conjoint search is free.

It looks like shopping. The task is a listing page and some product pages. Respondents recognize it, which helps engagement and data quality, and clients recognize it, which helps adoption. A conjoint task, by contrast, is a grid that resembles no purchase anyone has ever made.

1.4 What makes it work

Three design choices do the identification work, and each replaces an assumption that field studies must defend with a fact the analyst controls.

Position is randomized within every task, so the exogenous search-cost shifter that Yavorsky et al. (2021) had to argue for — driving distance, defended at length — is true by construction. The product page is written by the analyst, so the attributes a click reveals are observed rather than treated as an unobserved shock; preferences over post-click attributes are therefore identified in exactly the way conjoint part-worths are. And each respondent completes many tasks, yielding a panel of search spells where field data typically offers one, which is what supports individual-level estimates rather than population averages (Morozov et al. 2021).

These three do separate work, and it is worth keeping them separate. The randomized shifter is what allows the standard deviation of the post-search component to be estimated rather than normalized to one, as most of the literature must; the panel is what supports individual-level parameters. That normalization is not innocuous. Yavorsky et al. (2021) estimate the spread at 8.16 against the conventional value of one, and show that imposing one reverses the sign of a search-cost coefficient and understates a counterfactual by a factor of three to four. In our own simulations the normalization does worse than bias the counterfactual: at spreads in the range they report, it predicts that halving search costs would change behaviour hardly at all, when in truth clicks rise by roughly a third (Section 6). Section 5 develops these arguments and reports the design checks that should accompany every SIFT study.

1.5 What it costs

The method carries costs as well, and we set them out alongside the advantages rather than confine them to a concluding caveat.

Complexity. SIFT is a new instrument fit with an unfamiliar model. Conjoint has four decades of accumulated norms — sample-size rules of thumb, design efficiency criteria, validated share simulators, benchmarks by category — and SIFT has none of that yet. It is roughly where conjoint was in the mid-2000s: the statistical machinery works, but the practitioner scaffolding that makes a method safe in ordinary hands has not been built. Anyone adopting SIFT now is adopting a method, not a product.

Estimation. Sequential search likelihoods are expensive. Where a hierarchical Bayes conjoint model fits in minutes, a comparably sized SIFT model takes hours: our implementation measures roughly eleven hours for a commercial-scale fit, and only when the sampler is parallelized across respondents — single-threaded it is eight times that. We regard this as an engineering problem rather than a fundamental one, and it is improving quickly: Chung et al. (2024) report speedups of one to two orders of magnitude over the kernel-smoothed simulators that dominated the literature. But it is a real constraint on iteration today.

Less precise preference estimates. A conjoint task reveals every attribute of every alternative; a SIFT task reveals post-click attributes only for the alternatives a respondent chose to open, which at realistic inspection rates is a small minority. Conjoint therefore carries more information per task about preferences for attributes that require inspection, and in simulation it recovers them more accurately than SIFT does (Section 6). SIFT buys a consideration stage at some cost in preference precision, and a study whose only object is preference measurement should use conjoint.

Precision on the search cost. Preferences are recovered adequately at commercial sample sizes. The level of the search cost is harder, and carries a finite-sample bias that declines slowly in both sample size and simulation draws (Section 6). This is a known property of simulated maximum likelihood estimation of search models rather than a feature of our instrument — Yavorsky et al. (2021) reports the same of his own simulations — but it means that conclusions about search-cost magnitudes demand larger samples than conclusions about preferences, and we quantify how much larger.

A behavioural assumption. The model assumes respondents search optimally in the sense of Weitzman (1979). The experimental evidence on that assumption is mixed: laboratory subjects tend to search less than the optimum, exhibit order effects, and show sensitivity to sunk costs the theory says should be ignored. This is the most substantive scientific risk to the approach, and it applies with equal force to the structural search literature SIFT builds on. Our position is that the model should be judged as an approximation that recovers useful quantities, not as a claim about cognition, and that its predictions should be checked against holdout behaviour rather than assumed. Section 9 returns to this.

1.6 Roadmap

Section 2 positions SIFT against the search, conjoint, and consideration-set literatures. 2 describes the instrument. Section 4 develops the model, and Section 5 treats identification and estimation. Section 6 reports a simulation study establishing what the design recovers and at what sample sizes. Section 7 presents an empirical application, Section 8 draws out what the estimates support that conjoint cannot, and Section 9 takes up limitations.

2 Related Literature

SIFT sits between two literatures that have not had much to say to each other. Structural search models are estimated on data that platforms happen to generate; preference measurement designs data deliberately but assumes away search entirely. This section covers the work SIFT actually builds on, organized around four questions: what the sequential search model is and how it is estimated, what field data can and cannot identify, whether people search the way the model says, and how marketing has handled limited consideration until now.

A fuller annotated map of the surrounding literatures — including work SIFT touches but does not depend on — is in Section 11.3.

2.1 The Sequential Search Model and Its Estimation

The theoretical foundation is Weitzman (1979), which solves the problem of inspecting alternatives one at a time when inspection is costly and the distribution of what inspection will reveal is known. The solution is a reservation value for each alternative, computed independently of every other alternative, together with three rules: inspect in decreasing order of reservation value, stop when the best realized utility exceeds the highest remaining reservation value, and choose the best alternative inspected. The independence of the reservation values is what makes the model tractable, and it is why the sequential search model rather than a general dynamic program is the workhorse of this literature.

1 is the consolidation of two decades of empirical work with that model and the best single entry point to its current state. They set out the specification choices that matter — what the consumer knows before searching, whether search is sequential or simultaneous, how the outside option enters — and, importantly for us, they treat identification and estimation as first-class problems rather than implementation details. Their comparison of simulators is the reference point for Section 5, and their discussion of what is normalized rather than estimated frames the central methodological claim of this paper.

Estimation is genuinely hard, and the difficulty shapes what the literature has been able to do. The search and purchase probabilities have no closed form, so the likelihood must be simulated, and the model’s inequality restrictions make naive frequency simulators both noisy and discontinuous in the parameters. Kim et al. (2010) supplies the closed-form reservation value under normally distributed match values that nearly every subsequent paper uses. Practice has since moved through kernel-smoothed accept–reject simulation, which is general but requires tuning constants that must themselves be calibrated, toward smoother and faster alternatives: Ackerberg (2009) for importance sampling, and Chung et al. (2024) for a reformulation that makes the search cost rather than a pre-search taste shock the stochastic component of the reservation value, yielding a GHK-style recursive simulator that reports speedups of one to two orders of magnitude. We port and validate that estimator and compare it against our own in Section 5.

2.2 What Field Data Can and Cannot Identify

Empirical work with this model has relied on data that platforms produce for other reasons — clickstreams, booking logs, view-rank data — and the resulting identification arguments are constrained by what those data happen to contain.

Two papers define the frontier, and between them they show that the two things SIFT needs have never been available together.

Morozov et al. (2021) is the panel argument. Using multiple search sessions per consumer, they establish nonparametric identification of individual-level preferences and search costs, which cross-sectional search data cannot deliver. Their result is the closest existing warrant for SIFT’s central claim, and it is also the source of an important caution: the argument takes the number of sessions per consumer to infinity, and with a mean of five sessions in their data they report hyperparameters rather than individual parameters. They also supply the cleanest available statement of why preferences and search costs separate at all — stopping decisions respond to the realized utilities of alternatives already inspected but not to their search costs, which are sunk — and turn it into an observable moment we can compute directly from a click sequence.

Yavorsky et al. (2021) is the shifter argument. Estimating a search model on dealership visits in the U.S. auto market, they show that an exogenous and alternative-specific search-cost shifter — driving distance — allows the standard deviation of the match-value distribution to be estimated rather than normalized to one, as the literature had generally been forced to do. The consequences are large: they estimate it at 8.16, and fixing it to one reverses the sign of a search-cost coefficient and understates a counterfactual by a factor of three to four. Their setting has no panel, so heterogeneity is identified cross-sectionally or by parametric assumption.

Ursu (2018) illustrates the third constraint, which is that exogenous variation in this literature arrives as a gift rather than a design. She obtains randomized hotel rankings from a platform experiment and uses them to argue that position enters search costs rather than utility — an argument she has to construct from observational tests precisely because she cannot manipulate position herself. Ursu et al. (2023) and Morozov (2023) extend the model in directions that matter for what search costs mean, and Greminger (2022) relaxes the assumption that the choice set is known at the outset.

The pattern across all of this is that each study has one of the ingredients. SIFT’s contribution is not a new estimator but an instrument in which the panel, the exogenous shifter, and control over what inspection reveals are all present simultaneously and by construction.

2.3 Do People Search the Way the Model Says?

This is the most substantive scientific risk to the approach, and it applies with equal force to every paper cited above. We state it here rather than in a limitations paragraph at the end.

Experimental tests of optimal sequential search go back to SCHOTTER and BRAUNSTEIN (1981) and have accumulated steadily since. The verdict is mixed rather than negative. Hey (1987) finds subjects using rules of thumb in place of optimal reservation strategies. Cox and Oaxaca (1989), studying finite-horizon search, offer the more favourable reading: behaviour broadly consistent with reservation rules, plus noise. Sonnemans (1998) uses verbal protocols to classify the strategies people actually report using, and Brown et al. (2011) compares search in the laboratory against search in a real market. The deviations that recur are systematic rather than random — too little search relative to the optimum, order effects, and sensitivity to sunk costs the theory says should be ignored.

A second experimental literature has, for decades, run something mechanically very close to the SIFT task. The Mouselab paradigm of Payne et al. (1988) and Payne et al. (1993) presents subjects with an information board whose cells must be clicked to reveal attribute values, recording the acquisition sequence. That is, structurally, a listing page and a set of product pages. Reutskaja et al. (2011) adds eye-tracking evidence under time pressure with supermarket-style displays, and Caplin et al. (2011) uses choice-process data to show that people frequently stop before finding the best available option — the satisficing alternative against which any Weitzman-based model should be asked to justify itself.

Gabaix et al. (2006) is the closest existing precedent for what we do: a Mouselab-style acquisition experiment paired with a structural boundedly-rational model of directed cognition, estimated on the acquisition record. The model is not Weitzman’s, but the maneuver is the same one — design an information-acquisition task, then fit a structural search model to what people acquire.

Our position is that the experimental apparatus has been well validated for decades, and what has largely not been done is fitting a Weitzman model to designed acquisition data and reporting search costs in interpretable units. We treat the model as an approximation to be checked against holdout behaviour rather than as a claim about cognition, and Section 9 returns to what follows if the approximation fails.

2.4 Limited Consideration in Preference Measurement

Marketing has long recognized that respondents do not evaluate everything, and has answered with models that infer screening from choice outcomes. Gilbride and Allenby (2004) estimate conjunctive, disjunctive, and compensatory screening rules within a choice model. Hauser et al. (2010) model disjunctions of conjunctions, Yee et al. (2007) use greedoid methods for non-compensatory inference, Dzyabura and Hauser (2011) apply active machine learning to consideration heuristics, and Hauser (2014) reviews the resulting body of work.

The contrast with SIFT is a single distinction: these methods infer the consideration process as latent structure from what was chosen, whereas SIFT observes it directly and prices it in units of search cost. Both approaches are legitimate, and inference from choice outcomes has the considerable advantage of applying to data already collected. A latent screening rule cannot, however, be manipulated, and so cannot answer what happens when the analyst changes what is easy to see.

The broader economic foundations of conjoint are set out in Allenby et al. (2019), and Ding et al. (2005) and Ding (2007) established incentive alignment as a way to make stated preference behave more like revealed preference — a concern SIFT inherits and does not solve.

3 The SIFT Instrument

3.1 Task Architecture

A SIFT task presents the respondent with two kinds of screen, arranged the way online retail arranges them. The first is a listing page: a set of \(J\) alternatives displayed as an ordered vertical list, each occupying one row. The second is a product page, one per alternative, reached by clicking its row. The pair corresponds to what commercial practice calls the product listing page and the product detail page, and the correspondence is deliberate — the instrument is meant to look to the respondent like the environments in which the category is actually shopped.

The two screens differ in what they show. Every alternative is described by the same set of attributes, but the attributes are partitioned. A small number are displayed on the listing page for all \(J\) alternatives at once — position in the list, brand, and price are the natural candidates, and are the ones that list pages typically carry in practice. The remainder are shown only on an alternative’s product page. A respondent who reads the listing page and clicks nothing therefore knows a little about every alternative; a respondent who opens a product page knows everything about that one alternative and still only a little about the rest.

This partition is what gives the task its structure. Listing-page attributes are free: the respondent acquires them by looking at the screen she is already on. Product-page attributes are costly: acquiring them requires a click, a page load, and the time to read what the page contains. The respondent chooses how many of those costs to pay and in what order.

The protocol within a task is as follows. The respondent begins on the listing page. She may open any alternative’s product page, return to the listing page, and open another, as many times as she likes and in any order she likes. Every alternative she has already opened remains available to her — she may return to a previously viewed product page at no further cost, and she may select an alternative she viewed several clicks ago. The task ends in one of two ways. She selects an alternative to purchase, which she may do only from a product page she has opened, so that no alternative can be purchased sight-unseen. Or she declines to purchase anything and exits the task, which she may do at any point, including immediately, without opening a single product page.

Two features of this protocol deserve emphasis because the model in Section 4 rests on them. First, selection requires prior inspection, which is both realistic — one buys from the product page, not from the list — and what allows the observed choice to be read as a choice over resolved rather than expected utilities. Second, recall is free and unlimited, so a respondent is never punished for having looked at something and moved on. Together these place the task squarely in the Weitzman (1979) protocol: the listing page is a set of closed boxes whose contents are unknown but whose distributions are not, a click is the act of opening one at a cost, and the decision to stop and select is the decision to take the best prize already in hand rather than pay to open another. The no-purchase exit is the outside option, which the respondent may take whether or not she has searched.

The exit option matters for what the instrument can measure. A respondent who searches nothing and leaves, a respondent who searches half the list and leaves, and a respondent who searches half the list and buys are producing three different pieces of evidence about the same two parameters. Conjoint’s no-choice option records only the first distinction between them.

Respondents complete a sequence of tasks rather than one, exactly as in choice-based conjoint. Each task is a fresh listing page with a fresh set of alternatives, and the respondent searches and chooses within it independently. The current design fields ten tasks per respondent. This is the feature that separates the instrument most sharply from the field literature it borrows from. A platform dataset typically observes one search spell per consumer, so preferences and search costs must be told apart across consumers. Here each respondent supplies a panel of search spells, and the same respondent’s behavior across tasks can be used to separate what she likes from what looking costs her.

3.2 Randomization and Experimental Design

Three things vary from task to task, and all three vary by design rather than by circumstance.

Position. The order in which the \(J\) alternatives appear on the listing page is randomized independently of everything else about them. An alternative that appears first in one task appears fourth in another, with the same attributes and the same competitors. Position is thus uncorrelated by construction with quality, price, brand, and every other attribute — the correlation that makes observational ranking data so difficult to use, since platforms rank on relevance and relevance is a function of exactly the things that drive utility. Field work has obtained this variation only rarely and only as a gift, most cleanly in Ursu (2018). Here it is free and present in every task.

Composition. The set of alternatives available on the listing page varies across tasks: which alternatives appear, and how many. This is the analogue of varying the choice set in a conjoint design, and it serves the same purpose of placing each alternative against varied competition, but it does additional work here because the attractiveness of the alternatives a respondent has not yet opened is what determines whether she keeps searching.

Attribute levels. The attributes themselves are fixed across tasks — the same characteristics describe every alternative in every task, as in conjoint — while the levels those attributes take on are varied across alternatives and across tasks according to an experimental design. This is conventional conjoint practice and needs no defense; what is new is only that some of these attributes are revealed on the listing page and the rest behind a click.

A fourth lever is available and is not yet part of the design: the partition itself — which attributes appear on the listing page and which are held back for the product page — could be randomized across respondents rather than fixed. Doing so would make the information architecture an experimental treatment, and would let the instrument speak to what belongs on a list page versus a detail page. It also complicates identification, since it changes what the respondent knows before searching. We flag it here as a design option and return to it in Section 8.

We defer the question of what each of these three sources of variation identifies to Section 5, and the question of how they enter the likelihood to Section 4. The point to carry forward from this section is only that the variation is designed rather than found: it is orthogonal by construction, it is present in every task rather than in a single natural experiment, and it costs nothing to produce.

3.3 Data Recorded

The instrument records the full interaction, not merely its outcome. For each respondent \(i\) and task \(t\) we retain three layers.

The design. The complete attribute matrix for all \(J\) alternatives shown, whether or not the respondent ever saw them; the position each alternative occupied on the listing page; and the partition of attributes into listing-page and product-page sets. The design is known without error because it was generated, which is a meaningful advantage over field data, where the analyst typically knows what was clicked but must reconstruct what was displayed.

The event stream. Every action the respondent takes, in order, with a timestamp: each product-page opening (which alternative, at which position, at what time), each return to the listing page, each re-opening of a previously viewed product page, and the terminal event — a purchase of a named alternative or an exit without purchase. The event stream is the raw object; everything below is derived from it.

Derived search quantities. From the event stream we construct the objects the model in Section 4 takes as data: the search set, meaning which alternatives were opened; the search order, meaning the sequence in which they were opened; the stopping point, meaning how many alternatives were opened before the respondent stopped; and the choice, meaning which member of the search set was purchased, or that none was. Timing data yield a parallel set of duration measures — time to first click, dwell time per product page, total task duration — which are informative about search effort in the sense of Ursu et al. (2020) and which will also be needed to net out any imposed latency from voluntary reading time.

The implication for the alternatives a respondent never opened. They are not missing data. Under the model they are the outcome of a decision — the respondent judged that opening them was not worth the cost, given what she had already found — and they carry information about both her preferences and her search cost. Non-inspection is the observation that conjoint has no way to record, and recovering it is much of the point of the instrument.

4 Model

4.1 Setup and Notation

We follow the notation of 1 wherever the two are compatible, so that readers arriving from the sequential-search literature can map our objects onto theirs without translation. Two departures are forced on us by the instrument, and one by the object being searched over; we flag all three below. Section 11.1 gives a crosswalk to 1 and Yavorsky et al. (2021) for readers coming from either.

Indices. Respondent \(i = 1,\dots,N\) completes tasks \(t = 1,\dots,T\). Task \(t\) presents respondent \(i\) with a listing page holding the alternatives \(\Jset_{it} = \{1,\dots,J_{it}\}\), together with the no-purchase option \(j=0\), which is always available and never requires a click. The index \(j\) names an alternative, not its rank and not its screen position: because position is randomized (Section 3.2), the paper must hold three things apart at once — an alternative’s identity \(j\), the position it occupied on the listing page in task \(t\), and its rank in the respondent’s reservation-value ordering. Search order is carried by a separate index \(h\), where \(h=1\) denotes the first product page the respondent opened.

Utility. Respondent \(i\)’s utility from alternative \(j\) in task \(t\) is

\[ u_{itj} \;=\; \underbrace{\bfX_{itj}'\bfbeta_i + \mu_{itj}}_{\displaystyle \delta_{itj}\ \text{(known before clicking)}} \;+\; \underbrace{\bfL_{itj}'\bfkappa_i + \varepsilon_{itj}}_{\displaystyle \text{(revealed by clicking)}} , \tag{1}\]

where \(\bfX_{itj}\) collects the attributes displayed on the listing page, \(\bfL_{itj}\) the attributes displayed only on the product page, \(\mu_{itj}\) is a pre-search taste shock observed by the respondent but not by us, and \(\varepsilon_{itj}\) is a post-search taste shock. Write \(\xi_{itj} = \bfX_{itj}'\bfbeta_i\) for the observed part of pre-search utility and \(\delta_{itj} = \xi_{itj} + \mu_{itj}\) for pre-search utility. Both shocks are mean-zero normal, \(\mu_{itj} \iid N(0,\sigma_\mu^2)\) and \(\varepsilon_{itj} \iid N(0,\sigma_\varepsilon^2)\), with \(\sigma_\mu = 1\) imposed as the scale normalization. The no-purchase option has \(u_{it0} = \bfX_{it0}'\bfbeta_i + \mu_{it0}\) with no post-search component: nothing about it is learned by clicking, so its utility is resolved from the outset.

The term \(\bfL_{itj}'\bfkappa_i\) is the departure that matters most. In the match-value formulations that dominate empirical work, everything learned through search is a scalar shock \(\varepsilon_{itj}\) unobserved by the researcher both before and after search. Here the product page reveals attributes we designed, so part of what search delivers is observed, priced by \(\bfkappa_i\), and — critically — drawn from a distribution the analyst chose rather than assumed. 1 give exactly this general form in their §2.4 and note it is uncommon in practice; SIFT is a case where it is unavoidable, since recovering \(\bfkappa_i\) for product-page attributes is half of what the instrument is for. The reservation-value discussion below takes up what this costs computationally.

Search costs. Opening alternative \(j\)’s product page costs

\[ \cost_{itj} \;=\; \exp\!\left(\bfZ_{itj}'\bfgamma_i\right), \tag{2}\]

exponentiated to enforce positivity, as is standard. \(\bfZ_{itj}\) holds a respondent-level intercept and the randomized listing-page position, and may hold listing-page attributes that plausibly shift salience. Overlap between \(\bfX\) and \(\bfZ\) is permitted; Section 5 takes up what separates the two.

Where the randomness lives. Reservation values must carry a stochastic component, or search order would be a deterministic function of observables and no observed order could be rationalized (Ursu et al. 2024). Two specifications supply one. The first, used above and by most of the literature, puts a pre-search taste shock \(\mu_{itj}\) in utility. The second, due to Chung et al. (2024), drops \(\mu_{itj}\) and instead treats the search cost itself as random at the respondent-alternative level, with \(\cost_{itj}\) drawn from a distribution whose location carries the position effect. The two are observationally similar but econometrically quite different: under the first, the scale of reservation utilities and the scale of realized utilities are jointly determined by \(\sigma_\mu\) and \(\sigma_\varepsilon\), and estimating both requires an auxiliary step; under the second, search-cost dispersion fixes the scale of reservation values while \(\sigma_\varepsilon\) alone fixes the scale of realized utilities, and everything is estimable in one pass.

1 §4.4.1 object to the second on interpretive grounds, asking why the cost of clicking a product detail page would vary at the consumer-product level. The objection has least force here, because we randomize the thing that makes it vary: the same alternative sits at a different position for different respondents and in different tasks, so respondent-alternative variation in click cost is designed in rather than assumed. Section 5 takes up which specification to estimate; Chung et al. (2024) show that relative preference parameters survive getting this choice wrong, though search-cost levels do not.

Search and choice. Let \(\Sset_{it} \subseteq \Jset_{it}\) denote the set of alternatives whose product pages respondent \(i\) opened in task \(t\), \(\Sbar_{it}\) its complement, \(H_{it} = |\Sset_{it}|\) the number of clicks, and \(y_{it} \in \Sset_{it} \cup \{0\}\) the alternative purchased, with \(y_{it}=0\) recording exit without purchase. Recall is free, so \(y_{it}\) may be any member of \(\Sset_{it}\), and selection requires prior inspection, so it may not be drawn from \(\Sbar_{it}\).

Reservation values. The reservation value \(\resv_{itj}\) equates the marginal cost of opening a product page to the marginal expected benefit,

\[ \int_{\resv_{itj}}^{\infty} \left(u - \resv_{itj}\right) dF_{itj}(u) \;=\; \cost_{itj}, \tag{3}\]

where \(F_{itj}\) is the respondent’s belief about \(u_{itj}\) prior to clicking. Additive separability gives \(\resv_{itj} = \delta_{itj} + g(\cost_{itj})\), with \(g\) decreasing in search cost and depending on the belief distribution only through the post-search component.

Here Equation 1 exacts a price, though a smaller one than it first appears. When the post-search component is a single normal shock, \(g\) has the familiar closed form \(g(\cost) = m(\cost/\sigma_\varepsilon)\,\sigma_\varepsilon\) with \(m\) solving \(\cost/\sigma_\varepsilon = \phi(m) + m[\Phi(m)-1]\) (Kim et al. 2010), and this is what every existing implementation computes. With designed product-page attributes the pre-click belief about \(\bfL_{itj}'\bfkappa_i + \varepsilon_{itj}\) is instead a mixture of normals — one component per attribute-level combination the design can produce, with weights fixed by the design — and \(g\) has no scalar closed form.

Two properties of a designed instrument keep this cheap. First, the mixture is common across alternatives: a balanced design draws \(\bfL_{itj}\) from the same distribution at every position, so the respondent’s pre-click belief is identical for every alternative she has not yet opened, and the mixture enters the model only through \(g\) rather than alternative by alternative. The same balance delivers the independence across \(j\) that Weitzman’s rules require. Second, the left side of Equation 3 is then a probability-weighted sum of normal partial expectations, strictly decreasing in \(\resv_{itj}\), so one root-find recovers \(g\) for each distinct search cost — and because position is categorical, the number of distinct search costs per respondent is the number of positions, not the number of alternative-task pairs. The scalar-normal case is the one-component special case, and nothing in Weitzman’s rules requires normality, so the decision rules of Section 4.2 carry over unchanged.

A variance-matched normal approximation is also available, and it is the specification we estimate. Absorbing the design mean of \(\bfL'\bfkappa_i\) into \(\delta_{itj}\) and replacing the mixture with a normal of variance \(\tilde\sigma_i^2 = \bfkappa_i'\mathrm{Var}(\bfL)\bfkappa_i + \sigma_\varepsilon^2\) returns the model exactly to the Kim closed form with an inflated post-search standard deviation. This is more than a computational convenience. The benefit of search — the object the literature normalizes away for want of variation to estimate it (Ursu 2018; Morozov et al. 2021) — is here partly known, because \(\mathrm{Var}(\bfL)\) is fixed by the design rather than assumed. Because \(g\) then depends on its arguments only through \(\cost/\tilde\sigma_i\), a single one-dimensional inverse serves every respondent, which is what keeps estimation tractable at panel scale (Section 5.5). We report the exact mixture as a generalization and use it as a robustness check.

Parameters. Collect the respondent-level parameters in \(\bftheta_i = (\bfbeta_i, \bfkappa_i, \bfgamma_i)\) and the population parameters governing their distribution in \(\bfOmega\). We reserve \(\bftheta\) for the parameter vector throughout and write \(\log \sigma_\varepsilon\) explicitly where the log scale is meant.

4.2 Optimal Search Policy

Weitzman (1979) solves the problem posed by Equation 1 and Equation 2. What makes it tractable is that \(\resv_{itj}\) depends only on alternative \(j\)’s own parameters: it is computed once, before any searching, and never revised in light of what other alternatives turn out to be worth. Optimal behaviour is then characterized by four conditions, which we state in the notation of Section 4.1 with the task subscript carried throughout.

Write \(h = 1,\dots,H_{it}\) for the order in which product pages were opened, so that \(h=1\) denotes the first page opened, and recall that the no-purchase option \(j=0\) is available from the outset and never requires a click.

Selection. Product pages are opened in decreasing order of reservation value, so that every alternative inspected has a reservation value at least as high as every alternative left unopened,

\[ \resv_{ith} \;\ge\; \max_{l \in \Sbar_{it}} \resv_{itl} \qquad \text{for } h = 1,\dots,H_{it}. \tag{4}\]

Continuation. The respondent opens the \(h\)-th page only if its reservation value exceeds the best utility realized so far, the no-purchase option included,

\[ \resv_{ith} \;>\; \max\left\{u_{it0},\; \max_{k<h} u_{itk}\right\} \qquad \text{for } h = 1,\dots,H_{it}. \tag{5}\]

Stopping. Search ends when the best realized utility exceeds every remaining reservation value,

\[ \max\left\{u_{it0},\; \max_{h \le H_{it}} u_{ith}\right\} \;\ge\; \max_{l \in \Sbar_{it}} \resv_{itl}. \tag{6}\]

Choice. The respondent buys the best alternative among those inspected, with the no-purchase option always in contention,

\[ u_{it y_{it}} \;\ge\; \max\left\{u_{it0},\; \max_{h \le H_{it}} u_{ith}\right\}. \tag{7}\]

Two features of this system are specific to SIFT and worth drawing out.

The first click is costly. Much empirical work conditions on consumers who searched at least once, which forces the assumption that the first search is free — otherwise the sample is selected on an outcome the model is trying to explain. Because a SIFT respondent may leave a task without opening anything, and we record that, \(H_{it} = 0\) is an observed outcome rather than a truncation. Equation 5 at \(h=1\) then carries real content: it says the best reservation value on the listing page beat the no-purchase option. This is not a technicality. It means the effective-value and welfare expressions of 1 §3 remain available without the free-first-search patch, and it is one of the places where designing the instrument buys something a field study cannot easily obtain.

Continuation collapses to a single binding constraint. Equation 5 appears to impose \(H_{it}\) separate restrictions, but reservation values decrease along the search order by Equation 4, so the tightest is the last. Continuation is therefore equivalent to

\[ \max\left\{u_{it0},\; u_{it1},\; \dots,\; u_{it,H_{it}-1}\right\} \;<\; \resv_{it H_{it}}, \]

a single inequality involving the reservation value at the final click only. This is not merely tidy — it is what keeps the likelihood in Section 4.3 tractable, and overlooking it is an easy and silent implementation error.

4.3 Likelihood

The four conditions are inequalities in the unobserved shocks \(\mu_{itj}\) and \(\varepsilon_{itj}\), so the probability of an observed task outcome is the measure of the region in which all of them hold. Following 1 we write each condition as a difference required to be non-negative and index them \(\nu_1,\dots,\nu_4\) in the order selection, continuation, stopping, choice.

This labelling is not uniform in the literature: Yavorsky et al. (2021) adopts the opposite convention, with \(\nu_1\) denoting continuation and \(\nu_2\) selection. The crosswalk in Section 11.1 records both, and comparisons of smoothing parameters or fit diagnostics across the two literatures should be made with the convention in mind.

The task-level probability is

\[ \Pr\left(\Sset_{it}, y_{it} \mid \bftheta_i\right) = \int \mathbb{1}\left\{\nu_1 \ge 0,\ \nu_2 > 0,\ \nu_3 \ge 0,\ \nu_4 \ge 0\right\} \, dF(\boldsymbol\mu_{it}, \boldsymbol\varepsilon_{it}), \tag{8}\]

which has no closed form. Section 5 takes up how it is computed. Here we record only the structural fact that determines what is computationally possible.

Conditional on the observed purchase \(y_{it}\), all four conditions are a conjunction of linear inequalities. Equation 6 as written is a statement about a maximum and so in general a union of events, which is not an orthant and not directly amenable to the standard simulators. But Equation 7 names which alternative attains that maximum, and once it is named, stopping becomes \(u_{it y_{it}} \ge \resv_{itl}\) for each \(l \in \Sbar_{it}\) separately. Conditioning on the choice converts the one genuinely awkward condition into an orthant, which is what makes a GHK-type simulator available at all.

The panel multiplies. Respondent \(i\)’s contribution is

\[ \mathcal{L}_i(\bftheta_i) \;=\; \prod_{t=1}^{T} \Pr\left(\Sset_{it}, y_{it} \mid \bftheta_i\right), \tag{9}\]

and the sample likelihood integrates \(\mathcal{L}_i\) over the heterogeneity distribution governed by \(\bfOmega\). Independence across tasks is an assumption rather than a consequence, and it does real work: it is what turns \(T\) tasks into \(T\) times the information rather than one long correlated spell.

The design earns that assumption instead of asserting it. Because attribute profiles are drawn fresh in every task rather than being a fixed set of named alternatives recurring throughout (Section 3.2), a profile in task \(t+1\) genuinely is a new object, with no match value carried forward from task \(t\) to be rationalized as learning. The contrast is instructive: Morozov et al. (2021) estimate an MA(1) carryover of roughly \(0.3\) across sessions separated by weeks, and SIFT tasks are separated by minutes, so a design with recurring alternatives would face a substantially larger carryover and would have to model it. What remains is respondent-level learning and fatigue across tasks, which we test for directly rather than assume away.

4.4 Conjoint as the Zero-Search-Cost Special Case

NoteProposition 1

As \(\cost_{itj} \to 0\) for all \(j\), the model of Section 4.1 converges to a standard choice-based conjoint model over the joint attribute vector \((\bfX_{itj}, \bfL_{itj})\): the search set becomes the full choice set, \(\Sset_{it} \to \Jset_{it}\), and the choice rule reduces to \(y_{it} = \arg\max_{j \in \Jset_{it} \cup \{0\}} u_{itj}\) over fully realized utilities.

The argument follows from the structure of the reservation value. As \(\cost_{itj} \to 0\) the argument of \(g^{-1}\) goes to zero and \(\resv_{itj} \to \infty\), so every reservation value eventually exceeds every realized utility. Equation 5 then holds at every \(h\), search never stops before the listing page is exhausted, and Equation 7 becomes an unrestricted argmax over all alternatives with their post-search components resolved.

What matters is the corollary. Conjoint is not merely similar to SIFT at low search costs; it is the boundary of the SIFT parameter space. Whether a category should be studied with conjoint or with SIFT therefore has an empirical answer rather than a methodological one: estimate the search cost and test whether it is distinguishable from zero. If it is not, conjoint was correctly specified and nothing was lost by fielding the richer instrument. If it is, conjoint was misspecified for that category, and the SIFT estimates say by how much and in which direction.

The restriction is not, however, testable by conventional means. Conjoint sits at the boundary of the SIFT parameter space rather than in its interior: \(\cost \to 0\) corresponds to \(\log \cost \to -\infty\), and to \(g(\cost) \to \infty\) under the alternative parameterization. The null is therefore not an interior point, and the usual \(\chi^2\) reference distribution does not apply. The restriction must instead be assessed against observable implications or against a simulated reference distribution, as developed in Section 5.3.

4.5 Alternative Search Protocols

Sequential search is a modelling choice rather than a fact, and two alternatives deserve explicit treatment because each would change what the estimates mean.

Fixed-sample search. In the Stigler tradition the consumer commits in advance to inspecting \(n\) alternatives and then chooses the best, rather than deciding after each inspection whether to continue. The two protocols suit different environments — sequential fits click-through-and-inspect, fixed-sample fits “get three quotes” — and they are empirically distinguishable. De los Santos et al. (2012) and Honka and Chintagunta (2017) both test one against the other, and both must do so at the population level, because a single search spell per consumer carries too little information to discriminate individually.

A task panel changes that arithmetic. With \(T\) observations of the same respondent searching, the test can in principle be run respondent by respondent, and the answer may differ across people. We regard “what fraction of respondents search sequentially” as a question SIFT is unusually well placed to answer, and take it up in Section 9 rather than claim it here.

Satisficing. Caplin et al. (2011) use choice-process data to show that people frequently stop before finding the best available option — behaviour a reservation-value model will absorb into an inflated search cost rather than recognize as a different stopping rule. This is the sharpest behavioural alternative to Equation 6, and it is observationally close enough that we do not claim to adjudicate it. Our position is the one set out in Section 2.3: the model is an approximation whose predictions should be checked against holdout behaviour, and a satisficing benchmark is the right comparison to report alongside it.

5 Identification and Estimation

Search models are identified from patterns that ordinary choice data cannot produce: which alternatives a respondent opened, in what order, how many, and which of the opened alternatives was ultimately bought. This section sets out what each of those margins identifies, what SIFT’s designed variation adds to arguments that field studies must make on observational grounds, and what remains normalized rather than estimated.

Throughout, the object of interest is the pair \((\bfbeta, \bfgamma)\) — preferences and search costs — together with the post-search standard deviation \(\tilde\sigma\), which most of the literature fixes to one and which we argue must be estimated.

5.1 What the Designed Variation Buys

5.1.1 Position enters search costs, not utility

The identifying assumption underlying every position-based search-cost shifter is that a listing’s position affects what it costs to inspect an alternative, and nothing else. Ursu (2018) §4.3 takes this seriously rather than asserting it, enumerating the ways position could instead enter the model — through pre-search utility, through the post-search spread, or through combinations — and ruling them out with observational tests. Because SIFT randomizes position within every task and records the full click sequence, we can run those tests as experimental contrasts rather than conditional comparisons.

Two statistics suffice, and neither requires fitting a model.

T1. Among clicked alternatives, does position predict purchase? T2. Is the first click near the top of the page?

The sign of T1 warrants care, because it runs opposite to the natural intuition. One might reason that if position acts only on costs, it should bear on whether an alternative is opened but not on what is bought once it has been; a flat T1 would then indicate position-in-cost. Simulation shows the reverse, and the mechanism is selection. When position raises the search cost, an alternative in a poor position must have unusually high pre-search utility to be worth opening at all. Conditional on having been opened, poorly placed alternatives are therefore better on average than well-placed ones, and T1 is strongly positive. When position instead enters pre-search utility, the direct and selection effects approximately cancel and T1 is flat.

In simulations of 400 respondents completing 8 tasks each, T1 has slope \(+0.39\) (\(p \approx 5\times10^{-15}\)) when position acts on costs, and \(-0.05\) (\(p = 0.06\)) when it acts on pre-search utility. Interpreting a flat T1 as evidence for position-in-cost would invert the conclusion.

T2 separates the remaining case. If position widens the post-search spread, the reservation value rises with position, so the worst-placed alternatives would be opened first. That is what we see: mean first-click position 4.96 against a uniform benchmark of 3.5, versus 2.21 and 2.57 in the cost and utility arms.

The two tests together form a decision rule:

T2	T1	position acts on
first click near top	strongly positive	search cost
first click near top	flat	pre-search utility
first click near bottom	—	post-search spread

We regard this as a design check to be run and reported in every SIFT study rather than assumed. Details, code, and the full three-arm simulation are in the position comparison.

5.1.2 Separating preferences from search costs

The sharper problem is not whether position shifts costs but whether a high search cost can be distinguished from a low taste. Morozov et al. (2021) give the cleanest statement of the separation (their §4.3.2): stopping decisions depend on the realized utilities of alternatives already searched, but not on their search costs, which are sunk by the time the stopping decision is made. Trade the utility intercept against the search cost so as to hold the mean reservation value fixed, and the search order is unchanged — but the higher intercept raises realized utility and so makes stopping more likely.

Their Appendix D sharpens this into an observable moment: the recall rate, the probability of purchasing an alternative encountered earlier in the sequence rather than the most recent one, responds to the utility intercept but not to the search cost. SIFT observes recall directly, because respondents have free recall and we record the entire click sequence. This is our cleanest separation moment, and it is available in the raw data without any modelling.

5.1.3 What the panel buys, and what it does not

Morozov et al. (2021) prove nonparametric identification of individual-level preferences and search costs from panel search data. It is worth being precise about the strength of that result, because it is easy to overclaim. The argument assumes the number of sessions per consumer grows without bound; with between 2 and 35 sessions per consumer and a mean of 5, Morozov et al. (2021) estimate hyperparameters rather than individual parameters (their §5.1).

At the task counts SIFT can realistically field — call it \(T=10\) — we are in the same position. What the panel delivers is hierarchical posteriors in the conjoint tradition rather than nonparametric individual identification. That is the standard the conjoint audience already applies to part-worths, and it is the right standard here.

The panel also does something less obvious. Ursu (2018) reports baseline search costs above $102 and attributes the magnitude to observing a single click per search impression with no way to link a consumer’s impressions; restricting attention to impressions with two or more clicks cuts mean estimated search costs by up to 83%. Single-spell data mechanically inflates search costs. The panel is therefore a levels argument as much as a heterogeneity argument.

5.1.4 What neither the panel nor the shifter buys alone

The two ingredients address different problems, and the literature illustrates this by having one or the other but never both. Morozov et al. (2021) have the panel and still normalize the post-search standard deviation, reporting that little variation in their data speaks to it (their Appendix C). Yavorsky et al. (2021) have an exogenous search-cost shifter and no panel, and do estimate it — finding \(\tilde\sigma = 8.16\) against the conventional normalization of one, with a confidence interval that excludes one and consequences large enough to reverse the sign of one search-cost coefficient and to understate a counterfactual by a factor of three to four.

SIFT has both, and it is important not to conflate what each contributes. Simulations holding the total number of tasks fixed while varying tasks per respondent show that the panel does not improve recovery of the post-search spread; the randomized shifter does that work on its own. What the panel buys is individual-level parameters, which is a separate claim resting on separate variation.

SIFT also has something neither has: because we design the product page, part of the post-search component is fixed by the design rather than inferred.

This last point is stronger than it first appears. In a field setting the analyst never observes what inspection revealed; the post-search shock is a residual, and all that can be recovered is its variance. In SIFT the analyst wrote the product page. So for every clicked alternative the revealed attributes \(\bfL_{ijt}\) are known, and they enter realized utility as \(\bfL_{ijt}'\bfkappa\) — a mean shift, not a variance. Preferences over attributes that are only visible after a click are therefore identified in exactly the way conjoint part-worths are: from which alternative gets bought given what was shown.

The distinction is easy to lose in implementation, with consequences that are silent. The ex ante spread \(\tilde\sigma\) belongs in the reservation value, because that is computed before the click; the realized \(\bfL\) belongs in the utility, because that is evaluated after it. Use \(\tilde\sigma\) for both and \(\bfkappa\) enters the likelihood only through \(\tilde\sigma^2 = \bfkappa'\mathrm{Var}(\bfL)\bfkappa + \sigma_\xi^2\), where it trades off exactly against \(\sigma_\xi\) and is not identified at all — while every other parameter continues to recover normally, so nothing looks wrong. This is the sharpest version of the identification claim in the paper and it belongs in the introduction.

5.2 Normalization

Setting the standard deviation of the pre-search shock to one fixes the scale of utility, as in 1 §4.4.2 and Morozov et al. (2021) Appendix C. That normalization is innocuous and we adopt it.

The post-search standard deviation \(\tilde\sigma\) is a different matter. It is identified in principle by functional form — \(\tilde\sigma\) appears on both sides of \(g(\zeta) = \cost/\tilde\sigma\) — but Yavorsky et al. (2021) shows by simulation that functional-form identification alone is not enough to estimate it in practice, and that an exogenous, alternative-specific cost shifter is what makes estimation work.

The consequence of fixing it deserves to be stated without hedging. With both variances normalized, search costs cannot be monetized and search-cost counterfactuals are not interpretable (1 §4.4.4). That would rule out every friction counterfactual in Section 8. Estimating \(\tilde\sigma\) is therefore load-bearing for this paper rather than a refinement, and it is the single clearest reason SIFT needs a randomized position rather than merely a rich attribute design.

5.3 Testing the Zero-Search-Cost Restriction

SIFT nests conjoint: as the search cost goes to zero every alternative is inspected, choice is made over fully realized utilities, and the model collapses to a standard choice-based conjoint model. Whether search costs are distinguishable from zero is therefore a testable question rather than a maintained assumption — but the test requires care.

The natural test — a Wald test on the estimated search cost — is not valid here. Zero is a limit rather than an interior point of the parameter space: \(\cost \to 0\) corresponds to \(g(\cost) \to +\infty\) and to \(\log \cost \to -\infty\). The null therefore sits at a boundary under every parameterization, and the standard asymptotics do not apply.

We construct the test from observable implications instead, in three ways. The first is holdout fit: SIFT against the nested conjoint model on tasks withheld from estimation. The second is the observable implication itself — whether any respondent leaves alternatives unopened, which under zero search costs no one would. The third is a likelihood-ratio comparison against a simulated rather than \(\chi^2\) reference distribution.

This also bears on a parameterization choice. 1 §5.2 notes that estimating \(g\) directly avoids a root-find inside the optimization loop. Convenient — but it puts the quantity we most want to test on a scale where the null is at infinity. We estimate \(\log \cost\) and pay the root-find, which in our implementation is a spline evaluation rather than an iterative solve (see R/README.md).

5.4 Individual-Level Estimation from a Task Panel

There are two routes to the individual-level deliverable that makes SIFT a conjoint substitute rather than a search paper.

Full hierarchical Bayes is what the conjoint audience expects and what makes individual part-worths and individual search costs directly reportable.

Two-stage empirical Bayes estimates the hyperparameters by simulated maximum likelihood and then draws individual posteriors by Metropolis–Hastings conditional on those estimates. This is what Morozov et al. (2021) do to compute personalized prices (their §7.2 and Appendix H), and it is considerably cheaper.

Our runtime work reverses the ordering we originally expected. Because HB never integrates over the heterogeneity distribution — each respondent sits at their own \(\bftheta_i\) — the mixing-draw multiplier that makes random-coefficients SMLE expensive simply does not arise. We therefore treat HB as the primary estimator and empirical Bayes as the robustness check.

Runtime should be stated precisely, since the sampler’s cost is easy to underestimate from per-evaluation timings alone. The implemented sampler measures 1.8 ms per task per iteration — linear in the number of tasks, and only weakly increasing in the number of simulation draws, because per-task overhead rather than draw count dominates. Each iteration makes two passes over the data and carries per-respondent overhead in addition. For 800 respondents completing 10 tasks each, at 20,000 iterations, this amounts to approximately 80 hours single-threaded.

The \(\bftheta_i\) step is, however, embarrassingly parallel: each respondent’s Metropolis update touches only that respondent’s tasks. Parallelised across 12 cores we measure a 7.5-fold speedup, which is conservative in the sense that communication accounts for a material share of each iteration at that problem size. A commercial-scale fit then completes in approximately 11 hours.

The claim is therefore conditional. Hierarchical Bayes is the cheaper route to heterogeneity and completes within an overnight budget, but only when the respondent step is parallelised; a single-threaded implementation does not.

5.5 Estimation

Runtime is a binding practical constraint rather than an implementation detail: a fit has to complete overnight, not over a week. The published timings make the point. 1 Table 2, for 1,000 consumers with heterogeneity, report 5h20m for importance sampling, 29h48m for the kernel-smoothed simulator plus roughly 32 further hours to calibrate its scaling vector, and 198h24m for GHK. Chung et al. (2024) Table 1, for a cross-sectional design of comparable size, reports 0.48 minutes for their probability-mapping simulator against 5–7 minutes kernel-smoothed and 15–127 minutes for crude frequency — one to two orders of magnitude.

We have ported and validated the Chung et al. (2024) estimator and reproduced their Table 1; the port, its five validation checks, and two bugs we found in their shipped code are documented in R/README.md.

For SIFT itself we exploit a structural feature of the design. Conditional on the observed purchase, every Weitzman condition is a conjunction of linear inequalities — the stopping rule is a statement about a maximum and therefore a union in general, but the choice rule names which alternative attains that maximum. Continuation then collapses to the single binding constraint \(\resv_{s_K}\), because reservation values decrease along the search order. And conditional on \(\bfeta\) and on the post-search shock to the chosen alternative, the remaining shocks are independent one-sided constraints that integrate out in closed form, leaving a one-dimensional integral.

This yields a hybrid estimator: GHK on the pre-search block, quadrature on the post-search block. Measured against the kernel-smoothed accept–reject simulator of Yavorsky et al. (2021) at equal draw counts, its simulation variance is lower by a factor of 21 to 100 depending on the design. Because runtime is exactly linear in the number of draws, the relevant summary is time to a given precision, on which the hybrid is 3 to 21 times cheaper; crude frequency is not competitive on either margin. The kernel-smoothed simulator also shows the upward bias in the negative log-likelihood that Yavorsky et al. (2021) notes is intrinsic to it. The construction and its validation are in notebooks/sift_likelihood.qmd.

5.6 Software

All estimation code is in R and is released with the paper. The reservation-value inverse \(g^{-1}\) is evaluated by a cubic spline built once in the standardized argument \(\cost/\tilde\sigma\), so a single spline serves every \(\tilde\sigma\); it covers costs down to \(3.9\times10^{-16}\), well below the range any optimizer will propose.

6 Simulation Study

6.1 Design

6.2 Parameter Recovery

6.3 Power to Detect a Nonzero Search Cost

6.4 Misspecification and Behavioral Robustness

7 Empirical Application

7.1 Data and Design

7.2 Descriptive Evidence

7.3 Estimates

7.4 Convergent Validity against Conjoint

7.5 Holdout Prediction

7.6 Task-Order Effects

8 What the Model Delivers

A method earns its cost in what it lets a decision-maker do differently. This section sets out the quantities SIFT produces that conjoint cannot, and what each supports. The quantities themselves follow from the model of Section 4 and are computable from any fitted SIFT study; the magnitudes await the empirical application in Section 7, and we are explicit throughout about which is which.

8.1 Decomposing Non-Purchase

The single most useful thing SIFT produces is a decomposition that conjoint cannot represent. When an alternative fails to sell, the model separates two outcomes that look identical in choice data:

• Never inspected. The alternative’s reservation value never rose above the best realized utility, so the respondent never opened it. Whatever its merits, they were never evaluated.
• Inspected and rejected. The respondent opened it, learned \(\bfL_{itj}'\bfkappa_i + \varepsilon_{itj}\), and preferred something else.

Both appear in a conjoint study as “low share.” They call for opposite responses. A never-inspected problem is a problem of placement, distribution, salience, and media — the alternative needs to be seen. An inspected-and-rejected problem is a problem of product, price, or positioning — the alternative was seen and found wanting. Spending on media to fix an inspected-and-rejected problem buys more inspections of something people are already declining; reformulating the product to fix a never-inspected problem improves something nobody looked at.

For each alternative the fitted model returns the probability of inspection, the probability of purchase conditional on inspection, and their product. The first is a consideration measure derived from behaviour rather than from a survey question about consideration, and the second is a preference measure purged of the consideration stage. Reporting them separately, by respondent segment, is what we mean by the decomposition.

We expect — and this is a hypothesis the empirical application will test, not a result — that categories differ substantially in which margin binds, and that the categories where consideration binds hardest are the ones where conjoint has been quietly misleading for longest.

8.2 Position and Merchandising Counterfactuals

Because position enters the search cost (Equation 2) and is randomized by design, the model supports counterfactuals that reposition alternatives on the listing page. Moving an alternative from position six to position two lowers its search cost, raises its reservation value, moves it earlier in the search order, and raises the probability it is inspected at all. The model propagates that through to share.

Conjoint cannot pose this question. In a conjoint task every alternative is equally and costlessly visible, so position either does not exist or is a attribute like any other — and treating it as an attribute prices it as a preference for being listed higher, which is not what is happening.

This is the class of question live for marketplaces, retail media networks, app stores, streaming carousels, and category managers setting shelf sequence. It is also the class where the value of a paid placement can be quantified against the value of the product change that would achieve the same share.

One caution belongs here rather than in the limitations. The counterfactual is credible only over the range of positions the design actually varied. Our simulations show that a listing page with too steep a position penalty stops being informative about the bottom of the page, because the bottom is never opened (Section 6). Designing enough friction to identify the effect, but not so much that the tail goes dark, is a design decision with a measurable optimum.

8.3 Information Architecture

SIFT distinguishes attributes shown before a click (\(\bfX\), priced by \(\bfbeta_i\)) from attributes revealed after one (\(\bfL\), priced by \(\bfkappa_i\)). Both are recovered. That makes the placement of information itself a decision variable: what belongs on the listing tile, and what belongs on the product page?

The tension is that listing-page attributes influence whether an alternative is inspected, while product-page attributes influence whether it is bought once inspected. An attribute with a large \(\bfkappa\) but no presence on the listing page is doing no work in the consideration stage — it is a strength nobody learns about until they have already decided to look. Promoting it to the listing page should raise inspection; demoting a weak attribute should do the same by removing a reason not to look.

We can compute these reallocations. What we cannot yet do is validate them against a held-out design in which the architecture was actually changed, and we regard that as the natural second study rather than something to assert from the first.

8.4 Pricing Friction

The search cost is estimated in utility units and, given a price coefficient, convertible to money. That makes a family of interventions comparable on a single scale for the first time: reducing the number of clicks to a decision, improving filters and sorts, setting better defaults, and shortening a product page are all interventions on \(\cost_{itj}\).

The managerially useful form is a ratio rather than a level. “Moving from position six to position two is worth as much as a 4% price reduction” is a statement a category manager can act on, and — importantly for us — ratios are considerably more robust than levels to the finite-sample bias documented in Section 6, because the bias affects the search-cost scale rather than relative comparisons within it. We say more about this in Section 9, and we recommend that SIFT studies report ratios as the headline and levels with appropriate caution.

8.5 Search-Based Segmentation

Because \(\bfgamma_i\) is estimated at the respondent level alongside \(\bfbeta_i\), SIFT segments on how people shop as well as on what they prefer. A high-search-cost segment and a low-search-cost segment with identical preferences will behave very differently in the same environment: the first buys from a short consideration set assembled cheaply, the second explores and is more responsive to what is discoverable deeper in the page.

This is a different segmentation basis from the one conjoint supplies, and it maps onto different interventions — the low-search-cost segment is reachable through depth and discovery, the high-search-cost segment through prominence and defaults. Whether these segments are stable, and whether they correlate with the demographics clients already target on, is an empirical question we cannot answer from simulation.

8.6 Large Attribute Spaces

Conjoint degrades as attributes accumulate. Respondent burden rises, number-of-levels effects intrude, and simplification heuristics contaminate the tradeoffs the design exists to measure — with the result that practitioners cap attribute counts well below what many categories actually involve.

SIFT distributes the attribute space across two pages and lets the respondent choose what to examine. Crucially, a respondent who never opens an alternative is not a respondent who failed the task: non-attention is an outcome the model explains rather than noise the design must suppress. Whether this genuinely extends the workable attribute count, and by how much, is among the most practically valuable questions the method raises and among the least settled. We flag it as a claim to be tested rather than one we make.

9 Discussion

9.1 What the Nesting Result Changes

The proposition in Section 4.4 does more work than a technical observation usually does, and it is worth separating what it settles from what it does not.

It settles the framing. SIFT does not compete with conjoint; it contains it. A practitioner who has bought conjoint for a decade is not being told they were wrong, and a category in which search costs turn out to be negligible is a category in which conjoint was the right tool and remains so. The nesting makes that a finding rather than a concession.

It also converts a methodological argument into an empirical one. “Should this study be a conjoint or a SIFT” is answerable from SIFT data and not from conjoint data. The asymmetry is a real one: the richer instrument can diagnose whether the simpler one was adequate, but not the reverse. The cost of finding out is the cost of fielding SIFT.

What it does not settle is whether the test is easy. As Section 5.3 develops, the null sits at the boundary of the parameter space, so the comfortable \(\chi^2\) machinery does not apply and the test has to be built from observable implications or a simulated reference distribution. We regard the sharpest of those implications as the simplest: under zero search costs, nobody leaves alternatives unopened. That is checkable before any model is fit.

9.2 Do Survey Clicks Measure Search?

Clicking in a survey is nearly free. It costs a second of attention and no money, whereas inspecting a product in a real category may cost a drive across town, a phone call, or an hour of reading. There is therefore no reason to expect the estimated level of search cost from a SIFT study to transfer to the field, and we do not claim that it does.

Several mitigations are available and we regard them as design parameters rather than fixes. Incentive alignment in the tradition of Ding et al. (2005) and Ding (2007) raises the stakes of the choice itself. Artificial latency on the product page makes a click cost time. An explicit click budget makes it cost an alternative opportunity. Denominating cost in time rather than utility makes the units comparable across studies even if not across contexts. Each of these changes what the estimated search cost means, and a study should say which it used.

The standing defence, and we think it is the right one, is that relative and comparative statements transfer even when levels do not. This is precisely the defence conjoint has made for four decades about utilities that are not interpretable in isolation. It is defensible ground, but it constrains the claims a SIFT study should make. Section 8.4 accordingly recommends ratios as the headline quantity.

9.3 Rational Expectations About the Unopened

Weitzman’s solution requires the respondent to know the distribution of what a click will reveal. At the start of a survey she does not: she has never seen this category rendered this way, with these attributes at these levels.

Two responses are available. A practice block, unscored, lets the respondent learn the environment before the data collection begins — the same logic that motivates warm-up tasks in conjoint. Alternatively, learning can be modelled across tasks, at the cost of abandoning the independence assumption in Equation 9 that makes the panel multiply.

We take the first route and treat the second as a robustness check. The tension is that a practice block assumes learning completes quickly, and whether it does is testable by looking for trends in click counts and search costs across task order. Section 7 reports that diagnostic. If search costs decline systematically across tasks, the practice block was insufficient and the learning model becomes necessary rather than optional.

9.4 Do Respondents Follow the Reservation Rule?

The experimental record reviewed in Section 2.3 is mixed and the deviations are systematic: too little search relative to the optimum, order effects, and sensitivity to sunk costs the theory says should be ignored. Caplin et al. (2011) in particular show people stopping before they have found the best available option, which a reservation-value model will absorb as an inflated search cost rather than recognize as a different rule.

This is the deepest scientific risk in the approach, and it is worth being clear that it is not our risk alone: it applies with equal force to the entire structural search literature SIFT builds on, which has estimated Weitzman models on field data for two decades without resolving it.

Our position has three parts. First, the model should be judged as an approximation that recovers useful quantities, not as a claim about cognition. Second, its predictions should be checked against holdout behaviour rather than assumed — a SIFT study can hold out tasks and ask whether the fitted model predicts click counts, click order, and purchase in data it did not see. Third, a satisficing benchmark should be reported alongside, so a reader can see how much of the fit is doing work the simpler rule would have done anyway. Where the model loses to satisficing, the honest conclusion is that search costs are not identified in that category rather than that they are small.

9.5 Precision, Runtime, and What They Constrain

Two practical constraints shape what a SIFT study can currently claim.

Precision on search-cost levels. Preference parameters recover cleanly at commercial sample sizes. Search-cost levels carry a finite-sample bias that declines slowly in both sample size and simulation draws (Section 6). This is a known property of simulated maximum likelihood estimation of search models rather than a feature of our instrument — Yavorsky et al. (2021) reports the same of his own simulations, citing Honka (2014) and Ursu (2018) — but it means conclusions about search-cost magnitude demand more data than conclusions about preference, and a study should size itself against whichever claim it intends to make.

Runtime. A hierarchical Bayes conjoint model fits in minutes. A comparably sized SIFT model takes hours: our implementation measures roughly eleven hours for a commercial-scale fit, and only when the sampler is parallelized across respondents. We regard this as an engineering constraint rather than a fundamental one, and the trajectory is favourable — Chung et al. (2024) report speedups of one to two orders of magnitude over the kernel-smoothed simulators that dominated the previous decade. But it constrains iteration today, and iteration is how instruments improve.

9.6 Where Conjoint Remains the Right Tool

Conjoint remains the better instrument for feature-level willingness-to-pay on a defined bundle, where there is nothing to search over and the question is purely one of tradeoffs. It remains better for new-product configuration when the alternative does not yet exist in any market and no listing page could be constructed for it. It remains better when an orthogonal design is needed for clean, low-variance part-worths, since SIFT deliberately lets respondents choose what to examine and therefore gives up balance. It remains better where a mature share simulator is the deliverable and the client’s decision process is built around it. And it remains better in any study where established norms, benchmarks, and comparability to prior waves matter more than novelty.

Conjoint is also understood, defensible, cheap to estimate, and supported by four decades of accumulated practice. SIFT is none of those things yet. A method that is correct in principle but unfamiliar in practice imposes a real cost on the organization adopting it, and that cost is part of the comparison.

9.7 Extensions

Several extensions follow directly from the model and are, we think, more tractable than they appear.

Filters and sorts as search actions. Chen and Yao (2017) treat refinement as part of the search process rather than as a given. In a SIFT instrument the filter and sort controls are ours to design, so refinement could be observed and priced alongside inspection.

Discovery of alternatives. Greminger (2022) relaxes the assumption that the choice set is known at the outset. A listing page that paginates or lazily loads is exactly that setting, and it is a small change to the instrument.

Sequential versus fixed-sample, respondent by respondent. As Section 4.5 notes, the task panel makes this test available at the individual level, where the literature has been able to run it only at the population level. The distribution of search protocols across respondents is therefore identifiable in principle and, to our knowledge, has not been estimated.

Attribute-level search. Our model has respondents searching across alternatives; the Mouselab literature has subjects searching across attributes of displayed alternatives. Both happen in real shopping. A model that permits either would be a genuine extension rather than a variation, and the instrument already records enough to identify which is occurring.

10 Conclusion

References

Ackerberg, Daniel A. 2009. “A New Use of Importance Sampling to Reduce Computational Burden in Simulation Estimation.” Quantitative Marketing and Economics 7 (4): 343–76. https://doi.org/10.1007/s11129-009-9074-z.

Allenby, Greg M., Nino Hardt, and Peter E. Rossi. 2019. “Economic Foundations of Conjoint Analysis.” In Handbook of the Economics of Marketing. Elsevier. https://doi.org/10.1016/bs.hem.2019.04.002.

Bronnenberg, Bart J., Jun B. Kim, and Carl F. Mela. 2016. “Zooming in on Choice: How Do Consumers Search for Cameras Online?” Marketing Science 35 (5): 693–712. https://doi.org/10.1287/mksc.2016.0977.

Brown, Meta, Christopher J Flinn, and Andrew Schotter. 2011. “Real-Time Search in the Laboratory and the Market.” American Economic Review 101 (2): 948–74. https://doi.org/10.1257/aer.101.2.948.

Caplin, Andrew, Mark Dean, and Daniel Martin. 2011. “Search and Satisficing.” American Economic Review 101 (7): 2899–922. https://doi.org/10.1257/aer.101.7.2899.

Chen, Yuxin, and Song Yao. 2017. “Sequential Search with Refinement: Model and Application with Click-Stream Data.” Management Science 63 (12): 4345–65. https://doi.org/10.1287/mnsc.2016.2557.

Chung, Jae Hyen, Pradeep Chintagunta, and Sanjog Misra. 2024. “Simulated Maximum Likelihood Estimation of the Sequential Search Model.” Quantitative Marketing and Economics 23 (1): 105–64. https://doi.org/10.1007/s11129-024-09281-4.

Cox, JamesC., and RonaldL. Oaxaca. 1989. “Laboratory Experiments with a Finite-Horizon Job-Search Model.” Journal of Risk and Uncertainty 2 (3). https://doi.org/10.1007/bf00209391.

De los Santos, Babur, Ali Hortaçsu, and Matthijs R Wildenbeest. 2012. “Testing Models of Consumer Search Using Data on Web Browsing and Purchasing Behavior.” American Economic Review 102 (6): 2955–80. https://doi.org/10.1257/aer.102.6.2955.

Ding, Min. 2007. “An Incentive-Aligned Mechanism for Conjoint Analysis.” Journal of Marketing Research 44 (2): 214–23. https://doi.org/10.1509/jmkr.44.2.214.

Ding, Min, Rajdeep Grewal, and John Liechty. 2005. “Incentive-Aligned Conjoint Analysis.” Journal of Marketing Research 42 (1): 67–82. https://doi.org/10.1509/jmkr.42.1.67.56890.

Dzyabura, Daria, and John R. Hauser. 2011. “Active Machine Learning for Consideration Heuristics.” Marketing Science 30 (5): 801–19. https://doi.org/10.1287/mksc.1110.0660.

Gabaix, Xavier, David Laibson, Guillermo Moloche, and Stephen Weinberg. 2006. “Costly Information Acquisition: Experimental Analysis of a Boundedly Rational Model.” American Economic Review 96 (4): 1043–68. https://doi.org/10.1257/aer.96.4.1043.

Gilbride, Timothy J., and Greg M. Allenby. 2004. “A Choice Model with Conjunctive, Disjunctive, and Compensatory Screening Rules.” Marketing Science 23 (3): 391–406. https://doi.org/10.1287/mksc.1030.0032.

Greminger, Rafael P. 2022. “Optimal Search and Discovery.” Management Science 68 (5): 3904–24. https://doi.org/10.1287/mnsc.2021.4085.

Hauser, John R. 2014. “Consideration-Set Heuristics.” Journal of Business Research 67 (8): 1688–99. https://doi.org/10.1016/j.jbusres.2014.02.015.

Hauser, John R., Olivier Toubia, Theodoros Evgeniou, Rene Befurt, and Daria Dzyabura. 2010. “Disjunctions of Conjunctions, Cognitive Simplicity, and Consideration Sets.” Journal of Marketing Research 47 (3): 485–96. https://doi.org/10.1509/jmkr.47.3.485.

Hey, John D. 1987. “Still Searching.” Journal of Economic Behavior &Amp; Organization 8 (1): 137–44. https://doi.org/10.1016/0167-2681(87)90026-6.

Hong, Han, and Matthew Shum. 2006. “Using Price Distributions to Estimate Search Costs.” The RAND Journal of Economics 37 (2): 257–75. https://doi.org/10.1111/j.1756-2171.2006.tb00015.x.

Honka, Elisabeth. 2014. “Quantifying Search and Switching Costs in the US Auto Insurance Industry.” The RAND Journal of Economics 45 (4): 847–84. https://doi.org/10.1111/1756-2171.12073.

Honka, Elisabeth, and Pradeep Chintagunta. 2017. “Simultaneous or Sequential? Search Strategies in the u.s. Auto Insurance Industry.” Marketing Science 36 (1): 21–42. https://doi.org/10.1287/mksc.2016.0995.

Hortacsu, A., and C. Syverson. 2004. “Product Differentiation, Search Costs, and Competition in the Mutual Fund Industry: A Case Study of s&amp;p 500 Index Funds.” The Quarterly Journal of Economics 119 (2): 403–56. https://doi.org/10.1162/0033553041382184.

Kim, Jun B., Paulo Albuquerque, and Bart J. Bronnenberg. 2010. “Online Demand Under Limited Consumer Search.” Marketing Science 29 (6): 1001–23. https://doi.org/10.1287/mksc.1100.0574.

Koulayev, Sergei. 2014. “Search for Differentiated Products: Identification and Estimation.” The RAND Journal of Economics 45 (3): 553–75. https://doi.org/10.1111/1756-2171.12062.

Marshall, Pablo, and Eric T Bradlow. 2002. “A Unified Approach to Conjoint Analysis Models.” Journal of the American Statistical Association 97 (459): 674–82. https://doi.org/10.1198/016214502388618410.

McCall, J. J. 1970. “Economics of Information and Job Search.” The Quarterly Journal of Economics 84 (1): 113. https://doi.org/10.2307/1879403.

Meißner, Martin, Andres Musalem, and Joel Huber. 2016. “Eye Tracking Reveals Processes That Enable Conjoint Choices to Become Increasingly Efficient with Practice.” Journal of Marketing Research 53 (1): 1–17. https://doi.org/10.1509/jmr.13.0467.

Morozov, Ilya. 2023. “Measuring Benefits from New Products in Markets with Information Frictions.” Management Science 69 (11): 6988–7008. https://doi.org/10.1287/mnsc.2023.4729.

Morozov, Ilya, Stephan Seiler, Xiaojing Dong, and Liwen Hou. 2021. “Estimation of Preference Heterogeneity in Markets with Costly Search.” Marketing Science 40 (5): 871–99. https://doi.org/10.1287/mksc.2021.1287.

Payne, John W., James R. Bettman, and Eric J. Johnson. 1988. “Adaptive Strategy Selection in Decision Making.” Journal of Experimental Psychology: Learning, Memory, and Cognition 14 (3): 534–52. https://doi.org/10.1037/0278-7393.14.3.534.

Payne, John W., James R. Bettman, and Eric J. Johnson. 1993. The Adaptive Decision Maker. Cambridge University Press. https://doi.org/10.1017/cbo9781139173933.

Reutskaja, Elena, Rosemarie Nagel, Colin F Camerer, and Antonio Rangel. 2011. “Search Dynamics in Consumer Choice Under Time Pressure: An Eye-Tracking Study.” American Economic Review 101 (2): 900–926. https://doi.org/10.1257/aer.101.2.900.

SCHOTTER, ANDREW, and YALE M. BRAUNSTEIN. 1981. “Economic Search: An Experimental Study.” Economic Inquiry 19 (1): 1–25. https://doi.org/10.1111/j.1465-7295.1981.tb00600.x.

Sonnemans, Joep. 1998. “Strategies of Search.” Journal of Economic Behavior &Amp; Organization 35 (3): 309–32. https://doi.org/10.1016/s0167-2681(98)00051-1.

Stigler, George J. 1961. “The Economics of Information.” Journal of Political Economy 69 (3): 213–25. https://doi.org/10.1086/258464.

Ursu, Raluca M. 2018. “The Power of Rankings: Quantifying the Effect of Rankings on Online Consumer Search and Purchase Decisions.” Marketing Science 37 (4): 530–52. https://doi.org/10.1287/mksc.2017.1072.

Ursu, Raluca M., Qingliang Wang, and Pradeep K. Chintagunta. 2020. “Search Duration.” Marketing Science 39 (5): 849–71. https://doi.org/10.1287/mksc.2020.1225.

Ursu, Raluca M., Qianyun Zhang, and Elisabeth Honka. 2023. “Search Gaps and Consumer Fatigue.” Marketing Science 42 (1): 110–36. https://doi.org/10.1287/mksc.2022.1359.

Ursu, Raluca, Stephan Seiler, and Elisabeth Honka. 2024. “The Sequential Search Model: A Framework for Empirical Research.” Quantitative Marketing and Economics 23 (1): 165–213. https://doi.org/10.1007/s11129-024-09291-2.

Weitzman, Martin L. 1979. “Optimal Search for the Best Alternative.” Econometrica 47 (3): 641–54. https://doi.org/10.2307/1910412.

Yang, Liu (Cathy), Olivier Toubia, and Martijn G. De Jong. 2015. “A Bounded Rationality Model of Information Search and Choice in Preference Measurement.” Journal of Marketing Research 52 (2): 166–83. https://doi.org/10.1509/jmr.13.0288.

Yang, Liu (Cathy), Olivier Toubia, and Martijn G. de Jong. 2018. “Attention, Information Processing, and Choice in Incentive-Aligned Choice Experiments.” Journal of Marketing Research, ahead of print. https://doi.org/10.1509/jmr.15.0474.

Yavorsky, Dan, Elisabeth Honka, and Keith Chen. 2021. “Consumer Search in the U.S. Auto Industry: The Role of Dealership Visits.” Quantitative Marketing and Economics 19 (1): 1–52. https://doi.org/10.1007/s11129-020-09229-4.

Yee, Michael, Ely Dahan, John R. Hauser, and James Orlin. 2007. “Greedoid-Based Noncompensatory Inference.” Marketing Science 26 (4): 532–49. https://doi.org/10.1287/mksc.1060.0213.

11 Appendix

11.1 Notation Crosswalk

The two papers this one builds on use different symbols for the same objects, and in two places the same symbol for different objects. The table records the mapping, both for readers arriving from either literature and to keep our own implementation honest against the two reference codebases.

Table 1: Notation crosswalk

Object	This paper	1	Yavorsky et al. (2021)
Utility	\(u_{itj}\)	\(u_{ij}\)	\(u_{ij}\)
Pre-search utility	\(\delta_{itj}\)	\(\delta_{ij}\)	\(\delta_{ij}\)
Observed pre-search utility	\(\xi_{itj} = \bfX_{itj}'\bfbeta_i\)	\(\xi_{ij} = X_j'\bfbeta_i - \alpha_i p_{ij}\)	\(x_j'\bfbeta\)
Pre-search taste shock	\(\mu_{itj}\)	\(\mu_{ij}\)	\(\eta_{ij}\)
Post-search taste shock	\(\varepsilon_{itj}\)	\(\varepsilon_{ij}\)	\(\varepsilon_{ij}\)
Observed post-search utility	\(\bfL_{itj}'\bfkappa_i\)	\(L_j'\kappa_i\) (§2.4 only)	—
Post-search s.d.	\(\sigma_\varepsilon\)	\(\sigma_\varepsilon\)	\(\sigma\) (“MVSD”)
Search cost	\(\cost_{itj}\)	\(c_{ij} = Z_j'\gamma_i\)	\(c_{ij} = \exp\{\gamma_0 + d_{ij}'\gamma\}\)
Reservation value	\(\resv_{itj}\)	\(z_{ij}\)	\(z_{ij}\)
Reservation offset	\(g(\cost)\)	\(g(c)\), \(m(c/\sigma_\varepsilon)\)	\(\zeta_{ij}\)
Searched set	\(\Sset_{it}\)	\(S_i\)	(implicit)
Unsearched set	\(\Sbar_{it}\)	\(\bar S_i\)	(implicit)
Number of searches	\(H_{it}\)	\(H_i\)	\(K_i\)
Search-order index	\(h\)	\(h\)	(folded into \(j\))
Purchase	\(y_{it}\)	\(y_i\)	\(j^*\)
Parameter vector	\(\bftheta_i\)	\(\theta\)	— (\(\theta = \log\sigma\))
Population parameters	\(\bfOmega\)	\(\Omega\)	—
Smoothing constants	\(\rho_k\)	\(\rho_k\)	\(\lambda_k\)

Four collisions are worth naming explicitly, because each is a live source of error when porting code or reading the two papers side by side.

1. The differenced conditions \(\nu_1\) and \(\nu_2\) are swapped between the two papers. 1 label \(\nu_{1h}\) the selection (order) condition and \(\nu_{2h}\) the continuation condition; Yavorsky et al. (2021) label \(\nu_{1}\) continuation and \(\nu_{2}\) selection. Smoothing constants therefore do not correspond element-wise across the two implementations. We follow
2. \(\theta\) denotes different objects. 1 use it for the full parameter vector; Yavorsky et al. (2021) use it specifically for \(\log \sigma\). We follow the former.
3. \(\zeta\) and \(\xi\) are visually near-identical in print and denote unrelated objects across the two papers — the reservation-value offset in one, observed pre-search utility in the other. We retain \(\xi\) and write the offset as \(g(\cdot)\).
4. \(d\) is a simulation-draw index in one and a distance vector in the other. We use it for neither.

11.2 Additional Appendices

11.3 Annotated Bibliography

The main text (Section 2) discusses the handful of papers SIFT actually builds on. This appendix keeps the fuller map assembled while scoping the project: six literatures, with a note on what each contributes and where it stops short of what a designed instrument needs. It is a reading guide rather than an argument, and nothing in the main text depends on it.

All entries have been verified against the published record.

11.3.1 Optimal Sequential Search

The theoretical backbone. The instrument’s data-generating process is the Weitzman protocol run inside a survey.

• Weitzman (1979), “Optimal Search for the Best Alternative,” Econometrica. [H] The Pandora’s box problem, and the model the instrument is built to estimate. Each box has a known prize distribution and a known opening cost. The optimal policy factors into three parts: a reservation value computed for each box independently of every other box; a selection rule (open boxes in descending reservation value); and a stopping rule (stop when the best realized prize exceeds the highest remaining reservation value). The independence of the reservation values is what makes the model estimable at survey scale.
• Stigler (1961), “The Economics of Information,” Journal of Political Economy. [H] The fixed-sample-size alternative: the consumer commits to \(n\) searches up front rather than deciding adaptively. The competing protocol, and the one SIFT’s panel structure may be able to test against sequential search respondent by respondent.
• McCall (1970), Quarterly Journal of Economics. [M] Reservation-wage formulation of sequential search with recall; the labor-search ancestor of the marketing applications below.

Simultaneous versus sequential. These are two competing protocols, not two descriptions of one thing. Sequential search fits click-through-and-inspect; fixed-sample search fits “get three quotes.” Honka and Chintagunta (2017) is the reference on distinguishing them empirically, and is directly relevant here because SIFT’s within-respondent panel could run that test at the individual level rather than the population level.

11.3.2 Structural Search with Field and Clickstream Data

Context rather than precedent: all of this work is observational or platform data with no designed instrument. SIFT’s credibility with an economics audience depends on connecting to it cleanly.

• Hong and Shum (2006), RAND Journal of Economics. [H] Recovers search costs from price distributions alone, without search data. The extreme case of how little the field has typically had to work with.
• Hortacsu and Syverson (2004), Quarterly Journal of Economics. [H] S&P 500 index funds. Search frictions with differentiated products, in a setting where the products are nearly identical and the frictions are not.
• Kim et al. (2010), Marketing Science. [H] Amazon view-rank data for camcorders. One of the first structural search models fit to online browsing data in marketing.
• De los Santos et al. (2012), “Testing Models of Consumer Search Using Data on Web Browsing,” American Economic Review. [H] Comscore browsing data; tests sequential search against fixed-sample search and finds fixed-sample fits better in their setting. The empirical warning shot for assuming the Weitzman protocol rather than testing it.
• Koulayev (2014), RAND Journal of Economics. [M] Hotel search with click data; identification and estimation for differentiated products.
• Honka (2014), RAND Journal of Economics. [H] Auto insurance; separates search costs from switching costs. The template for arguing that an estimated friction is the friction you claim it is.
• Honka and Chintagunta (2017), Marketing Science. [M] Simultaneous or sequential; see above.
• Chen and Yao (2017), Management Science. [M] Sequential search with refinement — filters and sorts modeled as search actions — on Expedia clickstream data. Closest existing work in spirit to a list-screen interface, and the natural precedent if the instrument later adds filtering.
• Ursu (2018), “The Power of Rankings,” Marketing Science. [H] Expedia’s randomized-position experiment. The cleanest source of exogenous variation in the field literature, obtained once, as a rare gift from a platform. The contrast with SIFT is direct: here the same variation is free, designed in, and present in every task.
• Ursu et al. (2020), Marketing Science. [M] Search duration and time spent, rather than search incidence alone.
• Yavorsky et al. (2021), Quantitative Marketing and Economics. [H] Dealership visits in the U.S. auto market; identification of search costs from observed search sets. (Author’s own; the estimation code is an implementation asset, see Section 5.)
• Bronnenberg et al. (2016), “Zooming In on Choice,” Journal of Marketing Research. [M] Descriptive clickstream evidence on how consumers actually move through product pages. Face-validity evidence for the interface design in
• 1, Quantitative Marketing and Economics. [M] — novelty gate Framework and consolidation review of the sequential search model. Best single entry point to the current state of model specification, identification, and estimation practice; read first to establish what is already settled.
• Greminger (2022), Management Science. [M] Search with discovery of alternatives — the consumer does not know the full choice set at the outset. Relevant to how much of the list-screen the instrument reveals up front.

11.3.3 Lab and Experimental Evidence on Search

Two experimental literatures that have barely spoken to each other, and the direct methodological precedent for running search as a designed task.

11.3.4 Does anyone follow the optimal rule?

• SCHOTTER and BRAUNSTEIN (1981), Economic Inquiry. [M] Early sequential-search experiments with real incentives.
• Hey (1987), “Still Searching,” Journal of Economic Behavior and Organization. [M] Subjects use rules of thumb rather than optimal reservation strategies.
• Cox and Oaxaca (1989). [M] Finite-horizon search; broadly supportive of reservation-rule behavior with noise. The more favorable read.
• Sonnemans (1998). [M] Verbal-protocol evidence and classification of search strategies.
• Brown et al. (2011). [L] Real-time search in the laboratory and in the market.

Net read. Mixed, and this is the single largest scientific risk to the project. Behavior is directionally consistent with reservation-value logic, but the deviations are systematic rather than random: too little search relative to the optimum, order effects, and sensitivity to sunk costs. Section 2.3 and Section 9.4 take up what follows for the model.

11.3.5 Information acquisition: Mouselab and eye-tracking

Mechanically, the SIFT task is a Mouselab information board wearing a commercial skin.

• Payne et al. (1988); Payne et al. (1993), The Adaptive Decision Maker. [H] The Mouselab paradigm: information boards in which subjects click cells to reveal attribute values, with the acquisition sequence recorded. Descriptive and process-focused rather than structural.
• Gabaix et al. (2006), “Costly Information Acquisition,” American Economic Review. [H] — novelty gate A Mouselab-style experiment paired with a structural boundedly-rational model (directed cognition) estimated on the acquisition data. The closest existing template for the core maneuver — run an information-acquisition experiment, fit a structural search-type model to the acquisition record. Not Weitzman, but the same move.
• Caplin et al. (2011), “Search and Satisficing,” American Economic Review. [H] Choice-process data showing that people stop before finding the best available option. The satisficing alternative against which model fit should be reported.
• Reutskaja et al. (2011), American Economic Review. [H] Eye-tracking evidence on search under time pressure with supermarket-style displays.

Net read. The experimental apparatus is well validated and has been for decades. What has largely not been done is fitting a Weitzman model to acquisition data and reporting search costs in interpretable units.

11.3.6 Information Search inside Preference Measurement

The closest existing work in marketing, and the location of the sharpest threat to novelty.

• Yang et al. (2015), “A Bounded Rationality Model of Information Search and Choice in Preference Measurement,” Journal of Marketing Research. [M] — novelty gate Models within-task information search during a conjoint-style task using eye-tracking data, with an optimal-stopping flavor. If one paper threatens the contribution, it is this one. Read it first and read it carefully.
• Yang et al. (2018), Marketing Science. [M] Attention and information processing in incentive-aligned choice experiments.
• Meißner et al. (2016), Journal of Marketing Research. [M] Eye-tracking in conjoint; processing becomes measurably more efficient with practice. Directly relevant because a multi-task panel will exhibit learning across tasks, which appears in the model as drift in the search-cost parameter (Section 2.3, Section 9).

Distinction to preserve — and to verify before relying on it. These papers study search within a task, over the attributes of profiles already displayed, mostly in service of better preference estimates. SIFT studies search across alternatives, in a two-stage list-then-detail architecture, where the search cost is itself a deliverable rather than a nuisance parameter. Related, but not the same object. Confirm this holds before building on it.

11.3.7 Consideration Sets and Screening Rules in Conjoint

The existing marketing answer to “people do not evaluate everything,” and the comparison a conjoint-literate reader will reach for first.

• Gilbride and Allenby (2004), Marketing Science. [H] A choice model with conjunctive, disjunctive, and compensatory screening rules. The incumbent formulation.
• Hauser et al. (2010), Journal of Marketing Research. [H] Disjunctions of conjunctions; cognitive simplicity in consideration rules.
• Yee et al. (2007), Marketing Science. [M] Greedoid-based noncompensatory inference.
• Dzyabura and Hauser (2011), Marketing Science. [M] Active machine learning for consideration heuristics.
• Hauser (2014), Journal of Business Research. [M] Review of consideration-set heuristics.

Sharpest one-line contrast. This literature infers screening rules from choice outcomes, treating consideration as latent structure. SIFT observes the consideration process directly and prices it in units of search cost. Inference versus observation.

11.3.8 Conjoint Practice, Incentive Alignment, and Commercial Precedent

• Allenby et al. (2019), Handbook of the Economics of Marketing. [H] Economic foundations of conjoint analysis; the statement of the workhorse model that SIFT nests.
• Marshall and Bradlow (2002), JASA. [H] A unified approach to conjoint analysis models; precedent for treating distinct response formats as one likelihood family.
• Ding et al. (2005), Journal of Marketing Research; Ding (2007), Journal of Marketing Research. [H] Incentive alignment in conjoint. If estimated search costs are to carry units that mean anything, the task needs real stakes; this is the precedent for how to supply them.
• Sawtooth ACBC (adaptive choice-based conjoint). Not an academic reference, but the closest commercial product to a two-stage consideration instrument: a screening section (must-haves, unacceptables) followed by a choice tournament. Study as design precedent, and as the thing a client may already have bought.
• Virtual-shelf and shopper-simulation vendors. Several field shelf-set exercises with click data, reported descriptively (time to first click, click share). No known case of backing a structural search model out of one. This is where prior art most likely hides; check directly.

11.3.9 Positioning