---
title: "The SIFT Likelihood"
subtitle: "Construction, and why conditioning on the observed choice makes GHK available"
author: "Dan Yavorsky"
format:
  html:
    toc: true
    code-fold: show
execute:
  eval: false
  warning: false
---

## What this notebook is for

This notebook builds the likelihood for the SIFT instrument and checks it. The
code lives in [`R/sift_model.R`](../R/sift_model.R); the checks are in
[`R/test_sift_model.R`](../R/test_sift_model.R). Everything here runs at
deliberately small sample sizes — the point is to establish that the machinery
is correct, not that it is fast.

It draws on three sources:

- **@Yavorsky_2021** for the utility decomposition, the closed-form reservation
  utility of @Kim_2010, the kernel-smoothed logit accept–reject simulator, and
  the argument that an *exogenous, product-specific* search-cost shifter lets us
  estimate the match-value standard deviation instead of fixing it to one.
- **@Ursu_Seiler_Honka_2024** for notation and the smoothing convention.
- **GHK**, for the block of the likelihood where it legitimately applies.

## Setup

```{r}
source("R/sift_model.R")
ginv <- make_ginv()
```

Respondent $i$ faces task $t$ with $J$ alternatives on a landing page, plus a
no-purchase outside option. Writing $\mathcal{S}$ for the ordered set of clicks:

$$
\delta_{ijt} = \mathbf{X}_{ijt}'\boldsymbol{\beta} + \eta_{ijt}, \qquad
u_{ijt} = \delta_{ijt} + \varepsilon_{ijt}, \qquad
u_{i0t} = a_0 .
$$

$\delta$ is what the respondent knows from the landing page. $\varepsilon$ is
what a click reveals. The search cost depends on randomized position,
$c_{ijt} = \exp\{\gamma_0 + \gamma_p\,\mathrm{pos}_{ijt}\}$, and the reservation
value follows @Weitzman_1979 via @Kim_2010:

$$
z_{ijt} = \delta_{ijt} + \zeta_{ijt}\,\tilde\sigma,
\qquad g(\zeta) \equiv \phi(\zeta) - \zeta\,[1-\Phi(\zeta)] = c_{ijt}/\tilde\sigma .
$$

### Where SIFT departs from the field settings

Four things differ from the environments @Yavorsky_2021 and @Ursu_2018 study.

1. **It is a panel.** Each respondent completes $T$ tasks, so we observe many
   search spells per person. @Morozov_2021 needs panel structure to identify
   preference heterogeneity in a search model; conjoint gets it by design, and
   so does SIFT.
2. **Position is randomized**, so the search-cost shifter is exogenous by
   construction rather than by argument. @Yavorsky_2021 had to defend the
   exogeneity of driving distance at length (their §3.3.1); we do not.
3. **The product page is designed.** A click reveals $\mathbf{L}_{ijt}'\boldsymbol{\kappa} + \xi$
   where $\mathbf{L}$ is drawn by *us*. From the respondent's viewpoint the revealed
   component is a mixture of normals, not a normal. We use the variance-matched
   normal $\tilde\sigma^2 = \boldsymbol{\kappa}'\mathrm{Var}(\mathbf{L})\boldsymbol{\kappa} + \sigma_\xi^2$
   in the reservation value.
4. **Zero clicks are possible**, and non-purchase is an outcome we model rather
   than condition away — @Yavorsky_2021 conditions on search and purchase.

## The DGP

```{r}
d <- sim_sift(N = 60, T = 8, J = 6, seed = 3, ginv = ginv)
table(rowSums(d$cons_set != 0))     # clicks per task
```

At the default parameters this produces roughly 5% zero-click tasks, 74%
one-click, and a thin tail beyond — a click distribution with the same shape as
the field data in @Yavorsky_2021 (their Fig. 3: 91% one search), but with more
mass in the tail because we can make clicking cheap by design.

## Two likelihoods

### (A) Kernel-smoothed logit accept–reject

This is the @Yavorsky_2021 Appendix C simulator: form the violation margins
$\nu$ for each Weitzman condition and smooth the indicator with a logit kernel,

$$
\tilde L^q = \Big(1 + \textstyle\sum_j e^{-\lambda_1\nu_{1j}}
 + \sum_j e^{-\lambda_2\nu_{2j}} + e^{-\lambda_3\nu_3} + e^{-\lambda_4\nu_4}\Big)^{-1}.
$$

It is general and easy to extend. Its weakness is well known and stated plainly
in @Yavorsky_2021 (their footnote 21): the estimator is only *asymptotically*
unbiased, the bias falls as the scale factors $\lambda$ grow, and large
$\lambda$ destroys the smoothness the optimiser needs.

### (B) GHK on the $\eta$ block, analytic on the $\varepsilon$ block

The construction rests on one observation.

::: {.callout-note}
## Conditioning on the observed choice turns a union into a conjunction

The stopping rule $\max_{h\le K} u_h \ge \max_{l>K} z_l$ is a statement
about a maximum, so as written it is a *union* of events — any one of the
searched alternatives could be the one that clears the bar — and unions are not
orthants. But the choice rule tells us *which* alternative attains the maximum.
Conditional on the observed purchase, stopping becomes
$u_{j^*} \ge z_l$ for every unsearched $l$: a conjunction of linear
inequalities, which is exactly what GHK evaluates.
:::

Two further simplifications follow:

- **Continuation collapses to a single binding constraint.** Continuing past
  step $k$ requires $u_h < z_k$ for all $h<k$. Since $z$ decreases along
  the search order, the binding constraint is $z_K$ alone: continuation
  $\iff \max(u_0, u_{s_1},\dots,u_{s_{K-1}}) < z_{s_K}$.
- **Conditional on $\boldsymbol{\eta}$ and on $\varepsilon_{j^*}$, the remaining
  $\varepsilon$'s are independent one-sided constraints** and integrate out in
  closed form as a product of normal CDFs. Only a *one-dimensional* integral
  remains.

So GHK draws $\boldsymbol{\eta}$ from the truncated selection region — smooth, and low
variance — and the $\varepsilon$ block is quadrature rather than simulation.

```{r}
d2   <- sim_sift(N = 25, T = 6, J = 5, seed = 7, ginv = ginv)
info <- prep_sift(d2)
dr   <- make_draws_sift(d2, nd = 200, seed = 21)
nll_sift_ks (d2$true_par, d2, info, dr, ginv)
nll_sift_ghk(d2$true_par, d2, info, dr, ginv)
```

## The trap this construction sets

The continuation constraint applies to **every** $h<K$ — *including the chosen
alternative when it is not the last click*. If the respondent clicked three
things and bought the first, then continuing past it required
$u_{j^*} < z_{s_K}$, which caps $\varepsilon_{j^*}$ from above. Omitting
that cap is an easy mistake and a silent one: the likelihood stays positive,
finite, and smooth, and the optimiser converges happily on a model that is
simply wrong.

It is caught immediately by the enumeration test below. Before the fix, every
outcome in which the purchase *was* the last click had ratio 1.00 against brute
force, and every outcome in which it was not was inflated by 8–33×. That
signature named the bug precisely.

## A second trap: the ex ante and realized views of a click

The post-search component appears twice in the model and means something
different each time. **Before** a click the respondent does not know
$\mathbf{L}$, so from their viewpoint what a click will reveal has the ex ante
spread $\tilde\sigma$ — that is the object belonging in the reservation value.
**After** a click, $\mathbf{L}$ is revealed to the respondent *and observed by
us*, because we designed it, so realized utility carries
$\mathbf{L}'\boldsymbol{\kappa}$ in the **mean** and only $\xi$ stays random.

Collapsing the two — using $\tilde\sigma$ for realized utility as well — is
tempting because it shortens the likelihood, and it is wrong in a way that is
invisible from the likelihood value. It leaves $\boldsymbol{\kappa}$ entering
*only* through
$\tilde\sigma^2 = \boldsymbol{\kappa}'\mathrm{Var}(\mathbf{L})\boldsymbol{\kappa} + \sigma_\xi^2$,
where it trades off exactly against $\sigma_\xi$ and is not identified at all.

We found this the direct way. A recovery run returned $\hat\kappa$ sitting at
*exactly* its starting value of zero on both coordinates, with $\sigma_\xi$
inflated from 0.70 to 1.47 so as to hold $\tilde\sigma$ near its true 1.29 —
the signature of a parameter the likelihood cannot see. Everything else
($\boldsymbol{\beta}$, $a_0$, $\gamma_0$, $\gamma_p$) recovered fine, which is
what makes the failure easy to miss.

Folding the realized $\mathbf{L}$ into the mean for searched alternatives is
therefore not a refinement. It is what makes designing the product page pay off:
$\boldsymbol{\kappa}$ is identified by *which* alternative gets bought given the
attributes that the click revealed.

## Checks

`R/test_sift_model.R` runs five. The decisive one enumerates every reachable
(ordered click sequence, purchase) outcome at $J=3$ — 49 of them — and checks
that the model probabilities sum to one and match a brute-force simulation of
the DGP written independently of the likelihood.

| check | result |
|---|---|
| DGP click distribution sensible | ✓ |
| both likelihoods finite, per-task $\log L \le 0$ | ✓ |
| simulation variance, KS vs GHK | GHK lower; see the comparison study |
| truth beats perturbations | ✓ both |
| 49 outcomes sum to one | **1.0009** |

Per-outcome agreement with brute force is within Monte Carlo error everywhere
(ratios 0.97–1.02).

```{r}
# the decisive check
source("R/test_sift_model.R")
```

## A mini recovery run, and what it does not show

`R/mini_recovery.R` fits 60 respondents $\times$ 8 tasks ($J=5$, $n_d=120$) from
a cold start. It takes 5.5 minutes and converges.

| | true | before the $\mathbf{L}$ fix | after |
|---|---|---|---|
| $\beta_1$ | 1.000 | 1.058 | 1.031 |
| $\beta_2$ | −0.800 | −0.784 | −0.743 |
| $\kappa_1$ | 0.900 | **0.000** | 1.259 |
| $\kappa_2$ | −0.600 | **0.000** | −1.111 |
| $\log\sigma_\xi$ | −0.357 | 0.388 | 0.194 |
| $a_0$ | 0.400 | 0.371 | 0.085 |
| $\gamma_0$ | −1.050 | −0.887 | −0.207 |
| $\gamma_p$ | 0.120 | 0.101 | 0.052 |
| $-\log L$ | | 1080.5 | **950.7** |

The point of the table is the $\kappa$ rows and the log-likelihood, not the
errors. Before the fix $\kappa$ sat at exactly its starting value; after it,
$\kappa$ moves and the fit improves by 130 log-likelihood points on the same
data. That is the identification result.

::: {.callout-warning}
## This is not a parameter-recovery claim

480 tasks is far too small, roughly three quarters of them have a single click,
and $\gamma_0$ and $\log\sigma_\xi$ are known to lie on a ridge —
@Yavorsky_2021 reports a correlation of 0.95 between the search-cost intercept
and $\log\sigma$ even in field data with 6,511 consumers. The errors on
$\gamma_0$ (+0.84) and $a_0$ (−0.32) are consistent with that ridge and with the
sample size; they are not evidence that the likelihood is right, and they are
not evidence that it is wrong.

The evidence that the likelihood is *correct* is the enumeration test: 49
outcomes summing to 1.0024, each matching an independently written brute-force
simulation of the DGP within Monte Carlo error. Parameter recovery is a question
for @sec-simstudy, at commercial task counts, with a proper replication design.
:::

## Where this leaves the estimation plan

GHK's advantage is real but it is *not* the dramatic constant an earlier draft
of this work claimed. Measured properly — 20 independent draw sets per cell,
rather than the six that produced a headline of "399×" that did not survive
re-measurement — the variance ratio against the kernel-smoothed simulator is
**100× at 150 tasks and 21× at 320 tasks**. It moves around with the design,
because the kernel-smoothed estimator's spread is driven by a handful of
awkward tasks rather than by an average.

The number that actually matters is time to a given precision, since runtime is
linear in draws. On that basis GHK is **3–21× cheaper** than kernel smoothing
across the two designs measured, and crude frequency is not competitive at all —
175–260× more expensive, and badly biased besides, because zero cells have to be
floored. Kernel smoothing also carries a visible upward bias in the negative
log-likelihood (305 against GHK's 295 at 150 tasks), exactly the asymptotic-only
unbiasedness that @Yavorsky_2021 flags in his footnote 21.

Full detail, including the crude-frequency arm, is in
[the simulator comparison](../comparisons/compare-simulators.qmd).

That matters for the 12-hour budget. It also reframes the choice recorded in
[`sections/05_identification.qmd`](../sections/05_identification.qmd): the binding
constraint on SIFT is estimating **heterogeneity**, and the cheaper route there
is hierarchical Bayes, which never integrates over the heterogeneity
distribution at all.

## Still to build

- Random coefficients on $\boldsymbol{\beta}$ and $\boldsymbol{\kappa}$, and the HB sampler.
- Correlated $\mathbf{L}$ across attributes; the variance-matched approximation
  already accommodates it through $\mathrm{Var}(\mathbf{L})$, but the DGP currently
  draws $\mathbf{L}$ independently.
- A proper simulation study at commercial task counts, for
  [`sections/06_simstudy.qmd`](../sections/06_simstudy.qmd).