The Bradley-Terry Model

A Probabilistic Framework for Pairwise Comparisons

Go to Calculator

The Bradley-Terry model is a statistical framework for modelling pairwise comparison data, first proposed by Bradley and Terry (1952). One application of the model is to take a series of pairwise comparisons (people are good at A vs B judgments due to reduced cognitive load in binary choices[1][4]) and converting them to a ranking (people struggle with direct ranking of multiple items due to working memory limitations[4][3]).

Bradley-Terry Model Definition

Given a set of items \( \{1, \ldots, n\} \), each with an associated positive parameter \( \pi_i \), the probability that item \( i \) is preferred over item \( j \) is given by:

\[ P(i > j) = \frac{\pi_i}{\pi_i + \pi_j} \]

where \( \pi_i \) represents the "strength" or "utility" of item \( i \).

Bradley & Terry (1952), Zermelo (1929)

Model Properties

1. Scale Invariance

The model is invariant to positive scaling of the parameters:

\[ P(i > j) = \frac{c\pi_i}{c\pi_i + c\pi_j} = \frac{\pi_i}{\pi_i + \pi_j} \quad \forall c > 0 \]

This means the parameters are typically constrained (e.g., \( \sum \pi_i = 1 \) or \( \pi_1 = 1 \)) for identifiability.

Bradley & Terry (1952)

2. Logit Formulation

The model can be expressed in log-odds form:

\[ \log \left( \frac{P(i > j)}{P(j > i)} \right) = \log \pi_i - \log \pi_j \]

This reveals the model as a special case of logistic regression where the linear predictor is \( \log \pi_i - \log \pi_j \).

Bradley & Terry (1952), Springall (1973)

Transitivity Property

For any three items \( i, j, k \), if \( P(i > j) > \frac{1}{2} \) and \( P(j > k) > \frac{1}{2} \), then:

\[ P(i > k) > \frac{1}{2} \]

This stochastic transitivity property makes the model particularly suitable for ranking problems.

Davidson (1970)

Parameter Estimation

The standard approach estimates parameters via maximum likelihood. Given observed comparisons \( y_{ij} \) (number of times \( i \) was preferred over \( j \)), the log-likelihood is:

\[ \ell(\pi) = \sum_{i < j} \left[ y_{ij} \log \left( \frac{\pi_i}{\pi_i + \pi_j} \right) + y_{ji} \log \left( \frac{\pi_j}{\pi_i + \pi_j} \right) \right] \]
Estimation Algorithm (Zermelo, 1929)

The maximum likelihood estimates can be obtained via the following iterative scheme:

\[ \pi_i^{(k+1)} = \frac{w_i}{\sum_{j \neq i} \frac{n_{ij}}{\pi_i^{(k)} + \pi_j^{(k)}}} \]

where \( w_i \) is the total number of wins for item \( i \), and \( n_{ij} = y_{ij} + y_{ji} \).

Bradley & Terry (1952), Zermelo (1929)

Connection to Elo Rating

With a scale factor of 400, the Bradley-Terry model is equivalent to the Elo rating system:

\[ P(i > j) = \frac{1}{1 + 10^{(R_j - R_i)/400}} \]

where \( R_i \) and \( R_j \) are Elo ratings.

Bradley & Terry (1952)

Extensions and Variations

Extension Description Reference
Plackett-Luce Generalizes to rankings of multiple items Plackett (1975), Luce (1959)
Davidson Model Incorporates ties in comparisons Davidson (1970)
Covariate BT Includes item features as predictors Springall (1973)
BT Regression Trunk Tree-based modeling of subject covariates D'Ambrosio et al. (2023)

References

  1. Reips, U.-D., & Funke, F. (2008). Interval-level measurement with visual analogue scales in Internet-based research: VAS Generator. Behavior Research Methods, 40(3), 699-704. (Demonstrates cognitive advantages of binary choices)
  2. Bradley, R. A., & Terry, M. E. (1952). Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika, 39(3/4), 324-345. https://www.jstor.org/stable/2334029
  3. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81-97. (Working memory limitations)
  4. Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34(4), 273-286. (Foundational work on pairwise comparison psychology)
  5. Zermelo, E. (1929). Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem der Wahrscheinlichkeitsrechnung. Mathematische Zeitschrift, 29(1), 436-460.
  6. Davidson, R. R. (1970). On extending the Bradley-Terry model to accommodate ties in paired comparison experiments. Journal of the American Statistical Association, 65(329), 317-328.
  7. Plackett, R. L. (1975). The analysis of permutations. Journal of the Royal Statistical Society: Series C, 24(2), 193-202.
  8. Springall, A. (1973). Response surface fitting using a generalization of the Bradley-Terry paired comparison model. Applied Statistics, 22(1), 59-68
  9. D'Ambrosio, A., et al. (2023). The Bradley-Terry regression trunk approach for modeling preference data with small trees. Psychometrika, 88(4), 1443-1465