GodIndex methodology: how to read a participant index

Define the observation before analysing it

A rating is one person’s submitted response. A participant count attempts to count people or accounts. A country index aggregates responses associated with a location. The overall GodIndex is a published composite. The original homepage’s “last close” is a display of that composite at an unspecified underlying close date.

These distinctions are not pedantic. Counting repeated responses as unique people inflates audience claims. Treating an index change as an absolute rating changes its meaning. Treating a capture timestamp as a vote timestamp creates precision that the archive does not contain.

The scale does not recover the formula

The historical response scale ran from −5 to +5. That tells us the response range, but it does not fully specify the GodIndex calculation. The first report mentions algorithm calibration and country contributions; the original executable formula, its complete version history and underlying vote records have not been recovered.

We consequently do not present a simple average as a verified reproduction of the old index. A future formula should be published with examples, a version identifier and an explanation of missing data. If the calculation changes, the resulting break must remain visible in the series.

Self-selection and changing audiences

People who visit a rating website may differ from people who never encounter it. An article, a community link or a news event may bring in a new group with a different response pattern. An index movement can therefore combine within-person change with a change in audience composition.

AAPOR distinguishes probability samples from opt-in samples and stresses transparency when interpreting the latter. A large number of responses does not remove this issue. We do not attach a conventional population margin of sampling error to the recovered GodIndex.

Time, geography and missing information

A daily collection needs an explicit time zone, a closing rule and a policy for late or repeated submissions. Geographic analysis needs a defined location field: self-reported country and inferred network location do not measure exactly the same thing. Country sample sizes and suppression thresholds need to accompany comparisons.

The archival sources do not expose a complete record of those procedures. Missing entries in our recovered collection mean “not recovered,” not “no votes” or “no change.” Interpolation may create a visually pleasing line, but it would be a modelled series and must never be labelled an original observation.

A proposed standard for the next chapter

Before new results are used for research, the protocol should describe eligibility, question wording, translations, response scale, repeat-vote handling, bot controls, exclusions and revisions. It should publish both response volume and aggregate results, so a score is not detached from its evidential base.

Research conclusions should distinguish description, association and causal inference. Confidence statements should name the statistical model used and its assumptions. The relaunch is not yet collecting votes, and these are design requirements to discuss with collaborators — not claims that a validated research panel already exists.

Read the research invitation to propose a pilot, or examine the recovered source-linked observations.