Validation

What this page is

Neither Yallop (1997) nor Odeh (2004) reports a validation in the modern sense. Yallop calibrated his zone boundaries by eye against the same 295 records he published, deleted one contradicting entry as “false”, and explained away three more; Odeh's paper contains no confusion matrix, no error rate, and no held-out test at all. That is not a scandal — both were doing what the field did in the 1990s — but it does mean the numbers this site puts on a map have never been scored in public.

This page scores them. Everything below is regenerated from the data by scripts/run_validation.py; no figure in this prose is typed by hand. Generated 2026-08-13 from commit cb92167 against JPL DE440s.

The observations

Two datasets, doing two different jobs. Keeping them apart is the whole design: scoring a criterion against the records it was fitted to measures goodness of fit, not the ability to predict anything.

Only records after 2004 are counted as out of sample. Earlier Alrefay records are probably out of sample too — Odeh lists his six sources and KACST is not among them — but a guarantee from the calendar is worth more than an inference, so the headline uses the 379 records that cannot be in either fit set.

Does the code reproduce Yallop?

Before testing anyone else's criterion, test your own arithmetic. Yallop printed his ARCV, W′ and q for all 271 evening records, so each can be recomputed from JPL DE440s and compared with what he published in 1997.

Yallop's narrowest zone spans 0.061 in q. All 271 of 271 records reproduce his q to better than that, so no record is misfiled by the implementation alone.

Getting there corrected three mistakes of ours, every one found by the residuals rather than by reading the note. Table 4's dates are UT, and west of Greenwich the local evening is usually the day before — five records were computing the wrong evening, with q out by up to 2, which is eight zone widths. Its ARCL and ARCV are geocentric; comparing against a topocentric separation left a systematic −0.88°, one lunar parallax. And its Age is measured at best time, not at sunset, a systematic −0.44 h. We had previously checked two of Yallop's worked values; checking all 271 is what surfaced these.

Out-of-sample results

Each criterion is asked the one question all three genuinely answer in common: should an unaided observer expect to see it? Misses and false alarms are reported separately and never combined into an accuracy, because the balance between sightings and non-sightings here reflects which evenings people chose to go out and look.

The first column is the result that matters. On 157 crescents that were actually seen with the naked eye, all three criteria said “expect to see it” for every one but a single record. Whatever else these criteria get wrong, they do not tell you to stay indoors on a night the crescent is there.

The second column looks like Odeh beating Yallop three to one. It is not, and the next two sections explain why.

How reliable is each zone?

This is the question a reader of the maps actually has: the map says zone B — how often is it seen? Each criterion is shown against its own zone scale, because Yallop's A–F and Odeh's A–D are different scales that reuse letters. Bars are the observed fraction; whiskers are 95% Wilson intervals, drawn because several zones hold few enough records that the point estimate alone would be misleading.

Yallop (1997)

Odeh (2004)

SAAO (2001)

Two things stand out. Yallop's zone B — “visible under perfect conditions” — was seen with the naked eye in 1 of 36 cases, which is where his apparent false-alarm rate comes from: the zone is doing almost no work as a naked-eye prediction on this data. And SAAO's top zone is 75.1% reliable against Yallop's and Odeh's ~90%. That is not a defect. Caldwell & Laney describe their limits as deliberately optimistic — thresholds below which sighting is improbable for a skilled observer in ideal conditions, not predictions of what a typical observer will see. Finding their zone more permissive is a check that we have read them correctly.

Yallop and Odeh are one polynomial

Set the two criteria side by side and the width terms are identical to five significant figures. Within a single convention V = 10q + 4.672 exactly: an affine transform. The two scores are therefore perfectly rank-correlated, and any threshold-free comparison of them — an ROC curve, an AUC — has to come out the same by construction. It does: AUC 0.9945 for q against 0.9946 for V.

So a threefold difference in false alarms cannot be about the models. It is about which zones were counted as a naked-eye prediction — Yallop's own zone text puts both A and B there, Odeh's puts only A. Scored like for like, on each criterion's single top zone:

Within a point of each other, as the shared polynomial demands. The difference the first table showed is real and worth knowing — it is where Yallop's B/C boundary sits — but it is a fact about his zone scale, not about his model. We report both because reporting only the first would have been the more flattering and less true comparison.

What a convention is worth

This site evaluates each criterion in the convention its constants were fitted on: Yallop on geocentric ARCV with the TN 69 width W′, Odeh on topocentric ARCV and W. That rule has a price in complexity, and the obvious question is whether it buys anything.

It buys about two zone widths. Across the 379 out-of-sample records, V − (10q + 4.672) has a median of -1.1948 with a standard deviation of 0.1447. If the polynomials are identical, that entire offset is the convention — geocentric versus topocentric reduction, nothing else. In Yallop's units it is a shift of about 0.12 in q, against zone boundaries that are 0.061 apart at their closest.

What this does not show

• One latitude band. Every Alrefay site lies between 19.1°N and 28.4°N, in Saudi atmospheric conditions. Nothing here tests high latitudes, where the geometry is least favourable and where Yallop himself flagged his criterion as least trustworthy.
• No elevation. The dataset carries no elevation column although its sites are 0.6–1.5 km up. Elevation moves ARCV mainly by shifting sunset and moonset, so this is a real uncertainty rather than a negligible one.
• Our gate, not their criterion. A domain-of-validity gate of ours demotes a verdict whenever ARCV ≤ 0 or the moon is below the horizon, before any polynomial is consulted. It is not in TN 69 or in Odeh. The per-zone figures above note how many records it decided.
• Schaefer's codes are left uninterpreted. Yallop's observation column uses a code alphabet defined in prose we have not read against a copy of the note, and its parenthesised variants admit more than one reading. Rather than guess, we use his set only for testing our arithmetic. For context on what he achieved in sample: his own predicted code matches the observed code exactly in 141 of 271 evening records (52.0%), and agrees on the leading V/I letter in 80.1%. Those are his columns and his arithmetic, not ours.
• Odeh's own 737 records are not here. The only open copy of his Table VI is hosted on a site whose robots.txt disallows automated retrieval by us, so we do not fetch it.
• A single campaign is not the world. These are 379 records from one observing programme. They are enough to say that the criteria rarely miss, and not enough to certify any of them.

Reproducing this

The observation files are committed, with provenance and checksums in data/observations/SOURCES.md. To regenerate every number on this page:

uv venv --python 3.12 && uv pip install -e ".[dev]"
.venv/bin/pytest generator/tests/test_validate_metrics.py
.venv/bin/python scripts/run_validation.py

The tests assert the claims this page makes, so a change that would alter what it says fails the suite rather than silently rewriting the prose.

References

1. Yallop, B. D. (1997) A Method for Predicting the First Sighting of the New Crescent Moon. NAO Technical Note No. 69, HM Nautical Almanac Office
   The q-test, its zone boundaries, the best-time rule, and the definition of ARCV as a geocentric quantity. Table 4 lists 295 observations with worked q values; two of them pin our implementation.
2. Odeh, M. Sh. (2004) New Criterion for Lunar Crescent Visibility. Experimental Astronomy 18, 39–64
   The V-test, fitted to 737 observations, about half of them from ICOP. Source of the empirical Danjon limit of 6.4° and of the SAAO table reproduced as its Table IV. Record No. 514 is our end-to-end check.
3. Caldwell, J. A. R. & Laney, C. D. (2001) First Visibility of the Lunar Crescent. African Skies 5, 15
   The SAAO DALT criterion: the moon's apparent lower-limb altitude at sunset against two limit curves in relative azimuth.
4. Alrefay, T., Alsaab, S., Alshehri, F., Alghamdi, A., Hadadi, A., Alotaibi, M., Almutari, K. & Mubarki, Y. (2018) Analysis of observations of earliest visibility of the lunar crescent. The Observatory 138, 267–291
   The out-of-sample test set: 27 years of observations by professional and trained astronomers at the National Center for Astronomy, KACST, made under clear skies. Clear skies are what make its non-sightings usable — an ordinary negative report cannot distinguish an invisible crescent from a cloud.
5. Cross, E., Power, N. & Alexander, E. (2024) mphys-moon: machine-readable transcriptions of crescent sighting datasets. University of Manchester, MIT licence, commit 84f75ca
   Where our two observation files come from, verbatim: Yallop's Table 4 and Alrefay's campaign, transcribed to CSV. We use only the transcriptions of published papers, not the repository's direct scrape of the ICOP website.
6. Park, R. S. et al. (2021) The JPL Planetary and Lunar Ephemerides DE440 and DE441. The Astronomical Journal 161, 105
   The ephemeris behind every position on this site. Its accuracy exceeds what crescent prediction can use by orders of magnitude — the uncertainty here is in the eye, not the orbit.
7. Rhodes, B. Skyfield: high precision research-grade positions. Python package