Page MenuHomePhabricator

Bug 1610626 - Fix wrench reftest mode incorrectly printing unexpected in some cases.
ClosedPublic

Authored by gw on Jan 21 2020, 8:28 PM.

Details

Summary

The semantics of the inaccuracy reftest mode are that the overall
test succeeds as long as one of the multiple test images mismatches
the reference image.

However, the previous implementation would print an unexpected result
if any of the comparisons were equal (even though the overall test
result would be reported correctly).

This patch changes wrench so that it only reports an unexpected
failure for the overall test, not each individual reference.

Diff Detail

Repository
rMOZILLACENTRAL mozilla-central
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

gw created this revision.Jan 21 2020, 8:28 PM
Herald added a project: Restricted Project. · View Herald TranscriptJan 21 2020, 8:28 PM
phab-bot requested review of this revision.Jan 21 2020, 8:28 PM
phab-bot changed the visibility from "Custom Policy" to "Public (No Login Required)".
phab-bot changed the edit policy from "Custom Policy" to "Restricted Project (Project)".
phab-bot removed a project: Restricted Project.
Bert accepted this revision.Jan 22 2020, 1:45 AM
Bert added inline comments.
gfx/wr/wrench/src/reftest.rs
153

Minor, but perhaps something like "report_unexpected_equality" is a bit easier to reason about than the double negative of "failed to be unequal"?

This revision is now accepted and ready to land.Jan 22 2020, 1:45 AM
gw updated this revision to Diff 220653.Jan 22 2020, 2:22 AM
gw added inline comments.Jan 22 2020, 2:22 AM
gfx/wr/wrench/src/reftest.rs
153

Yup, makes sense - updated to report_unexpected_equality.