Page MenuHomePhabricator

Bug 1558926 - Part 1: Add data structures and pref for display item caching r=kvark
ClosedPublic

Authored by miko on Oct 23 2019, 1:55 PM.

Diff Detail

Repository
rMOZILLACENTRAL mozilla-central
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
kvark added inline comments.Dec 5 2019, 6:31 PM
gfx/layers/wr/WebRenderCommandBuilder.cpp
1672 ↗(On Diff #203356)

what's the semantic difference between "intern" and "reuse" here?

1680 ↗(On Diff #203356)

we only intern background color items and nothing else?

1699 ↗(On Diff #203356)

in what case would an item be re-used but not cached?

1707 ↗(On Diff #203356)

what are the guaranteed assumptions about the state of mCachedItemState versus aBuilder?

1707 ↗(On Diff #203356)

if aBuilder is required to actually hold the cached items, and it is aware of their indices (parameter here), did you consider making it responsible for knowing which indices are cached (as opposed to having mCachedItemState at all here)?

gfx/layers/wr/WebRenderCommandBuilder.h
263 ↗(On Diff #203356)

are we handling a case where this overflows?

263 ↗(On Diff #203356)

this is bumped before the use, should it be initialized to 0 then?

gfx/webrender_bindings/WebRenderAPI.h
581 ↗(On Diff #203356)

nit: should be a verb

gfx/wr/webrender/src/scene.rs
171 ↗(On Diff #203356)

nit: use map_or

gfx/wr/webrender/src/scene_building.rs
730 ↗(On Diff #203356)

should this be merged with InternedCollection by making the space optional?

758 ↗(On Diff #203356)

nit: we could just else {..}

788 ↗(On Diff #203356)

nit: use map_or

gfx/wr/webrender_api/src/display_item.rs
101

nit: rename to with_space_and_clip

gfx/wr/webrender_api/src/display_list.rs
258

doesn't this cache size always end up being 50k?

270

how does the concept of "collection" apply here? the current_key is set to a concrete key that (as far as I see) can only be associated with one cached item

276

why do we need this?

277

should be able to avoid explicit lifetimes here

1568

why are we switching back and forth between the data and the extra data, as opposed to writing everything into the same place?

This revision now requires changes to proceed.Dec 5 2019, 6:31 PM

In our previous code coverage analysis run, we found some files which had no coverage and are being modified in this patch:

Should they have tests, or are they dead code ?

  • You can file a bug blocking Bug 1415824 for untested files that should be tested.
  • You can file a bug blocking Bug 1415819 for untested files that should be removed.

If you see a problem in this automated review, please report it here.

miko marked 10 inline comments as done.Dec 5 2019, 7:15 PM
miko added inline comments.
gfx/layers/wr/WebRenderCommandBuilder.cpp
1586 ↗(On Diff #203356)

Good catch, it returns the count of bools.

1672 ↗(On Diff #203356)

I had a bit of trouble coming up with consistent naming. Intern here refers to something that can be stored and later reused. Will add comments.

1680 ↗(On Diff #203356)

It was an easy item to implement for MVP. I am hoping to make Text item interning work before landing.

1699 ↗(On Diff #203356)

isReusedItem here is set if Gecko display list merging reused the item. It is not related to caching, which is why we track the cache state separately. Will add comments.

gfx/wr/webrender/src/scene_building.rs
730 ↗(On Diff #203356)

This was an early optimization to avoid sending SpaceAndClipInfo unnecessarily. Depending on how options are serialized, it might be possible.

gfx/wr/webrender_api/src/display_list.rs
258

If the value of 50k will be hard coded, yes.
This supports growing the cache dynamically. At some point we probably want to set it based on the display list length.

276

The reduce the cache size. DisplayItem enum is around twice as large as CachedDisplayItem.
Hopefully they can be merged at some point.

1568

I am not a fan of this approach myself, but I think it was the easiest and probably the fastest way to implement two-part display list.
The interned display item data is gathered in a separate buffer, that is appended to actual display item buffer at the end, and later accessed using the extra_data_offset.
Will add comments.

miko updated this revision to Diff 203491.Dec 5 2019, 7:36 PM
miko marked 3 inline comments as done.

Revision updated.

In our previous code coverage analysis run, we found some files which had no coverage and are being modified in this patch:

Should they have tests, or are they dead code ?

  • You can file a bug blocking Bug 1415824 for untested files that should be tested.
  • You can file a bug blocking Bug 1415819 for untested files that should be removed.

If you see a problem in this automated review, please report it here.

miko updated this revision to Diff 203751.Dec 6 2019, 1:34 AM
kvark requested changes to this revision.Dec 6 2019, 4:30 PM
kvark added inline comments.
gfx/layers/wr/WebRenderCommandBuilder.cpp
1672 ↗(On Diff #203356)

It sounds like "cached" expresses this semantics

1699 ↗(On Diff #203356)

I'm curious if it should be related. Is it feasible to associate reused == cached/interned? It seems awkward to have 2 different layers of caching here side by side.

1719 ↗(On Diff #203751)

It feels to me that StartItem/EndItem API could be more explicit about what's happening at this level. At this level we know that this is a single item, and it needs to be encoded into the "cache" as opposed to display list itself. This means an entirely different code path on WR API side. It has to assume and expect only specific kind of items, and only one of them, precisely between Start and End...
However, we are still calling CreateWebRenderCommands, which isn't aware of what we are doing.

Would it be possible or feasible to have a separate CreateWebRenderCachedItem function instead?

gfx/wr/webrender/src/scene_building.rs
730 ↗(On Diff #203356)

ok, makes sense

gfx/wr/webrender_api/src/display_list.rs
1568

ok, sounds plausible

1570

nit: just extra_data_offset,

This revision now requires changes to proceed.Dec 6 2019, 4:30 PM
miko updated this revision to Diff 205447.Dec 10 2019, 3:34 PM
miko marked 8 inline comments as done.Dec 10 2019, 3:46 PM
miko marked an inline comment as done.

Code analysis found 2 defects in the diff 205447:

  • 1 defect found by clang-tidy
  • 1 defect found by Coverity

You can run this analysis locally with:

  • ./mach static-analysis check gfx/layers/wr/WebRenderCommandBuilder.h (C/C++)

If you see a problem in this automated review, please report it here.

gfx/layers/wr/WebRenderCommandBuilder.h
82 ↗(On Diff #205447)

Warning: Constructor does not initialize these fields: mPreviousPipelineId, mCachedItems, mReusedItems, mTotalItems [clang-tidy: cppcoreguidelines-pro-type-member-init]
Checker reliability is medium, meaning that the false positive ratio is medium.

82 ↗(On Diff #205447)

Checker reliability is high, meaning that the false positive ratio is low.
Non-static class member "mTotalItems" is not initialized in this constructor nor in any functions that it calls.
The path that leads to this defect is:

  • obj-x86_64-pc-linux-gnu/gfx/webrender_bindings/webrender_ffi_generated.h:625:
    • member_decl: Class member declaration for "mNamespace"..
  • gfx/layers/wr/WebRenderCommandBuilder.h:82:
    • uninit_member: Non-static class member field "mPreviousPipelineId.mNamespace" is not initialized in this constructor nor in any functions that it calls..
  • obj-x86_64-pc-linux-gnu/gfx/webrender_bindings/webrender_ffi_generated.h:626:
    • member_decl: Class member declaration for "mHandle"..
  • gfx/layers/wr/WebRenderCommandBuilder.h:82:
    • uninit_member: Non-static class member field "mPreviousPipelineId.mHandle" is not initialized in this constructor nor in any functions that it calls..
  • gfx/layers/wr/WebRenderCommandBuilder.h:150:
    • member_decl: Class member declaration for "mCachedItems"..
  • gfx/layers/wr/WebRenderCommandBuilder.h:82:
    • uninit_member: Non-static class member "mCachedItems" is not initialized in this constructor nor in any functions that it calls..
  • gfx/layers/wr/WebRenderCommandBuilder.h:150:
    • member_decl: Class member declaration for "mReusedItems"..
  • gfx/layers/wr/WebRenderCommandBuilder.h:82:
    • uninit_member: Non-static class member "mReusedItems" is not initialized in this constructor nor in any functions that it calls..
  • gfx/layers/wr/WebRenderCommandBuilder.h:150:
    • member_decl: Class member declaration for "mTotalItems"..
  • gfx/layers/wr/WebRenderCommandBuilder.h:82:
    • uninit_member: Non-static class member "mTotalItems" is not initialized in this constructor nor in any functions that it calls..
miko updated this revision to Diff 205499.Dec 10 2019, 5:04 PM
miko marked 6 inline comments as done.Dec 10 2019, 5:14 PM
miko added inline comments.
gfx/layers/wr/WebRenderCommandBuilder.cpp
1719 ↗(On Diff #203751)

How do you find the new approach? This introduces some extra layering, but I believe it makes things easier to follow.

Code analysis found 1 defect in the diff 205499:

  • 1 defect found by clang-tidy

You can run this analysis locally with:

  • ./mach static-analysis check gfx/layers/wr/WebRenderCommandBuilder.h (C/C++)

If you see a problem in this automated review, please report it here.

gfx/layers/wr/WebRenderCommandBuilder.h
82 ↗(On Diff #205499)

Warning: Constructor does not initialize these fields: mCachedItems, mReusedItems, mTotalItems [clang-tidy: cppcoreguidelines-pro-type-member-init]
Checker reliability is medium, meaning that the false positive ratio is medium.

miko updated this revision to Diff 205616.Dec 10 2019, 7:47 PM
kvark requested changes to this revision.Dec 10 2019, 9:00 PM

Amazing work! I think this is ready to land for the most part.
Requested changes are not critical, but are somewhat many.

gfx/layers/wr/WebRenderCommandBuilder.cpp
1718 ↗(On Diff #205499)

nit: easier to just do "<"

gfx/layers/wr/WebRenderCommandBuilder.h
109 ↗(On Diff #205499)

this would invalidate the cache, do we have (or need?) any guard against using this cache?
I.e. maybe we should call ResetStats here

gfx/layers/wr/WebRenderLayerManager.cpp
327 ↗(On Diff #205499)

if you need logging, please use the mozilla logging infrastructure, i.e. MOZ_LOG with a new or an existing logger

gfx/wr/webrender/src/scene_builder_thread.rs
540 ↗(On Diff #205499)

let's store it in self and re-use in order to avoid the heap allocation here

543 ↗(On Diff #205499)

this is changing the logic: we are no longer updating the pipelines that are removed
it makes sense, but is this an intentional change?

gfx/wr/webrender/src/scene_building.rs
782 ↗(On Diff #205499)

so, to clarify, this PR only saves the time on WR command builder in Gecko plus a few bits on the IPC side, correct?

gfx/wr/webrender_api/src/display_list.rs
233

we are unwrapping it right here, no need to check additionally

253

should this check if there is something already stored?

273

we'd probably want to move that array out in the future in order to coalesce heap allocations (not required at this point)

435

this looks awkward, isn't this checking for ReuseItem in addition to the other place that does? Can we have that other place accessing "self.cache.and_then(|c| c.get_item(key))" then?

1551

we should rename this variable to indicate that it's only temporary

This revision now requires changes to proceed.Dec 10 2019, 9:00 PM
miko marked 10 inline comments as done.Dec 13 2019, 1:20 PM
miko added inline comments.
gfx/wr/webrender/src/scene_builder_thread.rs
543 ↗(On Diff #205499)

Yes, it avoids the possibly costly processing of the display list extra data (in scene::set_display_list), when it would not be needed. I am not aware of any sites where this would happen, but it felt like a trivial and safe optimization to make.

gfx/wr/webrender/src/scene_building.rs
782 ↗(On Diff #205499)

Yes. And I believe that the code is generic enough so that it can also be used to speed up scene building.

gfx/wr/webrender_api/src/display_list.rs
253

I don't believe it is necessary since gecko is the source of truth for the display list, and it decided to send an item with that cache index.

miko marked 4 inline comments as done.Dec 13 2019, 1:48 PM
kvark accepted this revision.Dec 13 2019, 3:32 PM
kvark added inline comments.
gfx/wr/webrender_api/src/display_list.rs
253

We don't need to put all the trust in one place. Instead, we should defend in depth and double-check in WR whatever inputs makes sense to validate in order for WR to function. The better the boundary between WR and Gecko, the easier our life will be.

This revision is now accepted and ready to land.Dec 13 2019, 3:32 PM
miko updated this revision to Diff 221223.Jan 22 2020, 10:16 PM
miko retitled this revision from Bug 1558926 - WR DL deltas [WIP] to Bug 1558926 - Part 1: Add data structures and pref for display item caching.
cbrindusan reopened this revision.Jan 24 2020, 6:47 PM
This revision is now accepted and ready to land.Jan 24 2020, 6:47 PM
miko updated this revision to Diff 222689.Jan 27 2020, 2:17 PM
miko retitled this revision from Bug 1558926 - Part 1: Add data structures and pref for display item caching to Bug 1558926 - Part 1: Add data structures and pref for display item caching r=kvark.