From:  Greg Tatum <gtatum@mozilla.com>
Date:  03 Jan 2018 23:07:39 Hong Kong Time
Newsgroup:  news.mozilla.org/mozilla.dev.developer-tools
Subject:  

Re: Q4 performance work

NNTP-Posting-Host:  63.245.214.181

Thanks Alex for all of the work you and other DevToolers did on this! This
work is hard to see but the impact is huge. Also thanks for taking the time
to report on and quantify the work that was done.

On Thu, Dec 21, 2017 at 4:28 AM, Alexandre Poirot 
wrote:

> Tl;dr:
>
>    -
>
>    Tools update faster on page reload: Netmonitor is 40% faster and console
>    7% faster when reloading our reference test page (bild.de).
>    -
>
>    DAMP (DevTools perf test on Talos) is documented, is 2 times less noisy,
>    can be easily profiled with specific markers, has a dedicated dashboard,
>    covers more user interactions.
>    -
>
>    Background Hang Reports show improvements on long (>=512ms) hangs
>    related to DevTools. 2 times less related to console hangs, and
> reduction
>    of hangs over 1s related to netmonitor.
>
>
> Let’s review in details the Q4 goals we had defined for DevTools
> performance.
>
> Ship a refined version of DAMP that fixes known issues and is a trusted
> leading indicator for performance
>
>    -
>
>    Console, inspector and netmonitor reload tests now wait for full panel
>    update and not only page reload. Debugger test for page reload is still
> not
>    perfect, but yulia is on it.
>
> Netmonitor update time when reloading bild.de
>
> Console update time when reloading bild.de
>
>
>    -
>
>    Documentations about DAMP and performance:
>    -
>
>       General DAMP documentation
>       
>       -
>
>       How to write a perf test
>       
>       -
>
>       How to work on performance (to be landed
>       )
>       -
>
>       Use the profiler against DAMP
>       -
>
>    DAMP is two times less noisy and now catches more subtle regressions
>    -
>
>       Average stddev between two test run is 1.78% (was 4.43%) and max
>       stddev is 7% (was 22%).
>       -
>
>       This was done by forcing garbage collection
>        between tests
>       to prevent having it done during tests
>
>
>    -
>
>    A dashboard  was
>    created to help track DAMP, telemetry and BHR results
>
>
> To be done:
>
>    -
>
>    Track more interactions on DAMP
>    -
>
>       Inspector
>       -
>
>          cover inspect element from context-menu with/without toolbox
> opened
>          
>          -
>
>       Console
>       -
>
>          cover adding a console log while there is a lot of messages
>          already logged
>          
>          -
>
>          cover message filtering
>          
>          -
>
>       Netmonitor
>       -
>
>          cover request expand
>          
>          -
>
>       Debugger
>       -
>
>          Wait correctly for panel update on page reload
>          -
>
>          Cover times to hit a breakpoint and update the UI
>          -
>
>          Select a source file and display it
>          -
>
>          Step, resume
>          -
>
>    Split damp.js in multiple files. Adding many perf test doesn’t scale
>    with the current setup. Eventually have damp.ini file to ease
>    sheriffing/handle DAMP test intermittents.
>    -
>
>    Make DAMP run faster (it still spends a lot of time doing python things
>     at every run)
>    -
>
>    Keep writing documentation, especially about the typical slowness that
>    exists in DevTools and how to fix them.
>
> Instrument panel loading and top level user interactions (definitions
>  5Rio_8ryXyk/edit#heading=h.84rq0lfg4vp4>)
> in the inspector, debugger, console, and network monitor.
>
>    -
>
>    Toolbox open
>     aggregates=median&cumulative=0&end_date=2017-11-19&keys=
> webconsole!webconsole!netmonitor!jsdebugger&max_channel_version=nightly%
> 252F59&measure=DEVTOOLS_COLD_TOOLBOX_OPEN_DELAY_MS&min_
> channel_version=nightly%252F59&processType=*&product=
> Firefox&sanitize=0&sort_keys=submissions&start_date=2017-
> 11-13&trim=1&use_submission_date=0>
>    and panel updates
>     aggregates=median&cumulative=0&end_date=2017-11-05&keys=
> inspector!webconsole!__none__!__none__&max_channel_version=
> nightly%252F59&measure=DEVTOOLS_TOOLBOX_PAGE_RELOAD_
> DELAY_MS&min_channel_version=nightly%252F59&processType=*&
> product=Firefox&sanitize=0&sort_keys=submissions&start_
> date=2017-10-24&trim=1&use_submission_date=0>
>    on page reload is being recorded (still missing for debugger)
>    -
>
>    Goal’s document list
>     5Rio_8ryXyk/edit#heading=h.84rq0lfg4vp4>
>    has been discussed and we came up with this other list
>     N1fke4SO7D3m0o/edit#bookmark=id.i0k79epbc7l5>.
>    Still pending a final confirmation. We should probably use the new list
> in
>    further documents.
>
>
> To be done:
>
>    -
>
>    Telemetry data is remarkably flat over time.
>    -
>
>       We should review existing probes and dashboard output to verify that
>       telemetry durations are reduced when landing significant improvement.
>       -
>
>    All top interactions that are not toolbox opening and page reload should
>    have a related telemetry probe.
>
> Reduce the inspector, debugger, netmonitor and console's open time and
> refresh time on page reload.
>
>    -
>
>    I worked on speeding up overall toolbox opening in Q3, but there has not
>    been any focused efforts to speed up the open time
>    -
>
>       Update Debugger frontend (10-8)
>        improved it
> on
>       debugger by 20% on DAMP.
>       -
>
>       Split devtools/shared/client/main.js into multiple components
>        speed up
>       console open time by ~5% on DAMP
>       -
>
>    A lot of work has been spent on improving time to refresh after page
>    reload
>    -
>
>       Netmonitor: a series of improvements on DAMP: -13%, -5%, -3%, -6%,
> -5%
>       -
>
>       Console improvements on DAMP: -4% on complicated page reload, and
>       -43%, -75%, -12%, -3% on custom test cases (close-after-dir,
> expand, cached
>       messages)
>       -
>
>       Inspector improvements on DAMP: -40% and -21% on custom test cases
>       (mutations and open layout)
>       -
>
>       Debugger miss DAMP coverage, so I can’t state any win for now.
>
>
> To be done:
>
>    -
>
>    Carry on improving tools update when reloading pages. I feel that we
>    only started and there is a lot more we can do.
>
>
> Fix all hangs caused by devtools over 2s and most hangs over 512ms, using
> BHR for tracking.
>
>    -
>
>    Half as many hangs between 512ms and 2s related to the console (related
>    to a simple CSS fix
>     to prevent using
>    flex)
>
>
>    -
>
>    Debugger regressed significantly in late October. Around 4 times more
>    hangs. Probably related to this bundle update
>    , or one
>    before/after this one.
>
>
>    -
>
>    Still to be confirmed, reduced hangs over one second related to
>    netmonitor. Starting between Nov 22th and after. Thanks to immutable
>    removal and lazy loading of netmonitor data.
>
> What is sure is that we got rid of all the hangs related to Immutable that
> were coming from netmonitor:
>
> Thanks to that, netmonitor selectors are slightly faster:
>
>
> To be done:
>
>    -
>
>    Release tracking pages specific to each panel.
>    -
>
>       Today, we have a tracking graph for all Firefox and another one for
>       all DevTools. But it will be useful to track the progress for each
> tool.
>       -
>
>    At the beginning of Q4, netmonitor stacks were by far the most common
>    hangs (~80%). These days netmonitor, console and debugger are about 30%
>    each. We should probably switch our focus to Console and Debugger.
> _______________________________________________
> dev-developer-tools mailing list
> dev-developer-tools@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-developer-tools
>