[Webkit-unassigned] [Bug 184131] Added UI to show potential regressions in chart with t-testing against segmentations.

[bugzilla-daemon at webkit.org]

https://bugs.webkit.org/show_bug.cgi?id=184131

--- Comment #3 from Ryosuke Niwa <rniwa at webkit.org> ---
Comment on attachment 336807
  --> https://bugs.webkit.org/attachment.cgi?id=336807
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=336807&action=review

> Websites/perf.webkit.org/public/shared/statistics.js:113
> +    this.tTestForRangeWithSignificance = function (values, segmentations, significance) {

This is not running t-test. We're finding ranges which reject null hypothesis based on segmentations.
How about findRangesForChangeDetectionsWithWelchsTTest?

> Websites/perf.webkit.org/public/shared/statistics.js:124
> +            const currentTestingIndex = segmentations[i].seriesIndex;

What does testing index mean? I think this is really a change point: https://en.wikipedia.org/wiki/Change_detection
So it should be named as such: currentChangePoint

> Websites/perf.webkit.org/public/shared/statistics.js:126
> +            const leftBound = segmentations[i - 1].seriesIndex;
> +            const rightBound = segmentations[i + 2].seriesIndex;

Use start/end instead of left/right. "Bound" is ambiguous still.
This should be called previousChangePoint and nextChangePoint respectively.

Do we always include 0 and length - 1 in the segmentations?
If not, we'd not detect a change if the graph had exactly one change point.

> Websites/perf.webkit.org/public/shared/statistics.js:128
> +            for (let leftEdge = currentTestingIndex - 2, rightEdge = currentTestingIndex + 2; leftEdge >= leftBound && rightEdge <= rightBound; leftEdge--, rightEdge++) {

Why are we always adding & subtracting two?

> Websites/perf.webkit.org/public/shared/statistics.js:129
> +                const result = this.computeWelchsT(values, leftEdge, currentTestingIndex - leftEdge, values, currentTestingIndex, rightEdge - currentTestingIndex, 2 * significance - 1);

What does 2 * significance - 1 mean here?
Are we converting one-sided probability to two-sided probability or something?
If so, we should store that in a local variable with a descriptive name.

> Websites/perf.webkit.org/public/v3/components/chart-pane-base.js:176
> +        if(annotation.task)

Nit: Missing a space between if and (.

> Websites/perf.webkit.org/public/v3/components/time-series-chart.js:159
> +    setSuggestedAnnotations(suggestedAnnotations)
> +    {
> +        this._suggestedAnnoations = suggestedAnnotations;
> +        this._annotationRows = null;
> +        this.enqueueToRender();
> +    }

I don't think it makes sense for TimeSeriesChart to provide a facility to merge the list of annotations like this.
ChartPane should simply combine the list of analysis tasks & annotations for change point detections in a single array instead.

> Websites/perf.webkit.org/public/v3/pages/chart-pane.js:50
> +            function valueForIndexInSegmentation(index) {

Where is this function ever used?

> Websites/perf.webkit.org/public/v3/pages/chart-pane.js:68
> +                    fillStyle: '#808080',

Put this in ChartStyles as we talked about it in person.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.webkit.org/pipermail/webkit-unassigned/attachments/20180331/a8e6074f/attachment.html>