criterion performance measurements
overview
want to understand this report?
Comparison of foldr1/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 9.203854854831998e-2 | 9.632238473221494e-2 | 0.10113385899917865 |
| Standard deviation | 5.005464177887461e-3 | 7.4130142869778615e-3 | 1.2208053417535511e-2 |
Outlying measurements have moderate (0.207747265723464%) effect on estimated standard deviation.
Comparison of foldr1/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.316596385359547e-9 | 5.376886128757756e-9 | 5.454141996369686e-9 |
| Standard deviation | 1.639221697075381e-10 | 2.2887473233080653e-10 | 3.0916826227769913e-10 |
Outlying measurements have severe (0.6828323456041435%) effect on estimated standard deviation.
Comparison of foldl1/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.2172354124880965 | 0.22907433831084442 | 0.2460787641114924 |
| Standard deviation | 1.2111381141880134e-2 | 1.993913544871972e-2 | 3.3624925518219405e-2 |
Outlying measurements have moderate (0.15711879640219714%) effect on estimated standard deviation.
Comparison of foldl1/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.151522724365962e-9 | 5.249660781420899e-9 | 5.335365784957656e-9 |
| Standard deviation | 2.3173495450920128e-10 | 3.1077809478354604e-10 | 3.8984248432796805e-10 |
Outlying measurements have severe (0.8088258082433845%) effect on estimated standard deviation.
Comparison of foldr1'/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 3.0501775401085444e-2 | 3.2106548627713026e-2 | 3.60499122495513e-2 |
| Standard deviation | 2.0420436365750617e-3 | 5.249350663488086e-3 | 9.73314980935807e-3 |
Outlying measurements have severe (0.632663059908896%) effect on estimated standard deviation.
Comparison of foldr1'/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 2.9211950701076156e-2 | 3.095560223235236e-2 | 3.275656308559528e-2 |
| Standard deviation | 2.656668178316153e-3 | 3.747347981396925e-3 | 5.520332861381254e-3 |
Outlying measurements have severe (0.5118329840133178%) effect on estimated standard deviation.
Comparison of foldl1'/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.980666929011105e-3 | 8.059982226506512e-3 | 8.18454052773054e-3 |
| Standard deviation | 1.8785356370836032e-4 | 2.919645472554574e-4 | 4.568227740548464e-4 |
Outlying measurements have moderate (0.14323141750613225%) effect on estimated standard deviation.
Comparison of foldl1'/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.273360101375969e-3 | 8.361806860249234e-3 | 8.444462693482495e-3 |
| Standard deviation | 1.9214306951780282e-4 | 2.3959380267777055e-4 | 3.01534974323806e-4 |
Outlying measurements have moderate (0.11273892443566923%) effect on estimated standard deviation.
Comparison of maximum/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21993774376369402 | 0.2233435214640849 | 0.2323229029734648 |
| Standard deviation | 8.99961065259954e-4 | 7.246244698525923e-3 | 1.0151109138024344e-2 |
Outlying measurements have moderate (0.1388888888888889%) effect on estimated standard deviation.
Comparison of maximum/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.329929604763976e-9 | 5.389701908976391e-9 | 5.477135692585753e-9 |
| Standard deviation | 1.837249729590498e-10 | 2.4246139563260076e-10 | 3.206469941796648e-10 |
Outlying measurements have severe (0.7045000381654145%) effect on estimated standard deviation.
Comparison of minimum/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.21894921301663997 | 0.22181184365518777 | 0.23192169749866784 |
| Standard deviation | 4.229420345359964e-4 | 6.3743901561394145e-3 | 8.858210582989372e-3 |
Outlying measurements have moderate (0.13888888888888865%) effect on estimated standard deviation.
Comparison of minimum/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.108054314481752e-9 | 5.2369745639477e-9 | 5.333779445834811e-9 |
| Standard deviation | 2.814219553809037e-10 | 3.7451676835069497e-10 | 4.726605195010113e-10 |
Outlying measurements have severe (0.8611170830369371%) effect on estimated standard deviation.
Comparison of maximum'/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.966587638316409e-2 | 6.2275721429229575e-2 | 6.435671196121702e-2 |
| Standard deviation | 3.1197927812963407e-3 | 4.347035497011184e-3 | 5.842337376684378e-3 |
Outlying measurements have moderate (0.23329794310461696%) effect on estimated standard deviation.
Comparison of maximum'/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.0470230453609164e-2 | 6.299384896925393e-2 | 6.501195247587634e-2 |
| Standard deviation | 3.002470804088111e-3 | 4.239677172031121e-3 | 6.685634027021008e-3 |
Outlying measurements have moderate (0.23203407801047396%) effect on estimated standard deviation.
Comparison of minimum'/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.984189732059719e-2 | 6.260533537267308e-2 | 6.48554478255175e-2 |
| Standard deviation | 3.2055747277251983e-3 | 4.583537600205994e-3 | 6.217318076247931e-3 |
Outlying measurements have moderate (0.23481761873570012%) effect on estimated standard deviation.
Comparison of minimum'/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.566979596501875e-2 | 6.067290736319772e-2 | 6.343531031738793e-2 |
| Standard deviation | 2.631612250568646e-3 | 6.6476938237458125e-3 | 9.013368493614046e-3 |
Outlying measurements have moderate (0.39783607822480505%) effect on estimated standard deviation.
Comparison of sum/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.936109577589146e-3 | 9.026516072857852e-3 | 9.156027929503234e-3 |
| Standard deviation | 2.063644880757916e-4 | 2.769749608270454e-4 | 3.9527055837658675e-4 |
Outlying measurements have moderate (0.11689693404095271%) effect on estimated standard deviation.
Comparison of sum/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.182931108220752 | 0.19029345133215733 | 0.2034892989158834 |
| Standard deviation | 6.807743145635437e-3 | 1.359441135035016e-2 | 2.026657666004157e-2 |
Outlying measurements have moderate (0.1524926432233688%) effect on estimated standard deviation.
Comparison of product/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 8.986975547262622e-3 | 9.113910793976748e-3 | 9.354970586265178e-3 |
| Standard deviation | 3.065736262860594e-4 | 4.707000944784152e-4 | 6.995686885427126e-4 |
Outlying measurements have moderate (0.23855881413302862%) effect on estimated standard deviation.
Comparison of product/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 0.16953167311755046 | 0.1784758463811967 | 0.18685629451465502 |
| Standard deviation | 7.760718004583659e-3 | 1.1887728533254335e-2 | 1.6728485251521444e-2 |
Outlying measurements have moderate (0.15036141835053457%) effect on estimated standard deviation.
Comparison of head/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.15440578477852e-9 | 6.317273805877667e-9 | 6.4947877730587214e-9 |
| Standard deviation | 5.352950217665375e-10 | 5.989288531832977e-10 | 7.031316578541986e-10 |
Outlying measurements have severe (0.9159576414811805%) effect on estimated standard deviation.
Comparison of head/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.217669658059682e-9 | 5.319401609802882e-9 | 5.422214427208368e-9 |
| Standard deviation | 2.270980072662795e-10 | 3.1878697619020667e-10 | 4.4090276517654987e-10 |
Outlying measurements have severe (0.8121168865789394%) effect on estimated standard deviation.
Comparison of last/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.3782289997443885e-3 | 7.536869873215573e-3 | 7.6994900701944265e-3 |
| Standard deviation | 3.982062753217269e-4 | 4.8143738651339717e-4 | 6.206984000121286e-4 |
Outlying measurements have moderate (0.3389114832230848%) effect on estimated standard deviation.
Comparison of last/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.291295176596697e-9 | 5.347706874503006e-9 | 5.418504394540684e-9 |
| Standard deviation | 1.540901388429349e-10 | 2.090446409976176e-10 | 2.8090901432289293e-10 |
Outlying measurements have severe (0.6433431488422652%) effect on estimated standard deviation.
Comparison of tail/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 6.278620936019423e-9 | 6.440816415128437e-9 | 6.680288936075243e-9 |
| Standard deviation | 5.293873228524357e-10 | 6.914130937603543e-10 | 1.0648439726699574e-9 |
Outlying measurements have severe (0.9327327445608146%) effect on estimated standard deviation.
Comparison of tail/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.118753341007339e-9 | 5.231377445122373e-9 | 5.3302507267062764e-9 |
| Standard deviation | 2.5888358158039464e-10 | 3.4404853453256965e-10 | 4.552359494508223e-10 |
Outlying measurements have severe (0.8382822324833549%) effect on estimated standard deviation.
Comparison of init/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 7.630764483801614e-2 | 8.385736466800153e-2 | 9.019470574130536e-2 |
| Standard deviation | 6.4357689769059265e-3 | 1.1408304544489567e-2 | 1.9404129238421077e-2 |
Outlying measurements have moderate (0.47889427659264133%) effect on estimated standard deviation.
Comparison of init/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.340172358226782e-9 | 5.409028172047991e-9 | 5.500824214931663e-9 |
| Standard deviation | 1.948550425644063e-10 | 2.6577062699148075e-10 | 3.7953977838395033e-10 |
Outlying measurements have severe (0.7408670495468626%) effect on estimated standard deviation.
Comparison of !?vs!!/Delude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 4.120709610827161e-3 | 4.1725472702124344e-3 | 4.222078587044593e-3 |
| Standard deviation | 1.1237419407402399e-4 | 1.633252366701409e-4 | 2.3317090924915513e-4 |
Outlying measurements have moderate (0.19484833043720806%) effect on estimated standard deviation.
Comparison of !?vs!!/Prelude
| lower bound | estimate | upper bound | |
|---|---|---|---|
| OLS regression | xxx | xxx | xxx |
| R² goodness-of-fit | xxx | xxx | xxx |
| Mean execution time | 5.34877887495846e-9 | 5.406783596061858e-9 | 5.50285279171294e-9 |
| Standard deviation | 1.6455337761516333e-10 | 2.4322187207809423e-10 | 3.4630923317647426e-10 |
Outlying measurements have severe (0.704499221086319%) effect on estimated standard deviation.
understanding this report
In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.
- The chart on the left is a kernel density estimate (also known as a KDE) of time measurements. This graphs the probability of any given time measurement occurring. A spike indicates that a measurement of a particular time occurred; its height indicates how often that measurement was repeated.
- The chart on the right is the raw data from which the kernel density estimate is built. The x axis indicates the number of loop iterations, while the y axis shows measured execution time for the given number of loop iterations. The line behind the values is the linear regression prediction of execution time for a given number of iterations. Ideally, all measurements will be on (or very near) this line.
Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.
- OLS regression indicates the time estimated for a single loop iteration using an ordinary least-squares regression model. This number is more accurate than the mean estimate below it, as it more effectively eliminates measurement overhead and other constant factors.
- R² goodness-of-fit is a measure of how accurately the linear regression model fits the observed measurements. If the measurements are not too noisy, R² should lie between 0.99 and 1, indicating an excellent fit. If the number is below 0.99, something is confounding the accuracy of the linear model.
- Mean execution time and standard deviation are statistics calculated from execution time divided by number of iterations.
We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)
A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.