Last time I showed a comparison of Powertap to Vector power with a least-square line through the points. I noted this was "hard to interpret". Here's why.
A traditional least-square fit begins by assuming that the x-values are perfectly known, the y-values are uncertain. It additionally assumes that the y-values are normally distributed, with a Gaussian probability distribution, with the error distribution the same for all points, or with variances inversely proportional to the weighting factors for a weighted least-square fit.
This seems like a lot of technicality, but it introduces subtle biases in the result when comparing two values whcih contribute relatively equally to the error. For example, consider my power comparison for the Garmin Vector to the Powertap:
The slope was 0.97. Therefore, if I flip the axis the slope should be 1.03%. If the Vector is 3% lower than Powertap than the Powertap is 3% more than Vector, on average. But that's not what I get. I get 99%:
How can that be? The Vector is 3% lower than Powertap and Powertap is 1% lower than Vector?
The answer is in assumptions of the least square fit. Consider fitting the following,where PV is Vector power and PP is Powertap power:
PV = kVP × PP
Here kVP is a coefficient relating the PP power to the PV power.
The error associated with this estimate is thus as follows:
EVP = ∑ ( kVP × PP - PV )2
To set the value kVP to minimize this error, I differentiate with respect to that coefficient:
∂ EVP / ∂ kVP = ∑ 2 PP ( kVP × PP - PV )
= 2 kVP ∑ PP2 − 2 ∑ PP PV
I then set this derivative to zero (the usual minimization or maximization strategy for analytic functions). This yields the following solution:
kVP = (∑ PP PV) / ∑ PP2
Note this is most strongly influenced by points with larger values of PP. In this fitting algorithm, the errors are assumed to be in PV, which is on the "y-axis". PP values are considered "reliable" and are used to weight the sums. However, I had no reason to assume PP were more or less reliable than PV.
If I instead go through the same exercise assuming the Vector powers are reliable and the Powertap values are not, I get:
kPV = (∑ PP PV) / ∑ PV2
The product of these ratios will be 1, as expected, if the data are truly proportional, but if not, the ratio will no longer be one. That means the result depends on which power I assert is the more reliable. If I pick one at random, my conclusion about the ratio of Vector to Powertap power differs. This obviously isn't so good.
Note the culprit is the factor which appears both in the numerator and the denominator. In the kVP formula that is PP while in the kPV formula that is PV. To "balance" the fit between the two powers I should pick a common value for the two formulas. For example, I can choose the sum of the two powers (equivalent in this context to using the average). Thus I get a new pair of coefficients:
kVP' = (∑(PP + PV) PV) / ∑ (PP + PV) PP
kPV' = (∑(PP + PV) PP) / ∑ (PP + PV) PV
It's trivially obvious that PVP' PPV = 1, as I'd expect.
Here's the result of applying this formula to fitting the Vector power versus the Powertap power, comparing to the traditional least-square fit:
A summary of the slopes is as follows:
|PP||0.9696||least-square PV vs PP|
|PV||1.0057||1 / L.S. PP vs P>V|
|PV+PP||0.9874||1 / L.S. PP vs PV|
|1||0.9838||<PV> / <PP>|
Note the first line is the traditional least-square fit. The second line is fitting Powertap versus Vector, but then taking the reciprical to get the opposite. The third is the proposed balanced weighting scheme. Here it gives the same result to listed precision (but not exactly) as the average of the first two results, which is also clearly balanced. The fourth line is a simple ratio of averages. This places equal emphasis on the lower power points. It is also clearly balanced, and gives a result closest to the proposed scheme.
These fits have been to a one-parameter curve: a line of adjustable slope and zero intercept. A more flexible least square fit has an additional intercept term yielding two fitted coefficients. But the same approach can be applied there. The key indicator is that fitting y versus x should yield a slope equal to the reciprical of the slope yielded by fitting x versus y. The formulas described here still apply. First shift the points so the average values of x and y are zero. Then the formulas listed here apply to the slope. Slope is translation-invariant. You can then shift the points back and trivially calculate the intercept.
If the two axes have different units or if the uncertainty in the points is estimated to be different, then the following can be used:
kVP' = ∑(PP / σP + PV / σV) PV / ∑ (PP / σP + PV / σV) PP
where σP is the uncertainty in PP and σV is the uncertainty in PV. If σP = σV then this the new terms obviously cancel.