As I noted, when dealing with GPS problem cases in the Portola Valley Short-Hills version of the 2013 Low-Key Hillclimbs, I couldn't help but notice every one of the cases I grappled with was an Edge 500. This is anectdotal, so I wanted to take a closer look at the problem.
The initial plan was to scrape the HTML from the Strava pages with a Perl app, since the API doesn't provide computer type, but when this didn't work out for me since Strava requires user authentication to see this info bit I started thinking about PHP options but finally when I couldn't sleep last night I just went to the pages sequentially and transcribed the computer identifier from the browser. Brute force. Not elegant. I feel so dirty.
There were 69 riders @ Portola Valley who each reported the URLs of their Strava records. I then compared these using the root-mean-square average of the distance from the center of the lines the riders triggered the lines (units: meters). The ideal number isn't zero, because my lines are generally in the center of the road and riders are to the right, and in any case there's other sources of error than rider GPS, but "perfect" would probably be 2 meters or so.
Here's the average by computer type:
I am told Scott Byers was using an Osync Nav2Coach, which Strava failed to identify. That unit clearly works extremely well: his score was close to ideal. Indeed, it's a testiment to the superb registration of Google's satellite maps, based on which I placed the lines.
It's not as bad as it looks for Edge 500, though. There's plenty of decent results from Edge 500. It's just it has a virtual monopoly on the really bad results.... curiously along with one screwed-up Joaquin from an Edge 510.
Detailed results (hopefully this renders properly) with numbers linked to Strava activities:
|1||326||2.74391||Jeff Shute||Edge 305|
|2||49||2.85231||David Collet||Strava iPhone app|
|3||37||2.90553||Scott Byer||Osync Nav2Coach|
|4||58||3.11199||Andy Crews||Edge 705|
|5||132||3.17032||Stefano Profumo||Forerunner 405|
|6||156||3.22356||Todd Studenicka||Strava iPhone app|
|7||23||3.24592||Daniel Aminzade||Edge 800|
|8||204||3.34927||Bryn Dole||Edge 510|
|9||407||3.40899||Brandon Iles||Edge 500|
|10||65||3.41488||Giles Douglas||Edge 800|
|11||212||3.75249||Peter C Ingram||Strava iPhone app|
|12||125||3.90218||Frank Paysen||Strava Android App|
|13||83||3.9244||Rich Hill||Edge 500|
|14||53||4.12421||Tracy Colwell||Strava iPhone app|
|15||203||4.24395||Kevin Colagiovanni||Edge 500|
|16||12||4.29885||Will von Kaenel||Edge 800|
|17||411||4.32206||Lucas Pereira||Edge 705|
|18||412||4.32909||Kieran Sherlock||Edge 500|
|19||105||4.59216||Doug MacPherson||Edge 305|
|20||404||4.67041||Heidi Fraser||Edge 500|
|21||408||5.10658||Tom K.||Edge 510|
|22||152||5.14333||Daryl Spano||Edge 800|
|23||327||5.26018||Brandon Smith||Edge 510|
|24||150||5.27501||Gregory P. Smith||Strava Android App|
|25||413||5.36954||Liam Sherlock||Edge 705|
|26||414||5.59722||Tim Sullivan||Edge 500|
|27||415||5.68787||Jeff Weitzman||Edge 800|
|28||230||5.8452||Kris McQueen||Edge 500|
|29||171||5.94689||Phil Lovaglio||Edge 500|
|30||402||6.17513||Chris Evans||Edge 510|
|31||48||6.20068||John Clarke||Edge 510|
|32||147||6.41201||Marty Scott||Edge 200|
|33||401||6.42501||Gino Cetani||Edge 200|
|34||316||6.89468||Bogdan Marian||Edge 510|
|35||71||6.96707||Stephen Fong||Edge 500|
|36||166||7.1123||William Yee||Edge 510|
|37||135||7.35438||Mihai R.||Edge 500|
|38||207||7.46039||Robert Easley||Edge 800|
|39||95||8.01097||Mark King||Edge 500|
|40||27||8.66326||Kate Bergeron||Edge 800|
|41||223||8.72471||Eva Silverstein||Edge 500|
|42||410||8.8267||Paul McKenzie||Edge 800|
|43||403||8.96959||Scott Frake||Edge 500|
|44||35||8.97069||Sugar Brown||Edge 500|
|46||133||11.6204||Alec Proudfoot||Strava Android App|
|47||14||11.7634||Rich McLovin Brown||Edge 500|
|48||406||12.9423||Martin Hyland||Strava iPhone app|
|49||328||13.2219||Ray Smith||Edge 500|
|50||160||13.2689||Luis Valente||Edge 500|
|51||151||13.9149||Kevin M. Smith||Edge 500|
|52||304||14.4409||Paul Cothenet||Edge 500|
|53||31||16.0769||Blue Brown||Edge 200|
|54||126||16.2337||Lisa Penzel||Edge 705|
|55||161||16.307||Greg Watson||Edge 800|
|56||114||17.2566||Shahram Moatazedi||Edge 500|
|57||79||18.3404||Bill Harkola||Strava Android App|
|58||73||19.1305||Chris Furgiuele||Edge 500|
|59||301||19.4436||Amy Bruski||Edge 500|
|60||300||20.7875||Billy Bob Brown||Edge 500|
|61||32||21.6406||Haba?ero Brown||Edge 500|
|62||400||30.3386||Michael Andalora||Edge 500|
|63||405||49.0466||Bruce Gardner||Edge 500|
|64||98||50.2294||Michael Kowalchuk||Edge 510|
|65||409||54.9151||Bill Laddish||Edge 500|
|66||122||109.659||Bart Niechwiej||Edge 500|
|67||209||146.149||Janet Gardner||Edge 500|
|68||318||186.928||Trish Pacheco||Edge 500|
|69||130||233.005||Mark Powers||Edge 500|
So 11 of the worst 12 are Edge 500's. In contrast only 1 of the best 12 are Edge 500's.
31 of 69 are Edge 500's, so the probability of N out of 12 being Edge 500's, by luck alone, are (using the binomial distribution; Poisson statistics aren't good enough for Low-Key):
So the probability of, with luck alone, of no more than 1 in the first 12 being Edge 500 would be 2.4%. The probability of at least 11 of the final 12 being Edge 500 is 0.44%. The combined probability of both of these occurring is 0.011%.
My pick of the number 12 was a biased pick so this isn't really a fair comparison. But it's fairly clear the Edge 500 is particular prone to position error. This is perhaps not representative of new Edge 500's.
The Edge 500 was the most popular computer with 31. The Edge 800 was second, with 9. The third most popular was the Edge 510, with 8. If I do a ranking of all of the results, considering only Edge 500 and Edge 800, there are 40 total. In that ranking the Edge 800's rank 1, 3, 6, 9, 11, 16, 18, 20, and 28. So in that ranking, of the top 20 computers, 8 are Edge 800 and 12 are Edge 500. Of the bottom 20 computers 19 are Edge 500 and 1 is Edge 800.
Suppose I distribute 9 Edge 800's at random among 40 ranked slots. What's the probability at most 1 would be in the 20 lowest ranking slots (and at least 8 in the highest 20 ranking slots)? The number of ways to distribute 0 in 20 and 9 in 20 is 167960. The number of ways to distribute 1 in 20 and 8 in 20 is 2519400. So the number of ways to do either of these is the sum: 2687360. The number of ways to distribute 9 in 40 is 273438880 . The ratio is 0.983%. So the chance of this happening at random is 0.983%. This strongly suggests the Edge 800 is more accurate on average than the Edge 500. However, you can find plenty of good Edge 500 results.
So I establish the Edge 800 is likely better than the Edge 500. Is it better or worse than the Edge 510? 17 of the computers were either Edge 800 or Edge 510. OF those, the Edge 510s ranked 2, 5, 7, 9, 10, 11, 12, and 17. The Edge 800's ranked 1, 3, 4, 6, 8, 13, 14, 15, and 16. The Edge 800 did slightly better but it's too close to conclude anything from this.
There were 6 different Edge units at the Low-Key. There were 5 iPhones. The iPhones did better than 5 of the 6 Edge units; the 2 Edge 305's did better than the iPhone. Between the Edge units and the iPhones, there were 62 activities. The 5 iPhone activities ranked 2, 4, 9, 11, and 42 of 62. So of the top 11, the there were 7 Edges versus 4 iPhone apps. Of the bottom 51 there were 50 Edges and 1 iPhone apps. I won't calculate the probability of this occurring by chance: it's small.
It's interesting, because you'd expect a phone carelessly shoved into a pocket would be inferior to a specifically designed head unit mounted lovingly on the handlebars. On the other hand, the phones have two advantages. One is they are large. Larger = more room for an antenna. The first iPhone was infamous for its poor GPS antenna. I am told the antenna placement in the iPhone was late in the design process, so it was made to fit in available space, rather than being placed early in the process for better optimization. But iPhone users tend to upgrade their hardware, and I doubt there were any early-generation iPhones represented here. The other advantage phones have is they can access the cell towers and use those to help with position determination. Even if no GPS satellites are available, if the phone has access to at least 3 cell towers it can get a position fix. I don't know how much power the phones versus the Edge units are willing to devote to the GPS circuits.
Comparing the phones, the iPhone did a bit better than the Android, but I am reluctant to draw too many conclusions since Android runs on so many different hardware designs.
So lots of interesting stuff here. The conclusion is among the Edge units, the 500 has the most trouble. The 800 is clearly better than the 500 with high probability, and the 800 and 510 are close. By inference the 510 is better than the 500. The iPhone app does well, even in comparison to the Edge units. And older Edge models (the 305 and 705) seem to do about as well as the newer ones. There were no Edge 810's in the mix.