Tuesday, August 9, 2011

Strava Suffer Score decoded

Strava recently debuted its "Suffer Score", which attempts to quantify in one way how hard an activity was. Suffer score, they claim, is based on heartrate: ride further or harder and suffer score is higher.

Suffer score is a good move for Strava. Strava's market segment is informal competition. Other web sites have logged how far people ride, and where, but Strava really locked into the demand for competition: competition via social networking. To date these rankings have been based primarily on speed over "segments": routes defined by users, for example on roads or trails, typically up climbs. They then added contests for volume: most miles ridden in a week and most feet climbed, for example. But with Suffer Score, they combine the two: a metric combining both quantity and quantity.

Why's this important? Of course not everyone can climb fast enough to compete for KOMs on popular climbs. But a heartrate-based metric levels the playing field to a large extent. Anyone who's been riding regularly for long enough can go out with the goal of generating an "epic" suffer score for the day.

Suffer score is an obvious correlary with the Coggan "training stress score" (TSS). TSS is an attempt to quantify the training stress of a ride: it's effect on fatigue and its stimulus for physiological adaptation. That both of these can be captured with the same simple metric is far from obvious, but TSS is a blunt instrument, and its popularity among hardcore power meter fans proves people find it useful.

But Strava's focus isn't on hard-core training junkies. It's on the typical Mission Cycling or SF2G rider: people who like to ride, and when they ride like to ride quickly, and as a consequence of this quick riding tend to acquire fitness. However, the goal of every ride is still the joy of the ride, not strictly to improve fitness. So "suffer score" is a "don't take me quite so seriously" number by which long-hard-rides can be compared. "Pain is good", it suggests, not only for it's "training stress".

The big question, however, is how Suffer Score is calculated? To test this, I used a Perl script to generate ride data at constant heartrate. My "rides" consoisted of perfect 1 km radius circules in the Bonneville Salt Flats ridden at a perfect 10 meters per second. Each ride had a different heart rate, perfectly constant, or a different duration.

First I tried using the GPX file format for this purpose, but Strava wouldn't digest the heartrate numbers. So I switched to TCX. This worked nicely.

Here's one of my rides:


From a few tests, I was able to guess the following formula was used for Suffer Score:

Suffer score = K1 t1 + K2 t2 + K3 t3 + K4 t4 + K5 t5

where:
t1 = time in hours in zone 1
t2 = time in hours in zone 2
t3 = time in hours in zone 3
t4 = time in hours in zone 4
t5 = time in hours in zone 5

With coefficient roughly equal to the following (may be off by 1/hr):
K1 = 12/hr
K2 = 24/hr
K3 = 45/hr
K4 = 100/hr
K5 = 120/hr

I like this approach, although I'd be tempted to increase K5, for example to 200. Time in Z5 is very, very painful. Of course this depends on having a good number for maximum heart rate, which may not be the case. In fact a geometric progression would have worked well here: 12, 24, 48, 96, 192. I think this would have helped the criticism I've been reading that it's too endurance-focused.

37 comments:

Andrew R. Coggan, Ph.D. said...
This comment has been removed by the author.
djconnel said...

Whoops -- I'll fix the blog (I don't know where "bias" came from -- doing too many semiconductor device simulations for the day job).

To be fair, then did mention TRIMP in the support forum. So they're not practicing active deception (benign neglect if anything). Not sure about the TSS descriptions.

I still need to check the formula against time gaps in the ride to see if they're doing any time averaging. So far I checked linearity only against rides of constant HR. Spent too much time debugging my ride data generation script.

Jason said...

Do you mind posting a link to your perl script that generates the TCX file?

I won't judge your coding, I just want to save myself time from doing fun experiments like these on my own with Strava.

Mark Shaw said...

Hi Andrew, this is Mark Shaw from Strava. You're absolutely right, we should make reference to TRIMP in our "What's this?" section, as it is correct to say we're using a modified TRIMP score. It wasn't our intention to hide this fact and it was an oversight not to mention it directly on the site. As Dan points out, our intention is to be open about this, as we do discuss it on support forums. We've been getting a lot of feedback from our users which we're using to help make adjustments to the score and the language around it. We'll add a mention of TRIMP to our site - thanks for the heads up.

Mark Shaw said...

Hi Andrew, this is Mark Shaw from Strava. You're absolutely right, we should make reference to TRIMP in our "What's this?" section, as it is correct to say we're using a modified TRIMP score. It wasn't our intention to hide this fact and it was an oversight not to mention it directly on the site. As Dan points out, our intention is to be open about this, as we do discuss it on support forums. We've been getting a lot of feedback from our users which we're using to help make adjustments to the score and the language around it. We'll add a mention of TRIMP to our site - thanks for the heads up.

djconnel said...

Maybe I'd better not post the script publicly -- not really fair to Strava.

Péter I. Pápics said...

1st of all, very nice post, I was 100% sure that you would figure this formula out when I saw the SSS on the Strava website ;)

I completely agree with giving zone 5 a much higher score, and I like the idea of the geometric progression, so that's how I included it in my personal workout analyzer script:

k1...k5 is 12.5, 25, 50, 100, 200
while the categories are
leisurely 0-49
moderate 50-99
tough 100-199
very hard 200-399
mind-blowing 400-

It is really a pity that there are basically no Strava users in Europe... :(

Péter I. Pápics said...

Ps.: what I wrote earlier might be good to rate the toughness of the ride, but it should not be mixed with TSS values. I did not mean that. Now that you wrote about it I read a bit into these things, and I have found lots of interesting stuff, thanks again!

thehomme said...

Great post. Since going premium I've been confused by the suffer score. I do a very, very hard race of 1.5hrs with almost all my time in Z4 but I get a greater SS on a 4 hr club ride spent in z2 & z3. The race caused a lot more "suffering"

Why can't Strava just give us a TSS and IF score. And ideally all the analysis you get in WKO+ for a "super premium" membership fee?

djconnel said...

The terms "TSS" and "TSB", etc, are trademarked, so they can't be used without license. But I think Strava views its market as different: not focused on power meters, but more the average rider who just likes to ride hard without formal metrology. So it's intended to be a "fun" number. They've upgraded their metrology to include a "red zone" score, which is the component of "suffer score" from Z4-Z5. Actually, it would make a good API project to extract a TSS-like statistic from Strava data. I've got a bunch of projects on my plate but that wouldn't be too hard.

thehomme said...

There's no doubt Strava is a broad mainstream product. I can understand that as a business strategy at start up and beyond but there's a huge gap in the market for a single cohesive system.

There's currently very little you get from Strava premium. Most people I know pay cos they think they should rather than for what Strava provides. If they want to drive premium membership then they need to be looking to add significant functionality for the growing numbers of club cyclists. To my mind these are the people who will pay...if you just enjoy riding your bike then why go premium?

My ideal product would be trainer road, something akin to WKO+ all integrated with Strava.

Its funny the comment about no one being on Strava in Europe. It seems to be hitting a tipping point here in the UK.

djconnel said...

Yes -- they've made a big marketing push for Europe!

GoldenCheetah has some Strava integration. I've not followed it, but I suspect it does what you want. Try that out...

Strava has a simple API which makes it easy for apps to upload or download rides to Strava accounts. This makes it fairly easy to implement alternative front-ends. It will become simpler when version 3 of the API comes out within the next few months.

For example, I'd like to do an app to extract critical power curves from Strava rides. It's easy to do for one ride, harder if you want some sort of global curve. But GoldenCheetah already offers this capability.

Sourceminer said...

Awesome Post Dan, thanks for the reas up. Recently I have been perplexed with the idea of renewing my subscription of Strava as I love the Golden Cheetah program. The Suffer score I am sorry to see only works with a heart rate, they dont include it in a power meter which I believe should be. I hope to see more meaningful updates to Strava for the people that pay for a subscription.

Thanks for the great write-up!

djconnel said...

I fully agree: there's no reason they couldn't do a suffer score with power meter data. Perhaps they wish to limit the computational burden, since they're dealing with their exponential increase in users as their user base has expanded from it's previous focus in the San Francisco Bay area. In running they use pace for analysis.

Tina said...

Thanks for the INFO! Very few of us on STRAVA in Saigon, it helps with the motivation. I moved my max HR up by 2 beats, found that I am in much better shape, but get home from a 130km ride and I get a "tough".... would have been "extreme" (nearly 50 points more) for similar rides this time last year.

Any ideas about the Out and Back issues? I have emailed the company and there seems to be no fix in sight. Instead, people cut the course short and get outragous speeds. We really have about 80km or good riding in Ho Chi Minh City where we can really just let it out. So, this error is a bit of a shame!

thanks!

Tina said...
This comment has been removed by a blog administrator.
djconnel said...

Thanks, Tina! I have been to Saigon and really liked it. I didn't try cycling there, but other places in Vietnam, like De Lat 300 km to the north, seemed like excellent cycling locations.

On the out-and-back problem I have spoken to Strava in person about that and I expect it will be solved eventually. The problem is that the matching between the start and end point is relatively loose. So if you do an out-and-back, and match the start and finish (the same point), then if you have visited 80% of the points in between (80% of the route, cutting short 20%), then it might match you. What they need to do is to add in special checkpoints which must be matched in addition to the start and finish. For an out-and-back that would be the turn-around point. I had expected they would have done this by now but I think their priorities have been elsewhere, for example on handling the enormous number of new users.

I recently ran a 10 km race and it gave me credit for the 10 km race segment using only 8 km of my data: after 8 km I passed close to the finish line and it concluded I had matched the segment. So my time for that segment was extremely good... same problem.

Joao Frazao said...

Suffer score = K1 t1 + K2 t2 + K3 t3 + K4 t4 + K5 t5

where:
t1 = time in hours in zone 1
t2 = time in hours in zone 2
t3 = time in hours in zone 3
t4 = time in hours in zone 4
t5 = time in hours in zone 5

With coefficient roughly equal to the following (may be off by 1/hr):
K1 = 12/hr
K2 = 24/hr
K3 = 45/hr
K4 = 100/hr
K5 = 120/hr

ok... but hr means the total of hours spent in the exercise??

djconnel said...

Total hours during which HR was in those particular zones and data were recording. So if it jumps from zone-to-zone, they estimate time in each zone.

Joao Frazao said...

I still not understand!
so if k1= 12/hr
and hr means the time spent in the zone and then i multiply for t1 that is time spent in zone 1... the in the final i have just 12+24+... because the hr cut t and just leave the score for each zone??
I know that some of my cal is not correct, but what??? pls give an example!

K1 t1 + K2 t2 + K3 t3 + K4 t4 + K5 t5

where:
t1 = time in hours in zone 1
t2 = time in hours in zone 2
t3 = time in hours in zone 3
t4 = time in hours in zone 4
t5 = time in hours in zone 5

With coefficient roughly equal to the following (may be off by 1/hr):
K1 = 12/hr
K2 = 24/hr

Joao Frazao said...
This comment has been removed by a blog administrator.
djconnel said...

No: (12/hr) is a factor with units per hour. So if you multiply (12/hr) by (2.5 hr) then the hours cancel, and you get 30. If you multiply (12/hr) by (250 minutes) you need to multiply by (1 hr / 60 minutes) and now the hr, minutes both cancel and you are left again with 30. You do this separately for each HR zone, counting the number of individual seconds spent in each, and you end up with total suffer score.

Unknown said...

Nice analysis! Have you delved into how they calculate power at all?

djconnel said...

I am not certain how power is calculated, but unless rider weight + bike weight on Strava = the weight of everything together, including water bottles, water, food, clothes, helmet, shoes, tools, etc, the power calculation will be biased low. The issue here is I suspect most users use their bare bike mass and their naked body mass, so power estimates tend to be too low.

philgo20 said...

thanks for this!

I agree with thehomme and yourself: time spent in z2 should have less incidence on suffer score and time spent in z4 and z5 should be even higher.

Some fairly easy endurance long runs get higher SSS than tempo runs and hill repeats.

Joao Frazao said...

very work to find this power (watts) algoritm
the results are very close to strava em endomondo
http://www.ihpva.org/HParchive/PDF/hp48-1999.pdf

calories =(((2,706*total weight in kg*Velocity kph*%elevation)+(1,247*10^(-2)*Cda*(velocity kph+wind kph)^2*velocity kph)+(2,706*Crr*total weight in kg*velocity kph))/0,95)*3.90265912226099*time hrs *2

Watts = Callories* 1000 / 3600

crr for mtb = 0,0046
Cda = 1 * 0.0293 * (Height in mts ^ 0.725) * (Pweight kg ^ 0.425) + 0.0604 + 0.2645

Franz Kelsch said...

Have you written about the other new Strava premium features when you have a power meter, such as the fitness score, training load and fatigue?

djconnel said...

Not much, Franz. TIme is limited for these projects, unfortunately. Most of my blog writing is done on crowded Caltrain commutes.

But the training score, fitness, and fatigue are very similar to CTL, TSB, and ATL from TrainingPeaks. Those parameter names are trademarked. GoldenCheetah has similar metrics. The issue is you need to use a power meter every ride. I didn't use one during the Berkeley Hills Road Race, a considerable physical stress, and so that shows up as a total rest day in my metrics. It takes months to flush that omission from my scores.

At GoldenCheetah I proposed an algorithm that a regression could be done on distance and climbing of existing rides to estimate these scores for riders without power data. Strava could also use estimated power. Anything is better than assigning zero score to a ride without power meter data, assuming there's no data from car or train rides, for example, contaminating these.

oscar said...

Hi Dj,

I am confused. If T1 is equal to the number of hours in zone 1 and HR is equal to the number of hours in zone 1, then T1 = HR.

So, for axample (and examples are sorely needed), if I walk 30 minutes in zone 1 then T1 = .5 and K1 = 12/.5 so T1 * K1 = 12

djconnel said...

"hr" is units "hours". So it's not a variable, but a unit constant.

oscar said...

thanks for the quick reply, but HR hours in that particular zone and Tx hours in that particular zone?

djconnel said...

t1 is the time in zone 1. I say "in hours" to be easier for people to understand, but really that's redundant. Time has units of "hours" but it can also have units of "minutes" or "seconds", etc. Suffer score has no units. So I need to multiply t1 by something which has units of 1/hours. Then the hours will cancel (one in numerator, one in denominator) and I'll be left with something with no units, which is what I want.

If I were to specify time in minutes instead of hours then I'd need to multiply by a unit conversion factor, for example (1 hour / 60 minutes).

This is common in physics. For example, if I have a velocity (units distance/time) and multiply it by a time (units time) the times cancel and I'm left with distance, which is what I want for riding a particular speed for a particular time. When I put the multiplier in units 1/hr, you're guaranteed to get the correct result.

It's always important to keep track of units. Suffer score, in order to be simple to understand, is presented without units: you can't say "my suffer score was 100 hours" or "my suffer score was just over 4 days" or "6000 minutes".... it's just 100.

Anonymous said...

Hello!

Nice interesting article!
Your decoded SS formula seems to work well for bike rides (http://www.strava.com/activities/213873617/heartrate)

But I'd say Strava calculates SS differently for running, see for example this workout:
http://www.strava.com/activities/215099305/heartrate
Only 25 minutes in Z4 and 15 in Z3 got me SS of 130.
Could You please also try to decode running SS?

djconnel said...

Thanks for the comment! If I run the cycling calculation for your run I get 46.03, but Strava reports 129. So you're right -- it needs to be recalibrated! This could be done easily enough with several runs with different HR distributions. But that rate of inflation, which is a factor of 2.8 in your case (and this may be all there is to it: an inflation by a factor 2.8) seems high. Consider Ironman results: the average of the top 3 was 4:27 on the bike and 2:51 on the run, a ratio of 1.56. If Ironman is balanced then the ratio should be closer to this, assuming average heartrates are somewhat similar. I'd need to look at a few activities to isolate coefficients though (it's a simple algebra problem).

Anonymous said...

Hi! I was wondering for some time, how I can get SS of over 200 in an hour when running (my best was 247/245pir in 51minutes in 10km race), but biking uphill and giving my best would only get me about 120/100pir in about two hours (of which about an hour was working at full going up).
I'm also pretty sure I have zones set about right. I'm estimating my 1 hour max HR(FTP HR) to about 161 in it is in the middle of Strava's Z4(156-167), and getting into Z5(>167) is very hard (on bike I've been there only few times). Getting over 170 for me is "suffer as hell" :)

djconnel said...

I think "suffer score" is fairly arbitrary -- it's whatever Strava decided it should be based on their participants. I'm not sure comparing running to cycling is valid on any objective basis other than they took some sort of average of their cycling and running activities.

Anonymous said...

Well the way I see it - suffer score is (or rather should/could be) an estimate of how hard You were working (whatever that might be) based on Your HR.
It doesn't really matter whether You were running or cycling or doing something else - You got Your "pump" (Your heart that is) working hard. In this regard different activities could potentially be compared in terms of how much stress they caused to You heart.
Unfortunately Strava obviously doesn't have the same view on SS, because it is a lot harder (in terms of heart stress) to get high SS in cycling then it is in running.
Some kind of normalized SS would be much more informative if You could (provided You have Your zones set correctly) only get SS of maximum around 100 in one hour - that would correlate with FTP power/Lactate Treshold.
You could get Your heart working harder for shorter time, of course, or maintain hours of less stress, but getting over 100 in an hour would mean, You got Your zones set too low...
In fact according to my experiences getting SS for Cycling over 100 in an hour is almost impossible for me, but for running i can get over 250 in less then an hour...