Saturday, January 28, 2012

writing my Strava Android app, part 2

More progress...

First, I did more sketch work on my proposed pages for the app. One of these involves data plots: plotting the altitude, and on a separate plot, speed from a Strava activity on a graph. This is the biggest challenge of my GUI design: the rest consists of a text widget, a bunch of button widgets, and some labels, with more simple widgets in the "configuration" screen. Plots with limited pixels are a challenge: I believe the screen resolution is only 480 × 800 for which only a small subset, for example 400 × 100, will be available for each plot. Not so bad, actually, but real estates needs to be used efficiently. I'll avoid doing anything fancy: no scrolling or zooming, for example. The goal is just to identify portions of the ride in a clearly identifiable fashion. For this I can either do my own plots with graphic primitives or use a more general purpose plotting package. I'm not yet decided on this.

Second, I looked into the Strava API itself (it's a work in progress so I resist putting the link here). It's not nearly as bad as I feared. The Strava API consists of specific GET and PUT transactions, using the appropriate URL's, to read and write JSON data to the servers. JSON stands for "JavaScript Object Notation". The advantage of this is it is effectively language-indendent. For example, CPAN has a nice JSON package for Perl, my favorite scripting language. However, I will resist the temptation to do this project in Perl: Java is definitely the way to go. Perl is great for quick-and-dirty projects but for stuff intended for distribution on Android Market I should toe the line and follow best practices, which means Java. (Note: this won't stop me from at some point whipping together a quick script in Perl to export ride data to CSV format, something which I'll find personally useful). Previously I'd been a bit intimidated because the Strava API examples are in Ruby, a language of which I know virtually nothing. But Java should work equally well. The challenge with the API will be writing data: I still need to experiment with that. Worst case is the app will write data to a new activity, then leave it to the user to delete the old one to avoid duplication. But I strongly suspect I'll be able to work out activity deletion or even direct data substitution as well.

Here's an example of what version 1 of the Strava API produces: JSON data of a recent short ride I did. It's fairly straightforward, if inefficient. For example, were I to want to download all data for all crossings of the Golden Gate Bridge for statistical analysis (which would be interesting, to get a statistical distribution of speeds), that would be extremely slow; I'd want a compressed data format like FIT (used by Garmin: a Perl library and a link to the Garmin SDK are here) for that purpose. But for a single ride the rich text format provided by JSON is fine.

As an aside, version 2 of the API claims to allow data to be filtered before download (similar to Garmin "smart sampling"). This should help a lot on the bandwidth hit. But using FIT or similar would probably be at least a 10× improvement.

There is another issue: user authentication. Hopefully this can be handled by the standing Strava app. But I may need to add a log-in page to mine. This isn't something with which I've dealt before.

On the number-crunching side, Java is a fairly simple, sequential language and the sequential array processing should be simple enough.

So things are looking okay. The goal on any programming job is to take complex tasks and break them down into simpler sub-tasks. Then take these sub-tasks, and if necessary, break them down into even simpler sub-tasks. You keep breaking tasks down until the remaining tasks are so simple, the actual coding of them is trivial. It's a lazy shortcut to try to take too much in one bite: to do a complex task directly, and attempting to do so invariably turns the project into a mess. The actual coding part for each sub-task should be simple. The challenge is in organizing the tree of tasks, in knowing exactly what you want to do before you try doing it.

15 comments:

Cosmo said...

Another huge advantage of JSON is that it's super compact, especially when compared to XML-based formats.

Taking the relevant example of GPX, you end up with things like "<temp>18</temp>" where the metadata is almost 7 times the size of the data it's describing. JSON lists the metadata once ('"temp":) and from then on, it's just delimiters.

If you thought pulling the full JSON array from your rides was slow, just imagine kicking around TCX files, which are even more bloated then their GPX counterparts.

Great find on the .FIT data, by the way. I'd been looking for some more info on it.

djconnel said...

Thanks for the comment! I love your export scripts. Trying myself to get up to speed on server-side scripting.

I agree JSON is way better than XML formats. OpenOffice uses gzipped XML. But I admire even more Garmin for taking an old-school optimized binary formatting approach to the problem. FIT files are truly impressive.

For FIT, I've been using the Garmin:FIT perl module, although looking back on code I wrote with it awhile ago I have almost no clue what I was doing... it's pretty obscure.

Ben Lowe said...

I've just been acquainted with Strava and it ticks all my excite buttons (cycling, data-visualization, mobile technologies etc.) and have begun to throw together a Windows Phone 7 app (someone's got to!) to show me my Strava data how I want to see it (e.g. all my segments, order by whatever, alerts when any of my placings change and when new segments are added) which I store in Isolated Storage (on the device) so I only need to request changes when I need them and have all my history (and all segment's efforts) available instantly.

You mention about authentication. I've had a little play with the authentication webservice but I'm just getting 404 errors at the moment. I haven't needed to be logged in as yet for what I've needed but I'd like to get at the mapping data.

Did you get your authentication working? Any pointers?

Once I'm happy with the WP7 app then I'll do the same for a Windows 8 Metro App as I should be able to use 95% of the same code :-)

djconnel said...

Thanks for the comment! Yes -- it works, using Javascript with jQuery's ajax method (it's the only thing I use from jQuery: prefer sticking to native Javascript until I'm fully fluent there). I post "email" and "password" and I get back the token, as advertised.

Gotta wonder, BTW, of the security in passing unencrypted passwords this way. I recommend not using the same password for Strava you use for your bank account :).

But maybe you're not using "post"... (not sure this matters).

Ben Lowe said...

That's weird, was using the URL from the documentation for the authentication call and was still getting the 404. A bit of head scratching later and tried it with app.strava.com rather than www.strava.com and hey-presto, working! All the documentation shows www.strava.com but it seems all the other services redirect you automatically to app.strava.com. The authentication one doesn't.
I'll update all my service calls and hopefully we'll all be good.

Ben Lowe said...

p.s. this is the link to the documentation I've been using: https://stravasite-main.pbworks.com/w/page/51754271/authentication-login
Is there a more up-to-date/correct version anywhere?

djconnel said...

No, that's the documentation I use.

But I see you're using v1 and I use v2. Try v2.

https://www.strava.com/api/v2/authentication/login

Note the "https".

James Mattis said...

Dan,

I'm also struggling with Strava authentication on the iPhone.

I keep on getting the following error returned:

{"error":"It looks like that ride is not your own?"}

The HTTP URL is:
https://www.strava.com/api/v2/authentication/login

The HTTP Body is:

{"email":"james.mattis@gmail.com","password":"xxxyyyzzz"}

HTTP Method is POST

Any ideas?

djconnel said...

This works to me, inspired by similar code sent to me by Paul Mach:


<?
/* StravaLogin.php
*
* log into Strava and return athlete data and token
* code based on code provided by Paul Mach ( http://paulmach.com/ )
*/

$email = $_REQUEST['email'];
$password = $_REQUEST['password'];

if ((! $email) || (! $password)) {
echo json_encode(array("error" => "missing email and/or password"));
exit;
}

/*********************************************************************
* access Strava API authentication
*/
$c = curl_init();
curl_setopt($c, CURLOPT_URL, 'https://www.strava.com/api/v2/authentication/login');
curl_setopt($c, CURLOPT_POST, true);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
curl_setopt($c, CURLOPT_POSTFIELDS, 'email=' . $email . '&password=' . $password);
$login = json_decode(curl_exec($c));
$token = $login->token;
$athlete = $login->athlete;

if ($token) {
echo json_encode(array("athlete" => $athlete, "token" => $token, "email" => $email));
} else {
echo json_encode(array("response" => "badpassword"));
}

curl_close($c);
exit;
?>


On the other hand, I'm unable to get the upload to work. Paul's approach was to generate a TCX file and upload that. I'm trying to do type=JSON but haven't been able to get Strava to bite. Initially I did jQuery/AJAX from a local file and it was fine, but then when uploading to DeltaWebHosting I got a cross-host error. So I instead push to a PHP file then from there, synchronously, to Strava. This isn't working for me. PHP seems way behind the times these days, all the cool guys are using Sinatra/Ruby. It's taking quite a lot of time given the documentation is on the sparse side.

James Mattis said...

Hmmm...

Looks like I was just reading the documentation wrong. Trying to pass the HTTP body as JSON. They just wanted parameters.

So having the body set to:

email=james.mattis@gmail.com&password=xxxyyyzzz

Worked fine.

I'm going to be working on getting upload to work today. Going to be trying to upload TCX and/or GPX. If I get it to work I'll let you know what I did.

djconnel said...

Thanks -- I was hoping to do type=json to avoid the intermediate step of generating XML-like TCX. It's weird they transpose the table relative to download: in download, outer structure is hash of parameter-list pairs, where each list is the full set of time points for that parameter. In upload, outer structure is an array of time points, each of which is a list of parameter values, with a separate list provided to map list index to parameter name. May be pragmatic to just go to TCX.

James Mattis said...

I just got a TCX file uploaded.

TCX does have a larger file size because of all the extraneous junk in the format compared to the JSON format, but...so many web sites use it now, its kind of become a semi-universal data format. Also, being human readable is kind of cool.

Didn't do anything special. Just POST the token returned in the authentication response data, POST the type 'tcx' and posted the TCX data in UTF-8 encoding. Not sure if that encoding is required, but it worked...

djconnel said...

Thanks -- it was recommended I post the entire set of parameters and values as a single JSON bundle, so I'll try that, and then go on to TCX. Paul Mach also used TCX.

Brian Lockhart said...

Thanks for sharing your efforts here! I'm interested in what the current state of things is, and if you'd be so kind as to share your .PHP code?

I'm trying to write a small server-side uploader script myself and would love to learn from those who have gone down this path before... Is .TCX the only viable option still? JSON working yet? Ideally I'd love to just upload a .FIT file...

Thanks in advance!
Brian

Unknown said...

Just a point: when using the login call, make sure to use a content type of x-www-form-urlencoded to prevent the values from being sent across a query string values. This way, the data will be encoded properly whereas having in the query string means the values are in plain text in a number of places.