Friday, January 7, 2011

a Perl Garmin-FIT to space-delimited table converter

I've been working on a code to filter motorized segments from FIT files, using Kiyokazu Suto's Garmin::FIT package for Perl. However, to help with that work, I needed a way to quickly plot the data in a FIT file. So I wrote a script, fit_to_csv, to create a readable table from FIT data.

The code produces a space-delimited file with each column describing a field either contained in the FIT file or derived from data in one or more fields of the FIT file. I have a series of scripts for handling such files and generating plots from them, or they can be trivially loaded into a spreadsheet like oocalc or even Excel, or almost any other plotting package, using the "CSV" format. When importing the CSV, make sure spaces and not commas are selected as the delimiter.

The code's available here.

On Linux, simply save the file, make sure it has executable permission, then run it in the standard Linux fashion. On Windows, well, you're on your own. I avoid Windows whenever possible.

An example, where "% " is the command prompt, and where it is assumed "fit_to_cols" is in the execution path:

% fit_to_cols ~/Library/GoldenCheetah/djconnel/2010_11_14_10_13_04.fit

This plots data from a FIT file stored in my GoldenCheetah data directory.

% fit_to_cols /media/GARMIN/Garmin/Activities/2011-01-01-16-20-25.fit

This plots data directly off my Edge 500, which is plugged into the USB port.

% fit_to_cols -fn /media/GARMIN/Garmin/Activities/2011-01-*.fit

This plots all of my files from January 2011, identifying each file with a column "fn". Had I not used the "-fn" option, files would have been identified with a column "file" which contained the file name.

% fit_to_cols -h

This prints help information.

The major weakness of the code is it assumed fixed data in the FIT file. FIT is a lot more sophisticated than this: the data can vary from device-to-device. I've hard-coded my script with the data I'm most interested in from my Edge 500. Honestly I'm still trying to figure it all out.

After I'd gotten a functional version of this code together I found someone else had done something similar: fit2csv.pl. I think my code is somewhat more sophisticated than that one, since the latter assumes a particular order for data fields to be stored (I use the hash tables to look up the order) and it doesn't handle undefined values well. My code also allows for multiple files to be listed at once and has several derived parameters like dx, dy, and r.

Thanks to Kiyokazu Suto for doing such nice work on Garmin::FIT.

4 comments:

Paul said...

On the ant+ website they have a "Fit SDK" that includes an java FIT to csv converter.

http://www.thisisant.com/products/data-sheets-brochures

I found it fairly straight forward to create a little .fit to .csv to .tcx converter.

djconnel said...

Thanks -- very interesting!

Java I don't know so well, and C++ I find very cumbersome to efficiently process data due to the need for allocation (in Perl allocation is implicit by reference). But working with the official SDK is obviously hugely attractive.

In any case, I'm almost done with my Perl project, so converting it over would be a lot of work. Maybe I'll try, however, as an exercise.

David Pronk said...

This morning I managed to extract the contents of a .fit file using your perl scripts. What I am actually looking for is a way to repair a .fit file which appears to be corrupt. I can read it's contents but I am unable to upload it to the Garmin Connect site. Do you have any experience on this?

djconnel said...

I've never been able to repair damaged FIT files, except to extract out portions prior to the flaw. You'd want a FIT to TCX converter, though, similar to the one which could be written with the SDK linked by Paul in the first comment. There's also one with the Garmin:FIT Perl module.