Sunday, February 26, 2012

Slow going on coding project

I've embraced Ruby for my coding project. Ruby has been great: I really like it's object-oriented nature: it's inherently object-oriented, with that built in to the core of the language, rather than some sort of kludge built onto what is essentially a sequential language. Actually, I'm not sure whether "object" versus "sequential" is correct, here, since Ruby is still highly sequential, it just heavily uses objects. You can still write Ruby code which looks much like Perl code, with notable exceptions.

But I don't want to write Ruby code which looks Perlish. I want it to look Rubyish. So therein is what makes for slow going: I write stuff, check it, re-read the on-line documentation to make sure I'm not reinventing any wheels (there are a lot of standard methods in Ruby), make adjustments, run, and track down the inevitable bugs. Slow going, but it's what it takes to build aptitude in a language.

So my end-of-Feb deadline for this is not looking good... A few quick examples of issues with Ruby which caused me non-trivial hiccups:

  1. $LOAD_PATH: the default load path, stored in a global array called $LOAD_PATH, or $: for short, changed from Ruby 1.8 to 1.9. Ruby 1.8 loads by default in Ubuntu 2011.10 (which I have), and also in the Mac OS/X in my new, regrettably neglected Mac Air, but 1.9 includes substantial new features. The one which comes first to mind is a Complex variable class. I don't plan to use complex variables in this project, but it's cool that it's there. I prefer to use the latest version. Anyway, I eventually figured out that the local directory (".") is missing from the load path in 1.9, while it is there in 1.8. However, relying on the local directory isn't a good idea, anyway, since you may execute a script from another directory, in which case you want the script's directory rather than the local directory to be in the load path. This is fixed with the following code at the top of the Ruby script: $:.unshift File.dirname(__FILE__). Okay, that's a bit obscure, but it's essentially idiomatic.
  2. type conversion: In Perl, strings are converted to floating point numbers as needed. And all math is done using floating-point (real) numbers, not integers. In Ruby, there is no implicit string-to-number conversion. For example, the "*" operator, which is a multiplier for numbers ("Numeric" class), generates repeated copies of strings. Additionally in Ruby floating point math is not assumed. So if you type puts 1/2 you get zero, because it does integer division which produces an integer result, while if you type puts 1.0/2 you get 0.5, since the "1.0" in the numerator is assumed to be floating point, and it "coerces" the denominator to be also floating point to produce a floating point result. This sort of issue can be particular insideous because you say tau = 0.5 and things work great, then you later change to tau = 1 (perhaps via a command line option), then somewhere in the code you divide an integer by tau and things go south. The key is to convert numbers of uncertain format (and therefore type) to floats with the to_f method, or if you anticipate using complex numbers, convert them to complex. So while Ruby lacks the strong typing of C or Java, it still has distinct number types, and you've got to watch for integer division if that's not what you want. By the way, I've had the same issue with Tcl at work, and it's even worse in Tcl because that language does do string-to-number conversions implicitly, and whether that number is integer or float is determined by the format, so you get a lot of [expr 0.0 + $var] stuff in Tcl to force floating point.
  3. variable names determine scope: In most languages you can name a variable pretty much anything you want (in FORTRAN, the first letter determines number type, but I've not done FORTRAN in many, many years). However, in Ruby you need to be careful. "$" as the first character means global, a capital as the first letter means a class, a lower case as the first letter means a variable, and a "@" as the first character means a class variable. This description may not be precise, but you get the idea... So for example, if I have a parameter describing the "CdA" product for modeling wind resistance, I can't call it CdA because that gets interpreted as a class name, not a variable name. So I am forced to use "cda" instead.

Okay, enough of that. So to mix things up, I then wanted to figure out how I was going to integrate this stuff with the Web. Two obvious options are server side or client side. I think server side is the way to go here because client side would basically mean embedded JavaScript and that wouldn't work so well on mobile platforms. For server side there's many options. DeltaWebHosting, on which I have two accounts, supports PHP and Perl CGI's. I'm trying to liberate myself from PERL, after all, and PHP is perhaps better for relatively simpler tasks than I had in mind. I want to use Ruby. Remarkably, DeltaWebHosting doesn't provide Ruby...

But I found that heliohost.org, a free hosting service, does support Ruby PHP scripts. So I set up an account there and will see if I can get stuff working there. If I successfully convince DeltaWebHosting to install Ruby than I can always move things to my domain there.

Anyway, a lot happening, but progress is slower than I'd like. This balancing the heavy day job + long commute + cycling + running + updating this blog + doing a moderately challenging web project thing isn't as easy as I'd like. Fortunately my cycling is going well this year, my running is hanging in there (once per week), and I am learning a lot in this project, so I can't complain.

Saturday, February 25, 2012

SB1464: revised 3-foot passing bill for California

Last year California's Governor Jerry Brown pocketed SB910, a bill which would have effectively banned drivers from passing cyclists with less than a 3-foot margin. Additionally it would have allowed drivers to cross the center-line when it was safe to do so in order to provide for that mandatory margin, which is a practice well over 99% of the drivers already exercise, anyway. His logic in pocketing the bill seemed inconsistent with the bill itself, as I described, bringing into question whether politics and influence was the actual motive.

So another year brings another legislative session, and the California Bike Coalition continues, to its substantial credit, to push the matter. The Give Me Three campaign is not dead, not at all. Instead, SB910 has been replaced with SB1464. This bill is a direct response to Brown's concern, in his own words:

On streets with a speed limit of 35 or 45 mph, slowing to 15 mph to pass a cyclist could cause rear-end collisions. On the other hand, a cyclist riding near 15 mph could cause a long line of vehicles behind the cyclist.
Brown was worried about the provision which allowed for a car to pass with less than 3 foot margin if it was going less than 15 mph.

I will resist the temptation to rant once again against the twisted, convoluted, inverted logic used here. But what is done is done, so CalBike said okay, if you don't like the 15 mph exemption, fine. It's gone. So now we have SB1464. Brown has left himself no opening to oppose it.

The problem is the 15 mph limit was there for a very good reason: so a cyclist couldn't block car traffic at intersections. Intersections tend to be fairly tight and crowded, and three foot margins are simply not available in many cases, especially when the cyclist pulls up next to a car whose driver has no option to move sideways:

SFGate

With SB910, no problem: the drivers and cyclists could move ahead clear of the intersection and then, once they were up to speed, the 3-foot passing margin requirement would kick in. SB910 never said it was safe to pass with less than three feet at less than 15 mph, simply that it may be safe in a specific situation (slow traffic), and therefore such a pass was not going to be generally forbidden.

Well, with SB1464, there's no exception. At an intersection, drivers must wait for an opportunity to move left, period, no matter how slowly the rider accelerates. This would obviously create needless congestion.

The reality, however, is that when rules overstep reasonable bounds, they are applied with discretion. So very few tickets will be issued in these cases. It will become yet another example of a weakness in the law being left to officer discretion.

The problem is with regard to bike-car issues, officers have a terrible record of discretion. The police simply cannot be trusted, should not be trusted, to define the scope of the law. Sure, some discretion is always needed, but when possible the law should speak for itself. Otherwise we could vastly simplify the Code with a tidy "all drivers of motor vehicles shall behave in a safe, predictable, and courteous manner". This basically covers everything.

So once you open the obvious discretionary door for intersections, where else is discretion applied? That's the issue. You're tacitly approving of discretionary enforcement by requiring discretionary enforcement in an obvious, common situation where rigorous enforcement would be unpopular.

Still, this bill would be a huge improvement. If nothing else, it allows drivers to cross the center line when passing. Just this past Thursday I was on the Palo Alto Noon Ride on Arastradero Road (StreetView) when a driver came by the large pack of riders without touching the center line, forcing riders to squeeze to the right to avoid being hit. This was illegal on many levels, and on such a narrow road, it can be legitimately argued the driver should have waited to pass. But at least with the center line rule, the driver had absolutely zero justification in playing the "no choice but to stay in the lane" intimidation game, one I've seen more often played, ironically, by police: the same guys we're trusting with discretionary enforcement.

So everyone, please support SB1464, and if you haven't, fire a few bucks (they ask for three) to the "Give me Three" campaign to help CalBike fight the good fight. This is a fundamentally important matter for cyclists right to use the roads to which legally they have always had equal rights as drivers, since before there were even cars to drive.

Thursday, February 23, 2012

Strava lament

My KOM up Dyer's Pass on the South Island of New Zealand was crushed "yesterday"... I put "yesterday" in quotes because it's hard to figure out precisely what it means that many time zones away, especially with the date line:

The horror... the horror...

With my light bike, with fresher legs, with a better warm-up, I know I could have gone faster. But a full minute faster? Or rather -- 61 seconds faster?

That's the way Strava is. With exponentially increasing user base, records don't usually last long. In the San Francisco Bay area, things are hitting the linear growth regime, but elsewhere, the market penetration is still much smaller. Every week, more people are going to be posting rides, more climbs are going to be ridden hard, and KOMs come and all-to-quickly go.

I'd love to go back -- it was a great trip, and I miss the great riding there, even if I am appreciating with renewed freshness the riding here in the San Francisco area where I live and ride.

Tuesday, February 21, 2012

Ruby SmoothData class: now static-free

Last time I posted what I called a "SmoothData" class for Ruby. But even as I was hitting "publish" on Blogger I realized I wasn't being honest. There wasn't much "class" about it: it was a single static method which was similar to what one would write for any sequential procedural language. Indeed, since it was essentially ported from Perl, this shouldn't be a surprise.

Initially my attitude was "this function doesn't need to preserve any local state. There doesn't need to be multiple instances of it. It's just an algorithm an nothing more. Why make life too complicated just for a dogmatic adherence to the object oriented cult?"

But then I tried to use it and I realized... hmm... maybe I should reconsider.

The plan was to, for various measured Strava parameters (altitude, speed, power) to have both measured values and smoothed values. So, for example, for altitude, I'd have an altitude array and a smoothed altitude array, each a function of time, another array. I'm "remember" that altitude was smoothed with respect to time and not, for example, with respect to distance. Furthermore, I'd "remember" to calculate the smoothed values the first time I needed them, then after I could refer to the calculated values.

But then I realized this is exactly the sort of algorithm which could be improved with an object. What I want is something which I can initialize with arrays of raw x and y values, give it a smoothing time constant ("tau"), then if I ask for the smoothed value of y it can do the dirty work of figuring out if it needs to calculate that or refer to the saved value.

There's still that method though, the one which does the difficult work of calculating the array of smoothed y values. It could be done either as a static method or as an object method.

My first thought is it should be static: then I could call it from outside the scope of the class with arbitrary arrays, and it would return the smoothed array, without perturbing instance variables. But this was slightly cumbersome: I had to pass arguments to the smoothing function, and calling the function required prepending the class name to the function name.

So another reevaluation... did I really need to make the method static?

The bias in object-oriented programming is against static methods. They're like "gotos" and global variables in procedural languages. Sure, they have a place, but should be used only when really needed, because they are more often than not a good sign you're not doing things in the way for which the language is designed. This is a controversial topic: some people espouse the "get it done however you can get it done as quickly as possible and don't sweat the details" approach, but I prefer doing things after a certain model, because I believe it makes code easier to manage, expand, and reuse. In the end, investing more time up front saves time.

So I de-staticked the smoothing procedure, and things look much better now.

I stuck the code in this Google Docs folder. Here's how it looks:


class SmoothData
  def initialize(x=nil, y=nil, tau=0)
    @x = x
    @y = y

    # make sure tau is set to a floating point number to avoid integer division
    @tau = Float(tau)

    @ys = nil
  end

  def smoothData
    # if no x values, nothing to do
    return if @x.nil? || (@x.length == 0);

    # make sure x and y have the same length
    if @x.length != @y.length
      warn "smoothing attempted with unequal length of x and y arrays: adjusting y"
      # if x is shorter then y, then crop y
      if (@x.length < @y.length)
        @y = @y[0 ... @x.length]
      # if x is longer than y, then pad y with zeros
      else
        @y << Array.new(@x.length - @y.length, 0)
      end
    end

    @ys = Array.new(@x.length)

    # if no smoothing is requested, done
    if (@tau == 0)
      @ys = @y
      return
    end

    # otherwise run filter in both positive or negative directions
    [-1, 1].each do |d|
      xold  = nil
      yold  = nil
      ysold = nil

      # along each direction, we run an exponential convolution
      (0 ... @x.length).each do |i|

        # if this is the negative direction, use points starting
        # from the last, otherwise go in forward order.
        # So i is an initial counter but n is the point we're going
        # to process
        n = (d == 1) ? i : @x.length - 1 - i


        # for either the first point in the sequence or if there is
        # a gap much larger than the smoothing constant, use the unsmoothed
        # point
        if ((i == 0) || (@x[n] - @x[n - d]).abs > 100 * @tau)
          @ys[n] = @ys[n].nil? ? @y[n] : (@ys[n] + @y[n]) / 2
          xold  = @x[n]
          yold  = @y[n]
          ysold = @y[n]

        else
          # calculate the proper contribution of the new point with a running
          # average of old points
          # u is a normalized difference between this point and the preceding one
          u = (@x[n] - xold).abs / @tau

          # apply the exponential decay to z
          z = Math.exp(-u)

          # dy is the difference between the present point and the previous point
          dy = @y[n] - yold

          # the following works with non-uniform point spacing
          ysdir = ysold.nil? ? @y[n] : ysold * z + @y[n] * (1 - z) + (dy / u) * ((u + 1) * z - 1)
          
         # update ys
          @ys[n] = ys[n].nil? ? ysdir : (@ys[n] + ysdir) / 2

          # update "previous" points before going to next point
          xold = @x[n]
          yold = @y[n]
          ysold = ysdir
        end
      end
    end
  end

  def x=(xnew = nil)
    @x = xnew
    @ys = nil
  end

  def y=(yew = nil)
    @y = ynew
    @ys = nil
  end

  def tau=(tau = 0)
    @ys = nil
    # make sure tau is set to a floating point number to avoid integer division
    @tau = Float(tau)
  end

  def resetYS
    @ys = nil
  end

  def x
    @x
  end

  def y
    @y
  end

  def tau
    @tau
  end

  def ys
    if @ys.nil?
      smoothData
    end
    @ys
  end
end
Here's an example, using the interactive ruby shell (irb):
% irb
irb(main):001:0> require 'SmoothData'
=> true
irb(main):002:0> sb = SmoothData.new
=> #
irb(main):003:0> sb.x= [1, 2, 3, 4, 5]
=> [1, 2, 3, 4, 5]
irb(main):004:0> sb.y= [0, 0, 1, 0, 0]
=> [0, 0, 1, 0, 0]
irb(main):005:0> sb.x
=> [1, 2, 3, 4, 5]
irb(main):006:0> sb.y
=> [0, 0, 1, 0, 0]
irb(main):007:0> sb.ys
=> [0, 0, 1, 0, 0]
irb(main):008:0> sb.tau= 1
=> 1
irb(main):009:0> puts sb.ys
0.183939720585721
0.183939720585721
0.367879441171442
0.183939720585721
0.183939720585721
=> nil

In this case, I created a new SmoothData object with no parameters. It thus had no initial data. Then I sent it arrays for x and y. I still hadn't initialized the smoothing constant, so no smoothing was applied. When I asked for "ys", I just got y back. Then I set the smoothing constant to 1. Then when I asked for "ys", I got a smoothed representation of the data. The initial function was symmetric, and the smoothed function was also symmetric, as I wanted.

Monday, February 20, 2012

SmoothData : a simple Ruby data smoothing class

Ruby self-teach continues, so I put together a data smoothing class for Ruby. Nothing new here: another in a series of codes I've used for comparing Java to Perl to Ruby. Data smoothing is an important part of anything I do with cycling data. So this is a critical component for me on any sort of project I might want to accomplish. For example, suppose I wanted to write a code to identify and rate climbs in an activity. I know -- Strava already does this, but I've got my own ideas about algorithms. Well, if I were to take altitude data raw, a small glitch yielding a 50% grade between two closely-spaced points might generate a huge spike in the climb rating for just those two points. Instead it would be important to smooth the altitude out, for example with a 50 meter characteristic smoothing distance. 50 meters is a fairly good number on climbs this length and shorter you can generally use momentum to blunt the blow. It also covers up small errors in position and altitude. So here it is.... I also added the file to a Google Docs folder to avoid formatting issues with this Blogger page:
#! /usr/bin/ruby

class SmoothData
  def self.smoothData(x, y, tau = 0)
    ys = Array.new(y.length)

    return ys if (ys.length == 0)

    # if no smoothing is requested, simply copy over numbers
    if (tau == 0)
      ys y.clone
      return ys
    end

    # otherwise run filter in both positive or negative directions
    [-1, 1].each do |d|
      warn "direction = #{d}"

      xold = 0
      yold = 0
      ysold = 0

      # along each direction, we run an exponential convolution
      (0 ... x.length).each do |i|

        # if this is the negative direction, use points starting
        # from the last, otherwise go in forward order.
        # So i is an initial counter but n is the point we're going
        # to process
        n = (d == 1) ? i : x.length - 1 - i


        # for either the first point in the sequence or if there is
        # a gap much larger than the smoothing constant, use the unsmoothed
        # point
        if ((i == 0) || (x[n] - x[n - d]).abs > 100 * tau)
          ys[n] = ys[n].nil? ? y[n] : (ys[n] + y[n]) / 2
          xold  = x[n]
          yold  = y[n]
          ysold = y[n]

        else
          # calculate the proper contribution of the new point with a running
          # average of old points
          # u is a normalized difference between this point and the preceding one
          u = (x[n] - xold).abs / tau

          # apply the exponential decay to z
          z = Math.exp(-u)

          # dy is the difference between the present point and the previous point
          dy = y[n] - yold

          # the following works with non-uniform point spacing
          ysdir = y[n].zero? ? ysold : ysold * z + y[n] * (1 - z) + (dy / u) * ((u + 1) * z - 1)
          
          # update ys
          ys[n] = (d == -1) ? ysdir : (ys[n] + ysdir) / 2
          

          # update "previous" points before going to next point
          xold = x[n]
          yold = y[n]
          ysold = ysdir
        end
      end
    end
    
    return ys
  end
end

Sunday, February 19, 2012

StravaToCSV : It's Ruby's turn

StravaToCSV has become my "test app" for various programming languages (Perl, Java, and now Ruby). And for that it works fairly well: I need to process command line arguments, open an HTTP connection to Strava, download JSON data, convert it, then then output it as CSV. So there's a decent amount there.

This project went much smoother than my Java implementation. It was fairly quick, taking a bit longer when I wanted to avoid the program imploding when it was fed a bad activity specification.

This version takes as its only command line arguments activity numbers (for full activities) or activity-segment pairs, where the activity number is separated from the matched segment by a "#". It will sequentially load each of these, outputting a CSV stream with the header determined by what fields it finds in the first non-empty activity. It adds the activity number as the first column of the CSV stream.

At first I expected the JSON library to have some sort of validity checking method for the stream: does a stream represent valid data? But I didn't find any. Instead when attempting to convert data it will throw a "JSON::JSONError" exception if it can't figure things out. So I use the Ruby "begin... rescue... end" construct to check for this. This is about as simple as exception handling gets, in my experience.

Anyway, it seems to work. It's my first experience with Ruby, so I'm probably not doing things in the best fashion, but it's nice to see it do what I wanted, anyway.

I'm sure the Ruby coders are hurling at the mere sight of this... Feedback welcome!

#! /usr/bin/ruby
require 'net/http'
require 'json'

# strip a path from the program name: for error messages
progname = __FILE__.gsub(/.*\//,"")

# loop through command line options
# each should be a valid strava activity
activities = []
ARGV.each do |arg|
  # check that arg is a valid format
  if arg.match(/^\d+#?\d*$/).nil?
    warn "ERROR: poorly formatted arg #{arg}"
    exit 1
  end

  activities << arg
end

headers = []

activities.each do |activity|
  warn "#{progname}: processing activity #{activity}"

  url = "http://www.strava.com/api/v1/streams/#{activity}"

  # get the data structure from the file
  # get does not raise exceptions in Ruby 1.8, according to class documentation
  http = Net::HTTP.get(URI.parse(url))

  # convert to data
  begin
    data = JSON.parse(http)
  rescue JSON::JSONError
    warn "#{progname}: Error parsing Strava activity #{activity}!"
    next
  end

  # if we don't have data, then there was an issue: go on to next activity
  if data.length.zero?
    warn "#{progname}: No data found for activity #{activity}!"
    next
  end

  # list the returned fields
  if headers.length.zero?
    headers = data.keys

    # print the headers, but special case for latlng, which must be split
    puts "activity," + headers.join(",").sub("latlng","lat,lng")
  end

  # iterate over the indices of the first field
  (0 ... data.values[0].length).each do |i|
    output = [activity]
    headers.each do |k|
      z = data[k][i]
      output << (z.nil? ? "" : z)
    end
    puts output.join(",")
  end
end

Anyway, I'm getting a really positive feeling about Ruby. I don't feel like I'm fighting it: it's a fairly coherent design. With other languages I often feel they've been pushed beyond their original scope, that they've become a house of cards of layer upon layer forced in place to accommodate unforseen paradigms or needs. Java, with it's HTTP module, feels very much like that, as does all of C++. And Perl with it's ad hoc object handling and endless selection of CPAN libraries is just a big bucket of chaos. Maybe Ruby is moving in that direction now that the number of cooks has grown. But at least for this example I feel as if it all works together.

I ran some quick benchmarks, converting a recent long ride. Here's the execution times for the three versions, where I avoided internet activity while the code was running:

Perl: 32 seconds, then 42 seconds (two iterations)
Ruby: 38 seconds, then 44 seconds (two iterations)
Java: 30 seconds, then GSON threw an exception

Java barfed on me, I'm not sure why. I'd need to debug that... maybe the activity was too large, because it runs on small activities. But it doesn't surprise me: the syntax is all fairly opaque and that makes it prone to coding errors. Perl might have been faster than Ruby, maybe not... but in the Perl I cheated in downloading the URL with wget, which is optimized compiled code, while in the Ruby I'm using Ruby to download. I guess the main thing this proves is my AT&T/Yahoo DSL is slow, slow, slow.

Saturday, February 18, 2012

Perl, Java, Ruby

Well, I realized the Pine Flat Road Race was tomorrow (Sunday) not today (Saturday) and so, had I rested today, I probably could have gone. Ah, well. I have a rule that I need to do a hard group ride before I even think about racing, and I haven't yet done so this year, or even towards the end of last year. No, the Friday Noon Ride is not a "hard group ride".... nor is Wednesday. Only Tue and Thu come close to qualifying. So it probably wouldn't have been prudent trying to mix it up at Pine Flat, anyway. But on the positive side, that's 6-7 hours I will not be spending in a car, and more time I won't spend in a smelly motel room, more time to devote to my coding project.

On Facebook, it was recommended I look at Ruby. This was an obvious thing to do, since it is what Strava uses for its API example. However, I had gravitated towards Java because with Java I knew I could generate an Android app. All it takes is a bit of Googling to note that Android apps can be written in Ruby as well, but the most popular language for Android apps is clearly Java.

Ruby, or more particularly Ruby On Rails, is gaining popularity for serving web pages. I knew virtually nothing about Ruby until today, but even with a brain severely impared from the accumulated fatigue of four consecutive days totalling over 200 miles of hard bike riding (including today's semi-epic), I've found it surprisingly quick to pick up. The main web site has good documentation, including some nice tutorials. What I like about what I see there is it combines the ease of data handling (implicit typing, automatic type conversion, automatic array allocation, easy-to-use hash tables, etc) of Perl with a strong object orientation which I've read is similar to SmallTalk, an ancient rarely used language.

A simple example: converting a data structure to JSON format for communication with the Strava API.

With Perl, I whipped together a "Strava_to_CSV" script using the JSON library. Here was how I decoded the JSON stream:

my $json = JSON::PP->new;
my $data = $json->decode($s);
This gives me a reference to a hash table with the keys the field names, the values the field values. This is great, minimal fuss.

With Java, I've been using the Google-GSON library, which has been claimed to be relatively easy to use. However, it's much more work than the Perl library. With GSON, I need to set up a class with variables with the same names as the fields in the JSON stream, with types correctly set up for each field. So with easy API command returning its own unique JSON structure, that means setting up a class specific to each API command (or class of API commands: some may share the same class). It's not a huge roadblock, but is a hassle and adds lines of code, and with that, opportunities for errors. And with Strava something of a moving target, it increases the maintenance costs of any code I put out there.
So onto Ruby. There's a nice JSON library for Ruby which seems to work much like the Perl one. Here's a super-trivial example, where I generate an array in Ruby, encode it as a JSON string, then I both encode and decode it:

#! /usr/bin/ruby
require 'json'
a = [1, 2, 3, 4, 5, 6]
puts a
puts JSON.generate a
puts JSON.parse JSON.generate a

The output is:

1
2
3
4
5
6
[1,2,3,4,5,6]
1
2
3
4
5
6

The biggest hassle with this was getting the JSON library installed. I tried using "gem", but that didn't seem to work, so I used "apt-get ruby-json", and that did the trick. If I can get this sort of thing running in just a few minutes, that's a good sign.

Interacting with web pages is another issue. With Java, the following line is virtually straight from the Oracle tutorials

BufferedReader in = new BufferedReader(
  new InputStreamReader(
  new URL("http://www.oracle.com/").openConnection().getInputStream()));
If that code doesn't sent you screaming and running from the building I don't know what will!

For Perl, I typically cheat and open a pipe from the system "wget" command, but using Perl libraries isn't so hard. There's many library options with Perl. The HTTP::Tiny library is particulary simple:

use HTTP::Tiny;
my $http = HTTP::Tiny->new;
 
# get a single page
my $response = $http->get('http://example.com/');
die "Failed!\n" unless $response->{success};
print $response->{content};

But with Ruby, things are even simpler, especially since a standard library provides the simplicity. Here's a line from that Strava API example, in Ruby:

Net::HTTP.get(URI.parse(url))
Now that's a degree of simplicity I can handle! Imagine being asked, without reference materials, to have to write either code, for example for an exam. The Ruby code you should pretty much nail with maybe 30 seconds of study. For the Java, well, good luck.

So I'm not dismissing Java right yet. But it seems if I want to get something working, server-side web via Ruby and Rails is the quickest approach to providing some useful Strava functionality, assuming DeltaWebHosting will allow me access the JSON library, something I need to check.

Friday, February 17, 2012

the importance of a good Tuesday

The key for me in having a good week, fitness-wise, is Tuesday.

The ideal week begins with a tired Monday. If I feel like riding on Monday, I didn't ride/run/whatever hard enough over the weekend.

Tuesday I typically either ride to work, go out to do some intervals, or get a good run in. Actually, it's been awhile since I've actually done any intervals. But I've been fairly reliable on doing one of the other two options.

Wednesday can be ride to work, do the Noon Ride up Old La Honda, or go off and do a lunch ride on my own. Some weeks I've run on Wednesday instead.

Thursday is then an easy day. Just ride to work and back. But some weeks I've also ridden to work on Thursday.

Friday I typically recover if I rode in on Thursday, or else try to get is little ride in. The Apple lunch ride is a good option, or I just ride on my own around Mountain View.

Then the weekend: During Low-Key Hillclimb season, I was doing the climb on Saturday, then some sort of Sunday ride or run. Now I try to get in something decent each day. It could be a run, a long ride, or a mountain bike ride perhaps. The key is to push myself fairly hard, to be tired on Sunday, ready to start over the next week.

But rewind to the beginning of this week. Saturday was the MegaMonster Enduro, a 100+ mile ride from Paicines, south, and back again. With the climbing, wind, rain, and 200 miles spent in a car, it was a long and tiring day. I was exhausted.


partial data from MegaMonster, until my Edge 500 battery expired.

Bill Bushnell photo
Bill Bushnell photo
of me suffering at the MegaMonster

Sunday I worked on the results, which with all of the split times (3 for each rider, plus start time + finish time) always takes a lot more work than I think it will. Eventually I got that ironed out, but only after taking a break to do a ride around the city. My initial plans were truncated when I realized I was still exhausted. After getting home then working on results a bit longer, I collapsed on the futon for a long nap.

Okay, that's not a problem. Rebound...

Monday was a recovery day as usual. Nothing wrong with two recovery days in a row, and to try to ride hard on Monday would have just disrupted the plans for the rest of the week. Plus I was still tired.

Then came Tuesday...

I was at work, and all of a sudden it was 11:20.... time to change if I wanted to make the Noon Ride, for which I'd need to leave @ 11:30. No, in the middle of something, I'll go out on my own at 12:00...

Then it was 12:00. Still stuff to do. Plus I was hungry. I went to the cafeteria to get lunch instead...

Lunch over, eaten at my desk. I couldn't do anything right after lunch.... I'd go for a run later.

But later never came. It came time to rush off, ride the 2.5 miles to the train station to catch a northbound baby bullet. I contemplated doing a run when I got home, but I was too tired and hungry at that point. I should have stopped at the gym, perhaps...

So nothing on Tuesday. I knew then I wasn't going to race on Saturday, an option I'd been contemplating. If I was to race, anything hard on Thursday would have not been an option. But I badly wanted to get in a few good efforts during the week.

Wednesday: I rode to work ("Skyline Way"). Thursday, I did it again... Riding to work always makes for a good day. I feel energized and alive, productive straight through lunch and until I need to catch the train. Okay, so maybe I didn't drink enough on the rides and was feeling a bit light-headed for awhile... but mostly good days.

Friday: even though I was quite tired Thursday night, and still tired Friday morning, I went to catch the noon ride, wanting to work off some frustration. The Noon Ride has virtually a 100% success rate of this.... and it was a good ride, even if I did get popped from the lead group on the final climb, up Arastradero. I just lacked the extra 5% I needed there.

So I feel pretty good now, Friday night. But we're entering a holiday weekend, and I know if I got hard on Saturday, while I may be able to drag myself out on Sunday, that will have been five hard days in a row and there's no way Monday will make that six. So ideally I do something easy tomorrow, then get in two solid days Sunday-Monday, before recovering on Tuesday then doing a hard ride on Wednesday.

But that would be too intelligent. I'll probably go bang my head against some wall tomorrow and spoil it all.

So missing a Tuesday workout is always the same thing: the initiation of a cascade of destruction ending in tragedy. There's always a good reason to not go out, to take the train in instead of ride, to sit at my desk instead of go for a run or ride at lunch. Once I start accepting these excuses, it's over.

Wednesday, February 15, 2012

Setting Emacs tab stops for Java

I have a problem with Java code: with classes, methods, exception blocks, loops, etc, there's a lot of indentation. Especially when I'm posting code to this blog, with its narrow text column, things quickly flow off the right side of the page. And it's so unnecessary. I don't think 4-column tab stops, which I was using, is needed to produce readable code. The key to solve this is in the following file on my Linux system:
~/.emacs.d/init.el
This contains "elisp" commands which are executed upon emacs start-up. Elisp is the Emacs variant of LISP. Honestly, even though I am good with basic MIT Scheme, and also the Elk Scheme I use at work, I find Elisp fairly challenging. But Google is my friend, so I inserted the following:
;Basic unit of spaces for each indentation level.
; c-basic-offset: sets offset for "C-class" languages
; I specifically set it smaller in Java, which tends to have extra indents
; with its class structure.

(setq c-basic-offset 4)
(add-hook 'java-mode-hook
  '(lambda ()
    (setq c-basic-offset 2)
  )
)

; force emacs to use spaces
(setq-default indent-tabs-mode nil)
Now "C-class" languages default to an offset of columns, but because Java tends to have so many tab stops, I set the Java indent to 2 columns. Two works pretty well for me: enough indent for structure, but obviously not so much that things get pushed off to the right. The last two lines of the above suppress use of the tab character in files. Tab characters are a general disaster, as different environments assume different tab spacing. A few extra bytes to fill these in with spaces is well invested in any case where anything but an arbitrary value for tab spacing is unacceptable.

Tuesday, February 14, 2012

mired in Java

So, while this weekend was largely consumed with The MegaMonster Enduro, my quest to become a Java master continues.

The Megamonster went okay. It's a long trip down to Paicines: I am in Howard's debt for the ride. And since I have the Low-Key "stuff", it made for a full car. I didn't really help much while there: this year I rode. I did the 100-miler for the first time in 3 years, the preceding two not having the fitness for it. I felt fairly good, but I was slow, taking 5:35 for the course. It's not easy: wind, hills, and this year light rain. I don't think the rain was much of a factor but the wind always is.

Anyway, the big deal is doing the results after. I always have a hard time with Megamonster results: for each rider we have three checkpoint times, a start time, and a finish time. I'm prone to making typos, and my scoring scripts only get used once per year, so it's never as smooth as I want. I think things are in fairly good shape, however.

Anyway, that took a big chunk out of my goal for getting this Android project done by end of Feb. It's not looking good at this point, yet I forge onward.

Basically tasks which which I'm fluent in Perl I have to struggle with in Java. Sure, you can use only static methods and write a Java which is a fairly clean map of Perl code, but that's not the point. I want to embrace the Java model.

But even for simple tasks, sometimes it takes a bit of digging. For example, the simple matter of generating a key-value list and printing the result.

Here's Perl... I can write something like this almost in my sleep... I'm writing the following without even checking if it works:

#! /usr/bin/perl
use strict;
my %h =
  (
    a => 1,
    b => 2,
    c => 3,
    d => 4,
    e => 5,
  );
for my $k ( sort { $a cmp $b } ( keys %h ) ) {
  print "key: $k = $h{$k}\n";
}

How hard was that? (Okay: I cheated.. I missed a ")" first time I wrote it).

So onto Java...

As an aside, the default Java mode for emacs uses rather generous 4-column tab stops, which isn't at all good for this Blogger 400 pixel wide format. Ah, well. Even with a small font things are going to wrap.

import java.util.*;

class HashMapTest {
    public static void main(String[] args) {
        HashMap h = new HashMap(5);

        h.put("e", "5");
        h.put("a", "1");
        h.put("d", "4");
        h.put("c", "3");
        h.put("b", "2");

        System.out.println("using iterator:");
        Iterator i = h.entrySet().iterator();
        while (i.hasNext()) {
            Map.Entry m = (Map.Entry)i.next();
            System.out.println(m.getKey() + ": " + m.getValue());
        }

        // produce a sorted list of results
        System.out.println("sorted keys:");
        Object[] keys = h.keySet().toArray();
        Arrays.sort(keys);
        for (Object k: keys) {
            System.out.println("key: " + k.toString() + " = " + h.get(k));
        }
    }
}

Here two methods are used to step through the keys. One is with an Iterator, part of the Set class, which is returned by the entrySet() method of the HashMap class, which is a subclass of the Map class. At least I think I got that correct...

I wasn't sure how to work with the iterator to produce a result sorted by key, so I bailed out and used the toArray() method to give me something I knew how to sort. Iterators are still a bit hazy to me. I've seen them but I've never used them on my own.

So anyway, I forge onward. Hopefully I don't make a total mess of things.

I have managed to write code to smooth time-series data. That's a key component of doing anything analytic with ride data. I've done several blog posts on the topic: I really like the "biexponential" smoothing algorithm in which data are first convolved with a forward exponential smoothing function, then with a reverse one, then the two results averaged yielding a symmetric smoothing (no lag or lead). Since the convolution function is continuous, the Fourier transform is fairly clean (unlike a square smoothing filter). There's a bit of care taken here to deal with nonuniformly spaced data, for example as one might get from Garmin Edge "smart sampling".

private static double[] smoothArray(double[] x, double[] y, double tau ) {
  double[] ys = new double[y.length];

  if (ys.length == 0)
    return ys;

  // if no smoothing is requested, simply copy over numbers
  if (tau == 0)
    for ( int i = 0;  i < x.length; i++ ) {
      ys[i] = y[i];
    }

  // otherwise run filter in both positive or negative directions
  else  {
    int[] directions = {-1, 1};
    for (int d: directions) {
      System.err.println("direction = " + d);

      double xold = 0;
      double yold = 0;
      double ysold = 0;

      // along each direction, we run an exponential convolution
      for ( int i = 0;  i < x.length; i++ ) {

        /* if this is the negative direction, use points starting
           from the last, otherwise go in forward order.
           So i is an initial counter but n is the point we're going
           to process
        */
        int n = (d == 1) ? i : x.length - 1 - i;


        /* for either the first point in the sequence or if there is
           a gap much larger than the smoothing constant, use the unsmoothed
           point
        */
        if ((i == 0) || (Math.abs(x[n] - x[n - d]) > 100 * tau)) {
          ys[n] = y[n];
          xold  = x[n];
          yold  = y[n];
          ysold = y[n];
        }
        /* calculate the proper contribution of the new point with a running
           average of old points
        */
        else {

          /* u is a normalized difference between this point and the preceding one */
          double u = Math.abs(x[n] - xold) / tau;

          /* apply the exponential decay to z */
          double z  = Math.exp(-u);

          /* dy is the difference between the present point and the previous point */
          double dy = y[n] - yold;

          /* naive approach: doesn't work for non-uniform point spacing */
          // ys[n] = ysold * z + (1 - z) * y;

          /* proper approach: works with non-uniform point spacing */
          double ysdir =
            (u == 0) ?
            ysold :
            ysold * z + y[n] * (1 - z) + (dy / u) * ((u + 1) * z - 1);

          /* update ys */
          ys[n] =
            (d == -1) ?
            ysdir :
            (ys[n] + ysdir) / 2;
               

          /* update "previous" points before going to next point */
          xold = x[n];
          yold = y[n];
          ysold = ysdir;
        }
      }
    }
  }

  return ys;
}
Nothing objecty about that... I realized I'm not going to really get this stuff using only the Oracle tutorials, so I just ordered the O'Rielly book, Learning Java. It's huge. Ordering books is easy: reading them is the hard part. But I feel if I want to really learn this stuff I've got to stick with it.

Monday, February 13, 2012

Republican party and public health care

I'm going to take a small break from cycling to vent a bit...

The Republican campaign for president continues... but I have no interest. I gave up on the Republican party long ago.

Actually, I'm a Republican wanna-be. I want to vote for a party which is fiscally responsible, which balances its budget, which limits its federal involvement to things which the federal government is uniquely qualified to do like interstate commerce and international relations. I agree states should be granted broad freedom in their ability to decide on local matters, that competition between states is good, and that free market principles should be harnassed whenever externalities don't dominate. I have a strong appreciation of the tragedy of the commons. Further, I am a champion of individual freedom: I agree we should be able to express ourselves freely, travel freely, choose if and how we practice religion freely, and have input over the policies and actions of our government. These should be positions embraced by the Republican party.

But the Republican party really has little to do with these matters. Instead, it's just another face of the same old machine where money buys influence, influence used to drive the short-term profits of the few, virtually ignoring the long-term health of the nation. The fact the Republican Party wraps itself in the cloak of Christianity is the ultimate in hypocracy. This alone is sufficient reason to reject it.

This is nowhere more obvious than it is in the rhetoric on the public health debate. "Nobody should be required to pay for someone else's health care," I read over and over again. These people seem to have no issue with "public fire protection", or "public police protection", or "public military protection", or a "public road system", or in most cases even "public education system". But for some reason if I'm hit by a car and am bleeding to death on the street, my ability to pay is supposed to be assessed before I can be administered aid. Or if I'm stricken with a heart attack I'm supposed to be provide proof of insurance before my heartbeat is restored. It's all part of the inherent efficiencies of the free market.

Or perhaps this isn't their position. After all, it was Ronald Reagan who signed into law the Emergency Medical Treatment and Active Labor Act which requires emergency rooms on private and public hospitals to provide emergency care to people without regard for ability to pay. That care, of course, is passed on to those who are able to pay. This is socialized medicine in its purest form. In comparison, "ObamaCare" is it a laughable imitation.

So given this, that hospitals must care for those who can't pay for the service, there is immediately a direct public financial interest in helping people stay healthy, since emergency care is outrageously expensive (this is another matter, another problem; it's not obvious this needs be the case). This is in addition to the "well functioning society requires a healthy population" argument... an unhealthy population, like an uneducated one, is simply unable to provide for a competitive nation, any more than an unhealthy sports team is going to compete.

So what's the Bible say on the matter? The bible, many Republican supporters argue, is an important document for national policy. Well, it takes only a brief Google search to turn up the following verses from Luke 16:

19 “There was a rich man who was dressed in purple and fine linen and lived in luxury every day. 20 At his gate was laid a beggar named Lazarus, covered with sores 21 and longing to eat what fell from the rich man’s table. Even the dogs came and licked his sores.

22 “The time came when the beggar died and the angels carried him to Abraham’s side. The rich man also died and was buried. 23 In Hades, where he was in torment, he looked up and saw Abraham far away, with Lazarus by his side. 24 So he called to him, ‘Father Abraham, have pity on me and send Lazarus to dip the tip of his finger in water and cool my tongue, because I am in agony in this fire.’

Seems pretty socialized to me: everyone has the responsibility to contribute to the needs of the sick, or else they will burn in Hades. It makes me question which Republican candidate Jesus would support....

Not that he'd support Obama, either, a president who has continued a needless state of war, perpetuated our destructive squandering of a precious resource (oil), and overseen a budget which is condemning future generations to supporting a crippling debt load so we can live an unsustainable lifestyle.

It's easy to despair.

added: As I write this, NPR is doing a story on the success of the Massachussetts health care law, which was supported for and signed by then-governor Romney. Rick Santorum has sharply criticized Romney for this. However, this doesn't stop Romney from prominently featuring "repeal Obamacare!" on his policy page.

Wednesday, February 8, 2012

implementing StravaToCsv in Java

In an earlier post I wrote how I put together a "Strava_to_CSV" code using Perl in maybe half an hour. It was pretty easy once I'd installed the Perl JSON libary.

But my goal here is Java, not Perl, so I decided to try to write a similar code in Java. Well, I did it, but it took a lot longer than a half an hour (maybe 5 hours total) and it's a lot longer.

Of course, I know Perl fairly well, while Java I'm trying to learn, so the time is an unfair comparison. Still, in the Java there's stuff to look after.

I won't share the full code here, but will summarize the key points.

First, to get the data from Strava, I once again used their "streams" method from version 1 of the API:


System.err.println("processing Strava activity # " + stravaToCsv.activities[n]);
String urlString = "http://app.strava.com/api/v1/streams/" + stravaToCsv.activities[n];
System.err.println("accessing ride data via URL = " + urlString);
java.net.URL url = null;
try {
  url = new java.net.URL( urlString );
}
catch ( java.net.MalformedURLException exception ) {
  System.err.println( "URL is malformed : $url" );
  System.exit(1);
}
// get the data for the activity
s = GetUrl.getUrl(url);
Here "GetUrl" is a class I wrote based on an example I found on-line:

import java.io.*;
import java.net.*;

// reference for this class:
// http://www.devdaily.com/java/edu/pj/pj010011

class GetUrl {
  public static String getUrl(URL url) {
    InputStream inputStream = null;
    String buffer = null;
    StringBuilder sb = new StringBuilder();

    try {
      inputStream = url.openStream();

      // Convert the InputStream to a buffered DataInputStream.
      BufferedReader d = new BufferedReader(new InputStreamReader(inputStream));


      while ((buffer = d.readLine()) != null) {
        sb.append(buffer);
      }

    } catch (MalformedURLException exception) {

      System.err.println("ERROR malformed URL: " + url.toString());
      exception.printStackTrace();
      System.exit(1);

    } catch (IOException exception) {

      System.err.println("ERROR: I/O exception while reading URL: " + url.toString());
      exception.printStackTrace();
      System.exit(1);

    } finally {
      try {
        inputStream.close();
      } catch (IOException exception) {
        System.err.println("ERROR: I/O exception while closing URL: " + url.toString());
        exception.printStackTrace();
        System.exit(1);
      }
    }
    return sb.toString();
  }
}

So these give me the JSON data. In the Perl script, I cheated and used "wget" which is a separate code (on Apple OS/X, it would have been "curl"). I could have done something similar with Java, but wanted to stick within Java if possible.

The next was then to decode the JSON data. For that I needed a class in which to store it. Here's what I came up with:

class StravaActivityData {
    public String id;
    public double[][] latlng;
    public double[] time;
    public double[] distance;
    public double[] altitude;
    public double[] altitude_original;
    public double[] heartrate;
    public double[] cadence;
    public double[] watts_calc;
    public double[] watts;

    public StravaActivityData() {
    }

    int numPoints() {
        return time.length;
    }
}
The main point is I've defined arrays for all the data types I anticipate might be found in a Strava activity. This is bad, because if Strava changes the fields, I'd need to revise my code. On the other hand, the Perl code had special handling for "latlng" (whose elements are length-two arrays) but it assumed only that other fields would be scalar arrays. So with the Perl code, if Strava were to add fields (like "temperature" or "L-R balance", for example) then the Perl code would automatically pick these up. This class is fairly general, and I anticipate using it in my main project. I didn't want to bog it down with functionality I would use only for the CSV generator. So I then built a class on top of this one specific to the CSV project. Here's that one:

class StravaActivityDataCsv extends StravaActivityData {
    public void printHeader() {
       System.out.println("id,n,lat,lng,time,distance,altitude,altitude_original,cadence,heartrate,watts_calc,watts");
    }
    public void printData() {
      for (int n = 0; n < numPoints(); n ++) {
         System.out.printf("%s,%d,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s\n",
            (id == null)                ? "" : id,
            n,
            (latlng == null)            ? "" : latlng[n][0],
            (latlng == null)            ? "" : latlng[n][1],
            time[n],
            (distance == null)          ? "" : distance[n],
            (altitude == null)          ? "" : altitude[n],
            (altitude_original == null) ? "" : altitude_original[n],
            (cadence == null)           ? "" : cadence[n],
            (heartrate == null)         ? "" : heartrate[n],
            (watts_calc == null)        ? "" : watts_calc[n],
            (watts == null)             ? "" : watts[n]
          );
                               
        }
    }
    public void printCsv() {
        printHeader();
        printData();
    }

}
It extends the prior class with a methods to print a comma-delimited list with available fields. Note fields might be missing, other than time, and so it checks to see if arrays are null before trying to extract components from those arrays. This is a bit cumbersome, but it does the job. I don't like that the list of field is fixed, that it doesn't just use whatever it finds in the JSON stream, but ah, well. So all I needed to do was to store the JSON stream downloaded with the Strava API in that class. But I needed to decode the JSON. I checked on various libraries to do this and saw a recommended for Google's GSON library. I figured if Google use it it must be good, and it did in fact work fairly well once I'd figured out the basics. First, I defined my GSON object:

import com.google.gson.*;
Gson gson = new Gson();
Then decode the JSON, and print the CSV stream:

 // process the string with GSON
 StravaActivityDataCsv stravaActivityDataCsv = gson.fromJson(s, StravaActivityDataCsv.class);
 stravaActivityDataCsv.id = stravaToCsv.activities[n];

 // print out the data
 stravaActivityDataCsv.printCsv();
Okay, so this is just a fragment of the total code. The point here is progress has been made from a starting position of very little. My StravaToCsv was sort a test bed for doing some of the stuff I need to do in my goal app, and so it's encouraging I was able to make it work.

Tuesday, February 7, 2012

Comments on Contador

Clenbuterol

Charles Pelkey wrote a truly excellent commentary on the recent decision to "retroactively" suspend Alberto Contador for trace levels of Clenbuterol in his blood during the 2010 Tour de France. It's probably the most level-headed, well thought out analysis I've yet seen on the matter.

Clenbuterol is hardly an exotic doping agent. Even Sears sells it as a weight-loss supplement. There's a lot of the stuff flying around, and some of it ended up in Contador's blood in that Tour. How? The Court of Arbitration for Sport claims that the most likely method, having considered the alternatives, is a contaminated supplement. Indeed, the levels in Contador's blood were so low that no credible source claims they could have provided a performance boost. Either he ate contaminated food, took a contaminated drug or supplement, or transfused contaminated blood. The Court considered this last possibility, and claimed there was no supporting evidence.

Anyway, Charles' article really is worth reading. Here was my response:

Excellent, excellent article! However, there is one missing point, which is that no sport can thrive in such an environment of instability. Consider the 2011 Giro: Contador was a valid, licensed rider for a highly ranked, licensed team. Had he been denied entry in that race he and his team may well have had grounds to sue. Yet in accepting Contador’s entry, the entire epic race has been turned into a farce. Cycling, other than time trials, is highly tactical, which means the presence of a rider strongly affects the interactions of other riders. You can’t simply remove a rider’s effect by removing him from the standings. And in such a turbulent, uncertain environment, how are sponsors expected to commit to the sport? This is especially true if consideration is given not only to erasing a rider’s personal results retroactively, but also retroactively considering the grounds for a team’s license.

So the system clearly needs to be fixed. The decision, it seems, comes down strongly on the side of strict liability. So given that, no substance should have a 0 threshold. Set quantitative limits for everything. This will reduce the problems of advances in test sensitivity. Second, if the burden of proof is going to be against the rider, the minute a positive B-sample is announced the rider doesn’t race. Cycling cannot tolerate the cascade of destruction which results when a rider continues to participate and results are retroactively revised.

Eddy Merckx has said this case proves people are trying to kill cycling. Whether there is intent or not, further cases like this may well accomplish that end, at least in the form we know it now.

While it's easy to believe Contador was doping in the late naughts, this whole thing just doesn't sit well with me. It's one thing to be caught with performance-enhancing substances, but in this case he was caught with concentrations which enhanced nothing except the income of a lot of attorneys.

Sunday, February 5, 2012

update on Java coding project

While it seems at times my early progress on my Android/Strava app project has stagnated, really I've gotten a decent amount done.

On the computer side, I had to upgrade the Ubuntu on my Thinkpad T60 from 9.10 to 11.10 to allow for the installation of current packages. This was inhibited by an inability to access my CD drive, but a bit of isopropyl alcohol blown off with a blast of dry compressed air (borrowed from the tech support at work... shh!) on the sensitive bits seemed to fix that. As with my previous Ubuntu upgrade, I lost a lot of packages in the shuffle, even things as fundamental as emacs, but as these omissions are encountered I reinstall them. Things are basically up and running now.

Second, I gave in and ordered a Mac Air. The Thinkpad is in a fairly sorry state at this point, and I really think the Mac will provide a better machine for my needs. I'll get that in another week.

Then education: The Oracle (formerly Sun) Java Tutorial is quite good. I admit it's a bit thick going when contemplating when one chooses static versus dynamic nested classes, or anonymous versus named sub-classes, or how to organize classes and objects. But looking at some Perl code I've written, for example to calculate the rating of climbs based on the profile data, I welcome switching to an object oriented model. It at least offers the promise of enforcing a bit more localization and structure to things. With Perl it's too easy to fall into bad habits.

So to that end, I started "sketching out" the class structure for my application. I've even started coding some of the key methods: as I've noted, the key is to break down complex problems into simpler ones. For example, a key for processing GPS data is to apply reasonable smoothing functions. So I have a class for Strava activities, a sub-class for the ride data, a generic private method for applying smoothing functions, then specific public method for smoothing altitude, speed, and power. Another nested class in the Strava activity class is the header information: for example the "athlete", the date, time, etc. Then a final nested class I use for derived elements. I need to write methods to determine these derived elements, which will be public methods in the derived element nested class.

For the final execution, two candidates are server-side web and Android app. For the Android App, I'm well into Android Application Development for Dummies by Donn Felker. Although it does have the occasional typo to keep a reader on his feet, it's an excellent book, written with the goal of removing the intimidation factor from writing Android apps. Don't make the mistake of assuming the title implies the material is dumbed down: there's plenty of meat there. Right now I'm only slightly intimidated. But I won't really appreciate the task at hand until I dig into the Eclipse development environment used in the book's project and have at it. A mobile phone is not a super-stable or super-friendly platform for computation, so things must be done with a degree of care.

For the server side option, I've been reading Robin Nixon's book in the famous O'Rielly series. This looks like it may well be more benefit in some Low-Key Hillclimb coding I want to do to allow people to access historical results. But that's on the back burner for now. For server-side Java, which I want to use for this project, I'd need a bit more. Certainly if I'm thinking of both phone and web I want to reuse as much code as possible, and that implies using the same language for each, and Java is the obvious choice if the phone is going to be Android. iPhone apps, for example, tend to be built in Objective C (see, for example, iPhone App Development for Dummies, by Neal Goldstein). I've never written in Objective-C, but by all accounts like Java it's a good language, much better than C++, which I really dislike. So I'm still a bit fuzzy on the server-side web option: that's not my focus right now.

On the Java side, Strava uses JSON-formatted data so I've made good progress with the Google GSON library for Java. This seems among the simpler JSON libraries for Java, making it relatively straightforward to both encode and decode the sort of streams Strava uses. I've got the basic GSON examples running (I was hacking on those while listening to the 4th quarter of the superbowl on the radio). Next up is to write a little Java version of my Strava-to-CSV converter to make sure I have that nailed down. One thing I'll say for Perl is it took just a few minutes to dash off that code, while Java with its greater formality is a bit slower going.

So this project has been picking away at the perimeters of the problem. Honestly I'm still a bit worried about whether I'm going to reach my 29 Feb deadline for a working prototype. It'll be fun.

Saturday, February 4, 2012

commuting home on Potrero Hill

Since returning home from my wonderful New Zealand trip I've definitely been in a rut. The rut is a "go to work, work through lunch, come home in the evening, eat dinner and go to bed" rut. It's a "I don't have time or energy to go for a ride/run/gym/whatever" rut. A rut where I can't leave early enough to ride to work, where I need to get in too early to ride to work, and so I take the train instead.

It's a bit depressing. Being on vacation is being intensely alive: every minute of the day is one to be savored. "Normal life" is only a shadow of that, finding intensity in the fringes of our days.

But while I've not yet SF2G'ed this year, I've managed at least a few mid-say workouts. First last Friday I did a bit of a solo ride, hoping to catch the Apple lunch ride but unaware that it had been delayed. Then on Tuesday there was a group run from work, which was really nice. 17 km is a bit more than is justified from my recent activity, but a run I've handled many times before, and although it left me a bit tired on Wednesday I was fine. I was fine enough that I repeated it solo on Thursday: slow by historical standards but speed wasn't the priority. Running at lunch is interesting: it takes a big effort to get me going (too much to do, want to eat instead) but once I'm a block down the road all is good and I don't want to stop.

But even when I do "nothing", taking the train instead of riding in, I virtually always face the same thing when I get home in the evening. Motivation isn't an issue here: if I want to get home, I need to ride up Potrero Hill. Sure, I could walk. Sure, I could even take a taxi. (Bus? Dream on... a pogo stick would be faster). And sure, even riding there's a less direct route I could take with less total climbing. But the best route is the most direct, and that means every evening, there's going to be some pain. Pain is good.

Here's a profile taken from my new (purchased on eBay) Edge 500. As I've noted here, my previous one disappeared into the vapor of lost objects.

Potrero Hill commute

Back before I moved to San Francisco, so long ago, I used to seek out steep climbs to strengthen my legs. This is no longer necessary. Now the hills, or rather hill, seeks out me.

That penultimate climbing block, 20th from De Haro to Rhode Island, is the one where I know it's going to hurt a bit. I've got four gear options: 34/26 if I'm feeling lazy or want to go fast, 34/23 as my normal "stay honest" gear, 34/21 when I want to make it especially hard, and 34/19 for those special days. The block prior, 20th from Carolina to De Haro, is also tough, but that's quickly forgotten once I cross De Haro. Typically I take that earlier block one gear higher, although I don't think I've ever bothered doing it in bigger than that 34/19.

When going for a Strava time (here's the segment) I ride it differently: sprint on 20th, keep the sprint going past Wisconsin, upshift for the descent, then quickly downshift on the rise to De Haro. With good initial speed, momentum carries you most of the way up this climb. Then ride hard across De Haro (no slacking here!) before downshifting all the way to spin like a crazy guy to Rhode Island. It's all about minimizing lost time, about keeping the pressure on the pedals every second, about carrying as much kinetic energy into the segment as possible.

Ah, the lament. First Jonathan Withrington then Bret Lobree put that segment apparently beyond my reach. I've not made an attempt there in quite awhile. If I try again, I'll really need to hit that first intersection, with Wisconsin, with more speed than I've allowed. I'd want someone to watch for cars for me: I tend to err on the side of caution.

I defined the segment, one of several on Potrero Hill I've done. So why did I set it to start it where I did rather than at the Carolina intersection, at the bottom of the short descent? Because position errors would have a larger influence down there, where just a few meters of position error would take a nice chunk out of that first climbing block. On the other hand, position errors going into the descent have less influence, since the bike covers a given section of road quicker where it's going faster. Position errors may still play a large role at the finish of the segment, but what can you do? It would have been a mistake to extend the segment further, as that would have increased the luck requirement. The intersection with Carolina (bottom of the descent) is crossing a T so is relatively safe. The intersection with De Haro requires luck, but it's at least safe since the bike's going relatively slow here. Having the segment cross Rhode Island would add an additional intersection the rider needed to get across without delay. That didn't seem worth it to me.

Anyway, it's a good character-building commute... not that there ever passes a day I ride it that I don't contemplate the advantages of living lower (thoughts purged from my brain once I reach the top).

Wednesday, February 1, 2012

Email to Nancy Pelosi on federal transportation bill

The house is considering a staggering $260B transportation bill, a bill which places the focus on the usual suspects: roads, cars, and drilling. It's an appalling continuation of the 20th century priorities which got us into such a mess of squandered resources, wasteful land use, and reliance on the whims of unstable governments supplying our oil.

Here's my letter to Nancy Pelosi:

It is foolhardy for us to continue spending valuable resources on a highway network which indirectly increases our reliance on fossil fuels, primarily imported.

For energy independence we need to increase the efficiency of our transportation system, which means federal investments, if any, focused on public transit, not highway programs which are a relic of the last, misguided century.

Please end the foolhardy reliance on highway programs which not only increase the nation's already crippling debt burden, but do nothing to create the sort of jobs or infrastructure we need to compete internationally in the 21st century.

It just goes to further demonstrate the profound incompetence of the federal government, and how badly broken our legislative system is. If I or anyone else thought voting in Obama was going to change things, we were hopeless optimists.