Parameter choice for optimal model fitting

I implemented the nonlinear least-square fitting scheme I described, and it didn't work. That is, it didn't work until I fixed it.

The initial problem was the parameter choice. This is a classic problem with iterative fitting schemes: parameters which may span a broad range of values, yet which are restricted to being strictly positive (or equivalently, strictly negative) are often best not fit directly. Instead it may be smarter to fit their logarithm, then take the exponential of the result to restore the original parameter. Exponentials are strictly positive, so the logarithm maps the positive numbers to the full range of real numbers, so it's more robust. Additionally, the logarithm increases the significance of differences of small values, decreasing the significance of the same absolute difference of large values. This is typically what is wanted: the difference between τ = 1 and τ = 2 is probably more significant than the difference between τ = 24000 and τ = 24001.

Fortunately, I don't need to do any complicated calculus to change my parameters to their logarithm. I instead use the following simple transformation:

∂ f / ln x = x ∂ f / x

I thus in one easy step change my Jacobian matrix to use the following derivatives, which also happen to be simpler than the originals, which is almost always a sign that I'm on the correct path:

∂ f / ∂ ln P1 = P1 [τ1 / t] [1 − exp(−t / τ1)]

∂ f / ∂ ln P2 = P2 [τ2 / t]1/2 [1 − exp(−[t / τ2]1/2)]

∂ f / ∂ ln τ1 = P1 [ (τ1 / t) (1 - exp[-t / τ1]) - exp(-t / τ1) ]

∂ f / ∂ ln τ2 = 1/2 P2 [ (τ2 / t)1/2 (1 - exp[-(t / τ2)1/2]) - exp(-[t / τ2]1/2) ]

Then, when I get the Δ values from the solution of the linear algebra problem, instead of adding these to the previous values, I do the following:

P1 → P1 exp( Δ ln P1 )
τ1 → τ1 exp( Δ ln τ1 )
P2 → P2 exp( Δ ln P2 )
τ2 → τ2 exp( Δ ln τ2 )

To improve numerical stability further, I can add a damping factor δ :

P1 → P1 exp( δ Δ ln P1 )
τ1 → τ1 exp( δ Δ ln τ1 )
P2 → P2 exp( δ Δ ln P2 )
τ2 → τ2 exp( δ Δ ln τ2 )

Here δ is between 0 (exclusive) and 1 (inclusive), where 1 signifies no damping. The smaller the value, the slower the optimal convergence rate, but the more robust the process. I chose δ = 0.5 initially, but more on that later.

With these changes, I have had good success with the method. I'll show that next.

Comments

Post a Comment

Popular posts from this blog

Marin Avenue (Berkeley)

Image
On Saturday, I got an email from Tim telling letting me know that Murphy's Spring Classics was going to start up Marin Ave. I'd missed that. Murphy announces his courses the day before, and indeed it was already well into the night before when the course announcement went out. I'd skipped over the details, jumping straight to middle and end games. Marin? I laughed at the irony, because not long before when he'd told me he was choosing to repeat the Nifty Twn Fifty which we'd done together last year versus selecting Murphy Mack's Stage Mullett two-day, I responded I didn't think I had it in me to ride Marin Ave two years in succession. Well, so much for that. I was committed, so I went downstairs to swap my 11-26 cassette for my Recon 12-27. The name is easily overlooked: "Marin Ave" fails to strike the same impression as, for example, "Redwood Gulch" or "China Grade". But among those in the know, Marin Ave is infamous i...
Read more

hummingbird feeder physics

Image
I couldn't figure out how the hummingbird feeder worked. Why didn't it overflow? Forces need to balance, of course. Neglecting surface tension, there liquid level is higher in the inner reservoir than in the feeding chamber, so there must be a corresponding pressure difference. Suppose the pressure in the inner chamber were zero. Then the column height difference would need to be atmospheric pressure / (density of liquid × gravity). But this is over 9 meters! Obviously the height difference is only approximately 1% of this. So the pressure difference inside versus outside is only approximately 1%. The inside is only slightly below atmosphere. So air is getting in. How? Does it diffuse through the liquid? If this were the dominant mechanism, it wouldn't take long for the pressure inside to go from 99% to 99.3%, for example, which should be plenty to push the column of liquid down in the inside chamber and thus push liquid out through the holes. It would over...
Read more

Gallium Pro geometry

Image
On Caltrain yesterday, I was surprised to see among the usual set of commuting bikes a SRAM Red-equipped Argon Gallium Pro. The bike immediately stole your attention. The design was one which has been trending: clear-coated carbon with plenty of text detail so the viewer is sure to realize he's gazing upon the result of advanced, proprietary engineering and not at yet another paint job on the same old OEM frames being pumped out of the same old Taiwanese factories. But it worked: the bottom bracket area was huge, the downtube a large-diameter "inverted Kamm tail" design seemingly designed to maximize wind resistance, the top tube a broad, eccentric shape which screamed "vertically compliant yet torsionally stiff". All it lacked were the pencil-think seat stays Cervelo popularized, but this was a machine designed for stiffness over anything else. It seemed dramatic overkill, since the bike was small (the Argon "XS" I suspect). There wasn't muc...
Read more