Post by thread:[EE]: Fitting Curvaceous Curves

part 1 2847 bytes content-type:text/plain; (decoded 7bit) Well, this is really a math question, although the end product will be PIC code. I'd like to have a figure of merit that would tell me how closely some real world data fits a theoretical curve. I've got a data stream from a process that follows a certain very predictable curve. I've worked out a formula for the curve, and my project is to predict, early in the process, when the process will be finished. [details of the "process" are patent pending - no talkie no tellie] [No, Andy, it's not a toaster. ;-) ] See attached graph. Now we are not talking about a linear process here, that would be so easy I'd not bother posting about it. I've been able to model the process as a nice exponential function. The exponential model has two input variables. The task of my program will be to "guess" at those variables, then test the guess against data that has already been measured, adjust the guess, and test again. The attached graph shows the real world data, and a first "guess" at an estimate. Note that the guess is really close until time gets to about 145 then the guess starts to fail. At this popint the program must adjust it's guess, recalculate the curve, and update it's estimate of time to finish the process. The real world data and the estimated data follows this equation: Forcing Function = 112 - 112exp(-S1*T) = F1 Estimated data = F1 - F1Exp(-S2*T) where F1 is the first equation. This is a nice exponential response that you'd typically see in an RLC circuit. S1 and S2 are the unknown variables, T is time. NOW THE ACTUAL QUESTION: The question is, I'd like a simple mathematical function that I can implement in a PIC that can tell me how good my "fit" of my estimated data to my real world data is. This is sort of a statistics question: how can you tell, mathematically, when two data sets follow the same curve, and if not, how close they are? I've tried some simple items. First I tried sum of the differences between the two curves at each X data point. The result is that a really bad fitting curve can have a sum-of-differences of zero if it crosses symmetrically. Also tried minimizing average of the differences, and median of differences. (these allow big negative numbers) and sum of absolute value of differences (Same problems as sum of differences) None of these really gives a good indication. Hopefully the figure of merit will be some function that takes only a rew bytes of RAM memory locations to calculate (I don't want to have to have thousands of bytes of memory holding an entire history of my data in a PIC) If the PIC could glance at a graph, it would be easy, but a pic has no eyeballs! -- Lawrence Lile Sr. Project Engineer Salton inc. Toastmaster Div. 573-446-5661 Voice 573-446-5676 Fax part 2 8748 bytes content-type:application/octet-stream; (decode) part 3 105 bytes -- http://www.piclist.com hint: To leave the PICList spam_OUTpiclist-unsubscribe-requestTakeThisOuTmitvma.mit.edu

One way ive seen this done is to make a table of the slopes of the "type curve" for a given time interval, then get the slope of the measured data and compare them. that boils down to a table lookup, with linear interpolation if needed. {Quote hidden}> Well, this is really a math question, although the end product will be PIC > code. I'd like to have a figure of merit that would tell me how closely > some real world data fits a theoretical curve. > > I've got a data stream from a process that follows a certain very > predictable curve. I've worked out a formula for the curve, and my project > is to predict, early in the process, when the process will be finished. > [details of the "process" are patent pending - no talkie no tellie] [No, > Andy, it's not a toaster. ;-) ] See attached graph. > > Now we are not talking about a linear process here, that would be so easy > I'd not bother posting about it. I've been able to model the process as a > nice exponential function. > > The exponential model has two input variables. The task of my program will > be to "guess" at those variables, then test the guess against data that has > already been measured, adjust the guess, and test again. The attached graph > shows the real world data, and a first "guess" at an estimate. Note that > the guess is really close until time gets to about 145 then the guess starts > to fail. At this popint the program must adjust it's guess, recalculate > the curve, and update it's estimate of time to finish the process. > > The real world data and the estimated data follows this equation: > > Forcing Function = 112 - 112exp(-S1*T) = F1 > > Estimated data = F1 - F1Exp(-S2*T) > where F1 is the first equation. This is a nice exponential response that > you'd typically see in an RLC circuit. > S1 and S2 are the unknown variables, T is time. > > NOW THE ACTUAL QUESTION: > The question is, I'd like a simple mathematical function that I can > implement in a PIC that can tell me how good my "fit" of my estimated data > to my real world data is. This is sort of a statistics question: how can > you tell, mathematically, when two data sets follow the same curve, and if > not, how close they are? > > I've tried some simple items. First I tried sum of the differences between > the two curves at each X data point. The result is that a really bad > fitting curve can have a sum-of-differences of zero if it crosses > symmetrically. Also tried minimizing average of the differences, and median > of differences. (these allow big negative numbers) and sum of absolute > value of differences (Same problems as sum of differences) None of these > really gives a good indication. > > Hopefully the figure of merit will be some function that takes only a rew > bytes of RAM memory locations to calculate (I don't want to have to have > thousands of bytes of memory holding an entire history of my data in a PIC) > > If the PIC could glance at a graph, it would be easy, but a pic has no > eyeballs! > > > -- Lawrence Lile > Sr. Project Engineer > Salton inc. Toastmaster Div. > 573-446-5661 Voice > 573-446-5676 Fax > -- http://www.piclist.com hint: To leave the PICList .....piclist-unsubscribe-requestKILLspam@spam@mitvma.mit.edu

At 09:31 AM 1/31/01 -0600, Lawrence Lile wrote: > >NOW THE ACTUAL QUESTION: >The question is, I'd like a simple mathematical function that I can >implement in a PIC that can tell me how good my "fit" of my estimated data >to my real world data is. This is sort of a statistics question: how can >you tell, mathematically, when two data sets follow the same curve, and if >not, how close they are? Hi, Lawrence:- Typical "cost" functions used in curve fitting are sum-of-absolute-error and sum-of-error-squared. The latter assigns a higher cost to points that are further off the curve, I tend to use it for simple cases. There are some cases where it allows a closed-form solution to optimization problems, so the mathematicians like it too. In reality, there is really no limit to all the different cost functions you can come up with to reflect your physical situation and numerical needs. You may want to try a few and see if they reflect your "feel" for what is a "good" curve fit and whatever other needs you may have, and adjust it if it is not working out. Best regards, P.S. Nothing is worse than burnt bagels. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Spehro Pefhany --"it's the network..." "The Journey is the reward" speffKILLspaminterlog.com Info for manufacturers: http://www.trexon.com Embedded software/hardware/analog Info for designers: http://www.speff.com Contributions invited->The AVR-gcc FAQ is at: http://www.bluecollarlinux.com =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= -- http://www.piclist.com hint: To leave the PICList .....piclist-unsubscribe-requestKILLspam.....mitvma.mit.edu

>I've tried some simple items. First I tried sum of the differences between >the two curves at each X data point. The result is that a really bad >fitting curve can have a sum-of-differences of zero if it crosses >symmetrically. Also tried minimizing average of the differences, and median >of differences. (these allow big negative numbers) and sum of absolute >value of differences (Same problems as sum of differences) None of these >really gives a good indication. I don't see why the absolute value of the differences has the "same problem" as the sum-of-differences (that they could average out to no error). Nonetheless, the math types always love to SQUARE the error values to force them to be positive. I don't know why, and we'd surely like to avoid doing multiplies, but this is the start down the road to Standard Deviation...statistics weenie I ain't. Barry There are lies, damned lies, and statistics. --Mark Twain -- http://www.piclist.com hint: To leave the PICList EraseMEpiclist-unsubscribe-requestspam_OUTTakeThisOuTmitvma.mit.edu

At 10:53 AM 1/31/01 -0800, you wrote: . Nonetheless, the math types always love to SQUARE >the error values to force them to be positive. I don't know why, Because if you can write the cost function as a sum of squares, the derivative may easy to calculate (compared to absolute values). The minimum(s) or maximum(s) will occur at a point where the derivative is zero, obviously, so you can just set the derivative of the cost function to zero, and that finds the maxima/minima. Much easier said than done in real cases, of course. Best regards, =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Spehro Pefhany --"it's the network..." "The Journey is the reward" speffspam_OUTinterlog.com Info for manufacturers: http://www.trexon.com Embedded software/hardware/analog Info for designers: http://www.speff.com Contributions invited->The AVR-gcc FAQ is at: http://www.bluecollarlinux.com =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= -- http://www.piclist.com hint: To leave the PICList @spam@piclist-unsubscribe-requestKILLspammitvma.mit.edu

> The question is, I'd like a simple mathematical function that I can > implement in a PIC that can tell me how good my "fit" of my estimated data > to my real world data is. This is sort of a statistics question: how can > you tell, mathematically, when two data sets follow the same curve, and if > not, how close they are? > > I've tried some simple items. First I tried sum of the differences between > the two curves at each X data point. The result is that a really bad > fitting curve can have a sum-of-differences of zero if it crosses > symmetrically. Also tried minimizing average of the differences, and median > of differences. (these allow big negative numbers) and sum of absolute > value of differences (Same problems as sum of differences) None of these > really gives a good indication. My first knee jerk reaction is to do a sum of square differences. For linear metric that describes overall error, I would take the square root of the summed result, but you still have a monotonic metric without the square root. I also don't see why the sum of absolute value of differences should suffer from the symmetric error problem. ***************************************************************** Olin Lathrop, embedded systems consultant in Devens Massachusetts (978) 772-3129, KILLspamolinKILLspamembedinc.com, http://www.embedinc.com -- http://www.piclist.com hint: To leave the PICList RemoveMEpiclist-unsubscribe-requestTakeThisOuTmitvma.mit.edu

Thanks to everyone who responded: > This is a (simple case of) a nonlinear least squares problem. > A very good place to find practical solutions for these is to > look at Numerical Recipes in C. > > Hope this helps. > > --ian "Numerical Recipes in C" is a really excellent book for any programmer. Thanks for the tip! Dennis Wrote: >Sum of the squares of the difference. This is a typical stat method used >like in stock market stuff. The advantage is that the results are always >positive irrespective of the change between the two. This would work with a "normal" function. I needed to preserve the "directional" information in the sign of the differences. I kept having no luck using various sum-of-squares schemes. I was able to solve the problem like this: Theoretically my input data should follow the curve F1(t) = 112 - 112 exp (-s1*t) F2(t) = F1(t) - F1(t) exp(-S2*t) = estimate These functions depend on two variables, S1 and S2. So I generated two figures of merit: Data /(1-exp(-S2*t) ) = F3(t) F3(t) - [112 - 112exp(-s*t) ] = figure of merit for S1 variable Data - [(F2(t) - F2(t)exp(-S*t)] = figure of merit for S2 varialble The key to this was to mathematically isolate the S1 variable, by dividing S2 out of the data. I then kept a running sum of these figures of merit. Dividing the figure of merit by the time kept the numbers from climbing to infinity, and made a useful input to a PID control loop that slowly refines the estimates for S1 and S2 until they lock onto the input data. Finding a figure of merit that was only dependant on one variable at a time made all the difference. If anyone is still awake and reading this message at this point, my hat's off to you. -- Lawrence Lile -- http://www.piclist.com hint: The list server can filter out subtopics (like ads or off topics) for you. See http://www.piclist.com/#topics