tag:blogger.com,1999:blog-8781383461061929571.comments2018-11-15T15:49:41.260-05:00OR in an OB WorldPaul Rubinhttps://plus.google.com/111303285497934501993noreply@blogger.comBlogger1617125tag:blogger.com,1999:blog-8781383461061929571.post-39003032678881945802018-11-15T13:52:04.299-05:002018-11-15T13:52:04.299-05:00OK, again thanks much. Very helpful and I learned ...OK, again thanks much. Very helpful and I learned something new :-)Unknownhttps://www.blogger.com/profile/05810668373168924861noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-41007459946052958332018-11-15T13:27:22.177-05:002018-11-15T13:27:22.177-05:00I've seen pw-linear functions of two arguments...I've seen pw-linear functions of two arguments, but not in a while, and I can't point to any examples.Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-86142445191095733622018-11-15T13:09:52.732-05:002018-11-15T13:09:52.732-05:00Thanks Paul.
When I include temp2 (capital gains ...Thanks Paul.<br /><br />When I include temp2 (capital gains tax associated with taxable income) in the objective function the cgt constraint works as I want but it has the, for me, unintended side effect of causing the model to minimize income tax which is slightly different from maximizing spendable. The minimization gives unwanted transfers of money from 401k to investment account. All that does lower the taxes and increase the object function while slightly lowering spendable. <br /><br />I think I will try to convert the capital gains function from 2d to 3d and having piecewise linear plains to represent the function. Have you ever seen this done? know of any examples?<br /><br />Thanks Much,Unknownhttps://www.blogger.com/profile/05810668373168924861noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-39441738159246357132018-11-15T11:15:40.601-05:002018-11-15T11:15:40.601-05:00I assume that first constraint was supposed to be ...I assume that first constraint was supposed to be == or >= rather than <=; otherwise the model is unbounded. Assuming that, spendable increases as CG tax decreases, and your model lets you have zero CG tax by setting temp2 equal to temp1. I suspect you need to tie temp2 to income tax in a way that discourages padding temp2, but that is an accounting question and I am (blissfully!) ignorant of accounting.Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-3992184703544826722018-11-14T19:26:40.935-05:002018-11-14T19:26:40.935-05:00Sorry for not being more clear. The object functio...Sorry for not being more clear. The object function I want is to maximize spendable dollars. Call this 'spendable.' Then I have another constraint that takes income sources with their various type of taxes and other expenses to define s. The model is to create a plan for using retirement funds optimally. So as it maximizes after tax spendable money the constraint defining s includes these income(s) minus their taxes. So, yes, the lower the tax the higher the spendable money. <br /><br />maximize spendable<br />s.t. <br />401k Withdrawal + Investment Account Withdrawal - Income tax - CapGainsTax <= spendable<br />Income tax >= 7 linear functions for the income tax brackets x=taxable income<br />temp1 >= 3 linear functions for the capital gains brackets x=taxable income + capital gains <br />temp2 >= 3 linear functions for the capital gains brackets x=taxable income<br />CapGainsTax >= temp1 - temp2<br /><br />I hope that is more clear.<br />Thanks,Unknownhttps://www.blogger.com/profile/05810668373168924861noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-86851290586250538692018-11-14T18:25:48.972-05:002018-11-14T18:25:48.972-05:00I'm not sure I'm following this, but my fi...I'm not sure I'm following this, but my first guess would be that you might be minimizing ctg rather than minimizing total tax. If I were filling out a tax return and focused only on minimizing capital gains tax, I would declare all my income as "ordinary". That would give me not capital gains tax (but would have me paying more tax overall, since I just passed up the lower CG rate in favor of the higher ordinary rate).Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-57531668367573562692018-11-14T17:23:31.837-05:002018-11-14T17:23:31.837-05:00Very useful post. I used it to guide the construct...Very useful post. I used it to guide the construction of a income tax calculation and it works great. I then tried to use it for calculation of capital gains taxes but this does not work. Basically I created the same piecewise linear tax function and then tried ctg >= f(ti+cg) - f(ti) where cgt is the capital gains tax, f() is the piecewise linear function, ti is taxable income and cg is capital gains. What happens is that f(ti) rises to match f(ti+cg) so cgt becomes zero. I don't know how to correct this or how to rewrite the capital gains tax function to get around this. Any ideas?<br />Thanks much,Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-70718070017336515972018-11-12T09:07:30.573-05:002018-11-12T09:07:30.573-05:00As written, no; the function uses lm to fit models...As written, no; the function uses lm to fit models. Hypothetically, you could change lm to glm inside the function to fit generalized linear models. The catch is that the add1 and drop1 functions use a deviance F-test for glm models, so I'm not sure it would truly conform to the original intent (picking variables based on p-values). Still, the deviance test is an F-test, so maybe this would be closer to what you want than what leaps::regsubsets does.Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-7025218761925560792018-11-12T04:57:12.221-05:002018-11-12T04:57:12.221-05:00Thanks for the update Paul.
Question - Does it w...Thanks for the update Paul. <br /><br />Question - Does it work with other type of models than lm? Thank you.Adaszhttps://www.blogger.com/profile/17705078212744425021noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-81235869986615993442018-11-12T04:18:23.387-05:002018-11-12T04:18:23.387-05:00OK, thank you!OK, thank you!Adaszhttps://www.blogger.com/profile/17705078212744425021noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-5768286298425792992018-11-09T16:26:31.215-05:002018-11-09T16:26:31.215-05:00You had the right idea, but the previous code migh...You had the right idea, but the previous code might not have worked as expected with omitted alpha values. I've just updated the code again. You can now skip alpha.to.enter for backward regression, or skip alpha.to.leave for forward regression. Keep an eye out for a short post with the latest updates.Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-5716201448663323482018-11-08T08:41:00.230-05:002018-11-08T08:41:00.230-05:00Hi Paul,
this post at Cross Validated mentions th...Hi Paul,<br /><br />this post at Cross Validated mentions that it is possible to use your function not only as stepwise, but as a backwards and forwards too. Can I ask you how?<br /><br />If I wish to do backward elimination, I guess I should only define the argument full.model and alpha.to.leave? Analogically, for a forward selection, should I start with an initial.model and only enter alpha.to.enter?<br /><br />Thank you!Adaszhttps://www.blogger.com/profile/17705078212744425021noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-72919926146806750562018-10-30T11:46:29.228-04:002018-10-30T11:46:29.228-04:00I'm not sure I was entirely clear in my previo...I'm not sure I was entirely clear in my previous response. The code currently uses p-values for the F test of each coefficient to pick which variable gets in (or out). Those p-values are identical to what you would get for the two-sided t-tests (what you normally see in regression output from software), so it's basically already doing what you had in mind. (One-sided t-tests are possible, but not commonly used in this context.)Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-84507990819986328102018-10-29T18:50:36.868-04:002018-10-29T18:50:36.868-04:00No, that's not it. The first two arguments to ...No, that's not it. The first two arguments to the add1() function are the current model and the full model. The full model is used only as a source of variables to consider. The add1() function takes each variable in full but not in current and evaluates what happens when that variable is added to current. It does not use p-values or t-statistics from the full model.<br /><br />Two questions come to mind. First, what are your choices for alpha-to-enter and alpha-to-leave. Second, what does the code say the p-values are for the "yo-yo" variable (both when it is chosen to enter and when it is chosen to leave)? Those are not currently printed, so you'll need to hack the code to print them (pmin and pmax respectively).Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-76203335357192894872018-10-29T18:28:12.315-04:002018-10-29T18:28:12.315-04:00Yes, that's exactly what is happening. I start...Yes, that's exactly what is happening. I start with "output" ~ 1, and it adds about five variables, then when it adds the sixth, it drops it, then adds it again, then drops it again, and continues to do so until I interrupt it.<br /><br />I looked pretty carefully at the code and I think I know why. When it attempts to add, it looks into the full model results. When it attempts to drop, it looks at the current model results. Brent Danielnoreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-27956051722659791002018-10-29T16:25:17.304-04:002018-10-29T16:25:17.304-04:00Are you saying that a variable enters, exits immed...Are you saying that a variable enters, exits immediately (without any intervening changes to the model), enters again etc.? That should be impossible. If it happened, I would start to wonder whether there was some numerical instability in either the regression fitting or the calculation of the p-values. On the other hand, if it's something like "x enters, y enters, x leaves, z enters, y leaves, x enters, ...", I think that may be possible with annoying correlated predictors ... maybe. I don't recall ever seeing it happen, though. As far as blocking the variable that just left, that won't help in the second case, and might turn the first case into the second case. Hard to say.Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-5869816686002875692018-10-29T16:13:21.585-04:002018-10-29T16:13:21.585-04:00Thanks, Richard. It's an interesting anecdote,...Thanks, Richard. It's an interesting anecdote, and a useful reminder that some problems are in fact sufficiently closely tied to reality that modest improvements to the claimed objective value are verifiable and important. I suspect that many network-type models fit that characterization, especially when the underlying network represents a physical and fairly static structure of some kind (such as a highway system).Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-41529921285341765122018-10-29T15:32:40.838-04:002018-10-29T15:32:40.838-04:00Ok, thank you very much for your reply!Ok, thank you very much for your reply!Unknownhttps://www.blogger.com/profile/04062121076027483953noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-9261405557332225942018-10-29T11:52:51.284-04:002018-10-29T11:52:51.284-04:00Fantastic! My manager is more familiar with SPSS t...Fantastic! My manager is more familiar with SPSS than R and wants to see it done like this. My only issue is that, despite setting alpha to enter less than alpha to exit, my dataset keeps looping with the same variable entering then exiting over and over. I pulled the first variable from the data set that triggered that, and then it started doing that to another variable.<br /><br />I'm thinking about adding some code that stores the previously dropped variable to make sure we don't add it right back in, but I wanted to check in here, first, before messing with it too much.<br /><br />Have you seen that happen? Brent Danielnoreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-81682455256695367802018-10-29T04:45:48.567-04:002018-10-29T04:45:48.567-04:00Regarding your question of "[...] whether the...Regarding your question of "[...] whether there is any benefit to running the solver all the way to "proven optimality"?". I think it depends on the model structure, and whether you get a sensible solution even far away from the proven optimality. I am responsible for a program that solves a very specific version of a capacitated MST, and my users are really sensitive to "good looking" layouts, which typically only occur close to proven optimality. If I stop at, say, 10%, they will come to me and say "this solution is bad", and they loose confidence in my program.<br /><br />So to answer your question: of course it depends, but especially when the quality of the solution is easily visualized, a goofy solution may lead to trust erosion with the users, which helps nobody.Richard Oberdieckhttps://www.blogger.com/profile/03386871665462162276noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-19927348114506975002018-10-28T14:07:47.052-04:002018-10-28T14:07:47.052-04:00Does the code currently allow that? No. Could the ...Does the code currently allow that? No. Could the code be modified to use the p-value? Yes (assuming you are talking about the two-sided p-value, which is what is generated in the summary of lm models in R, and in most other software). Would it make a difference? No. Using two-sided p-values would be equivalent to using alpha-to-enter/alpha-to-leave with a two-sided t-test of each coefficient. If my memory serves me correctly (iffy these days), the two-sided t-test and the F-test are algebraically equivalent (the F statistic is the square of the t statistic), so decisions on which terms were significant (and thus which variables to add or drop) would be identical to what the current code gives.Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-61916280201006003012018-10-27T19:23:12.059-04:002018-10-27T19:23:12.059-04:00Hello, is it possible to use as criteria to add or...Hello, is it possible to use as criteria to add or drop variables the P-Value instead of the F-test?Unknownhttps://www.blogger.com/profile/04062121076027483953noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-4514111081142826272018-10-16T11:22:11.703-04:002018-10-16T11:22:11.703-04:00The comment about strict inequality had to do with...The comment about strict inequality had to do with the consequent, not the antecedent. In other words, "if then " works fine using what I posted. On the other hand, "if then " is trickier because math programming models generally do not tolerate strict inequalities. That limitation does not appear relevant to your case.Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-18395236601989385552018-10-16T00:17:23.932-04:002018-10-16T00:17:23.932-04:00Yes, I do not care about Y value when X is zero. W...Yes, I do not care about Y value when X is zero. What I want is; Y should be "non-positive" when X is "strictly negative", and When X is "strictly positive" then why should take "non-negative values". You mentioned about strictly negative and strictly positive constraints at the end but did not quit get it. Can you please elaborate it a little bit more. Mohan Lalhttps://www.blogger.com/profile/04018479790889820141noreply@blogger.comtag:blogger.com,1999:blog-8781383461061929571.post-88185065642060616692018-10-15T18:30:01.953-04:002018-10-15T18:30:01.953-04:00You have not said what happens to Y if X is zero. ...You have not said what happens to Y if X is zero. If you don't care which sign Y has when X is 0, then the solution above works just fine.Paul Rubinhttps://www.blogger.com/profile/05801891157261357482noreply@blogger.com