Repository navigation
BUG: Fix the classical F-statistic of weighted and quasi-demeaned models - #797
Merged
Merged
Conversation
The classical (homoskedastic) F-statistic compares the residual sum of squares of the model with that of the model that only has the constant. The residuals of the restricted model were y - mean(y), where y is the dependent variable after the model weights and transforms it and the mean is the weighted mean. That is only the residual of the regression on the constant if the transformed constant is a column of ones. Weights w make it sqrt(w), and the quasi-demeaning of RandomEffects makes it sqrt(w)(1 - theta_i), which differs between entities when the panel is unbalanced. The restricted sum of squares was wrong, so the F-statistic of every model with weights and a constant (PooledOLS, PanelOLS, BetweenOLS, RandomEffects), and that of RandomEffects and of BetweenOLS with reweight=True when the panel is unbalanced, was wrong, in the example below by a factor of 4. The p-value follows, and so does the F-statistic in the summary. The restricted model is now the regression of the transformed dependent variable on the transformed constant. The transformed constant is the transformed regressors times the coefficients that combine the original regressors into a constant, which a regression of a column of ones on exog finds, so a constant that is implicit, such as the dummy variables of all categories, works as well. Fits without weights whose constant transforms to a column of ones (PooledOLS, PanelOLS, balanced RandomEffects) do not change, and neither do models without a constant. On the wage panel with the hours worked as weights, PooledOLS on a constant, expersq, married and union had F = 348.93 and now has 89.07, which is the F-statistic of the weighted least squares regression of statsmodels, and PanelOLS with entity effects had 806.82 and now has 193.82, the F-test that the slopes are zero in the weighted regression on the dummy variables of the entities. RandomEffects on the unbalanced wage panel had 160.98, and now has 159.61, which is the Wald statistic of the unadjusted covariance. The tests compare the F-statistic with statsmodels (weighted least squares of the observations and of the means of the entities, and the F-test in the regression on the dummy variables of the effects), and check that the classical statistic equals the Wald statistic that uses the unadjusted covariance for every model.
The weights of a model are scaled to have a mean of one once the observations with missing values are dropped. FamaMacBeth.fit took the weights and the data from the arrays that the model saves before this (and not from the weights, dependent variable and regressors that it keeps), so the residuals used the weights as they were given, while the total sum of squares used the scaled weights. The residual sum of squares, the R-squared, s2, the log-likelihood, the weighted residuals and the classical F-statistic depended on the scale of the weights, and on the observations that were dropped, which change the mean of the weights that are kept. Only weights whose mean is one and that are not changed by dropping observations gave the right values. The model uses the retained observations and the scaled weights. The averages of the R-squared and of the adjusted R-squared of the period regressions centered the weighted dependent variable at its unweighted mean, as the F-statistic did. They use the weighted constant. The parameters, the covariance and the fit of models without weights do not change, since the period regressions do not depend on the scale of the weights. On the wage panel with the hours worked as weights, FamaMacBeth on a constant, expersq, married and union had a residual sum of squares of 2,585,729 and an R-squared of -2183.08, and now has 1180.02 and 0.00328, the values from the definition with the weights scaled to a mean of one. Weights divided by 2000 gave -0.092. The tests compare the fit measures with the definition, check that weights that are multiplied by a constant and constant weights give the results of the model without weights, and that dropped observations are handled as manually dropped observations.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #797 +/- ##
==========================================
+ Coverage 99.59% 99.62% +0.02%
==========================================
Files 128 140 +12
Lines 24435 28193 +3758
Branches 1820 1978 +158
==========================================
+ Hits 24336 28086 +3750
- Misses 39 41 +2
- Partials 60 66 +6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The classical (homoskedastic) F-statistic compares the residual sum of squares of the
model with that of the model that only has the constant. The residuals of the restricted
model were y - mean(y), where y is the dependent variable after the model weights and
transforms it and the mean is the weighted mean. That is only the residual of the
regression on the constant if the transformed constant is a column of ones. Weights w
make it sqrt(w), and the quasi-demeaning of RandomEffects makes it sqrt(w)(1 - theta_i),
which differs between entities when the panel is unbalanced. The restricted sum of squares
was wrong, so the F-statistic of every model with weights and a constant (PooledOLS,
PanelOLS, BetweenOLS, RandomEffects), and that of RandomEffects and of BetweenOLS with
reweight=True when the panel is unbalanced, was wrong, in the example below by a factor
of 4. The p-value follows, and so does the F-statistic in the summary.
The restricted model is now the regression of the transformed dependent variable on the
transformed constant. The transformed constant is the transformed regressors times the
coefficients that combine the original regressors into a constant, which a regression of
a column of ones on exog finds, so a constant that is implicit, such as the dummy
variables of all categories, works as well. Fits without weights whose constant
transforms to a column of ones (PooledOLS, PanelOLS, balanced RandomEffects) do not
change, and neither do models without a constant.
On the wage panel with the hours worked as weights, PooledOLS on a constant, expersq,
married and union had F = 348.93 and now has 89.07, which is the F-statistic of the
weighted least squares regression of statsmodels, and PanelOLS with entity effects had
806.82 and now has 193.82, the F-test that the slopes are zero in the weighted regression
on the dummy variables of the entities. RandomEffects on the unbalanced wage panel had
160.98, and now has 159.61, which is the Wald statistic of the unadjusted covariance.
The tests compare the F-statistic with statsmodels (weighted least squares of the
observations and of the means of the entities, and the F-test in the regression on the
dummy variables of the effects), and check that the classical statistic equals the Wald
statistic that uses the unadjusted covariance for every model.