Empirical likelihood

Author: Paul Schrimpf
Date: 2018-11-25

Creative
Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

About this document

This document was created using Weave.jl. The code is available in the course github repository. The same document generates both static webpages and associated jupyter notebooks.

\[ \def\indep{\perp\!\!\!\perp} \def\Er{\mathrm{E}} \def\R{\mathbb{R}} \def\En{{\mathbb{E}_n}} \def\Pr{\mathrm{P}} \newcommand{\norm}[1]{\left\Vert {#1} \right\Vert} \newcommand{\abs}[1]{\left\vert {#1} \right\vert} \DeclareMathOperator*{\argmax}{arg\,max} \DeclareMathOperator*{\argmin}{arg\,min} \def\inprob{\,{\buildrel p \over \rightarrow}\,} \def\indist{\,{\buildrel d \over \rightarrow}\,} \]

Empirical likelihood

An interesting alternative to GMM is (generalized) empirical likelihood (GEL). Empirical likelihood has some appealing higher-order statistical properties. In particular, it can be shown to have lower order asymptotic bias than GMM. See Newey and Smith (2004). Relatedly, certain test statistics based on EL are robust to weak identification (Guggenberger and Smith 2005). In fact, the identification robust tests that we have discusses are all based on the CUE-GMM objective function. The CUE-GMM objetive is a special case of generalized empirical likelihood.

A perceived downside of GEL is that it involves a more difficult looking optimization problem than GMM. However, given the ease with which Julia can solve high dimensional optimization problems, GEL is very feasible.

As in the extremum estimation notes, suppose we have moment conditions such that \[ \Er[g_i(\theta)] = 0 \] where \(g_i:\R^p \to \R^k\) are some data dependent moment conditions. The empirical likelihood estimator solves \[ \begin{align*} (\hat{\theta}, \hat{\pi}) = & \argmax_{\theta,\pi} \frac{1}{n} \sum_i \log(\pi_i) \;\; s.t. \\ & \frac{1}{n} \sum_i \pi_i = 1, \;\; 0\leq \pi \leq 1 \\ & \sum_i \pi_i g_i(\theta) = 0 \end{align*} \]

Generalized empirical likelihood replaces \(\log(\pi)\) with some other convex function \(h(\pi)\), \[ \begin{align*} (\hat{\theta}^{GEL,h}, \hat{\pi}) = & \argmin_{\theta,\pi} \frac{1}{n} \sum_i h(\pi_i) \;\; s.t. \\ & \frac{1}{n} \sum_i \pi_i = 1, \;\; 0\leq \pi \leq 1 \\ & \sum_i \pi_i g_i(\theta) = 0 \end{align*} \] setting \(h(\pi) = \frac{1}{2}(\pi^2-(1/n)^2)\) results in an estimator identical to the CUE-GMM estimator.

A common approach to computing GEL estimators is to eliminate \(\pi\) by looking at the dual problem \[ \hat{\theta}^{GEL} = \argmin_{\theta}\sup_\lambda \sum_i \rho(\lambda'g_i(\theta)) \] where \(\rho\) is some function related to \(h\). See Newey and Smith (2004) for details. There can be some analytic advantages to doing so, but computationally, the original statement of the problem has some advantages. First, there is more existing software for solving constrained minimization problems than for solving saddle point problems. Second, although \(\pi\) is high dimensional, it enters the constraints linearly, and the objective function is concave. Many optimization algorithms will take good advantage of this.

Let’s look at some Julia code. Since the problem involves many variables with linear constraints, it is worthwhile to use JuMP for optimization. The code is very slightly more verbose, but the speed of JuMP (and the Ipopt solver) are worth it.

using JuMP, Ipopt, LinearAlgebra, Distributions, Plots
Plots.gr()
function simulate_ivshare(n,β,γ,ρ)
  z = randn(n, size(γ)[1])
  endo = randn(n, length(β))
  x = z*γ .+ endo
  ξ = rand(Normal(0,sqrt((1.0-ρ^2))),n).+endo[:,1]*ρ
  y = cdf.(Logistic(), x*β .+ ξ)
  return((y=y,x=x,z=z))
end
n = 1000
k = 2
iv = 3
β0 = ones(k)
π0 = vcat(diagm(0=>ones(k)),ones(iv-k,k))
ρ = 0.5
(y,x,z) = simulate_ivshare(n,β0,π0,ρ)

function gi_ivshare(β,y,x,z)
  ξ = quantile.(Logistic(),y) .- x*β
  ξ.*z
end

function gel_jump(y,x,z)
  n,d = size(x)
  n,k = size(z)
  Ty = quantile.(Logistic(),y)   
  m = Model(solver=IpoptSolver())
  @variable(m, 0.0 <= p[1:n] <= 1.0)
  @variable(m, θ[1:d])
  @constraint(m, prob,sum(p)==1.0)
  @constraint(m, momentcon[i=1:k], dot((Ty - x*θ).*z[:,i],p)==0.0)
  @NLobjective(m,Max, sum(log(p[i]) for i in 1:n))
  return(m)
end

m = gel_jump(y,x,z)
@show m

m = Maximization problem with: * 1 linear constraint * 3 quadratic constraints * 1002 variables Solver is Ipopt

solve(m)

This is Ipopt version 3.12.11, running with linear solver mumps. NOTE: Other linear solvers might be more efficient (see Ipopt documentation ).

Number of nonzeros in equality constraint Jacobian…: 16000 Number of nonzeros in inequality constraint Jacobian.: 0 Number of nonzeros in Lagrangian Hessian………….: 7000

Total number of variables……………………….: 1002 variables with only lower bounds: 0 variables with lower and upper bounds: 1000 variables with only upper bounds: 0 Total number of equality constraints……………..: 4 Total number of inequality constraints……………: 0 inequality constraints with only lower bounds: 0 inequality constraints with lower and upper bounds: 0 inequality constraints with only upper bounds: 0

iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_p r ls 0 4.6051712e+03 1.97e+01 1.21e-21 -1.0 0.00e+00 - 0.00e+00 0.00e+0 0 0 1 6.9077661e+03 1.77e+00 2.03e-04 -1.0 9.95e-02 - 1.00e+00 1.00e+0 0h 1 2 6.9077965e+03 5.93e-03 2.25e+00 -1.0 8.95e-01 - 1.00e+00 1.00e+0 0h 1 3 6.9078602e+03 2.54e-06 2.35e-02 -1.0 1.40e-03 - 1.00e+00 1.00e+0 0h 1 4 6.9078602e+03 2.67e-12 1.91e-05 -2.5 9.47e-07 - 1.00e+00 1.00e+0 0h 1 5 6.9078602e+03 3.77e-12 6.72e-07 -3.8 1.40e-06 - 1.00e+00 1.00e+0 0h 1 6 6.9078602e+03 1.81e-13 8.24e-09 -5.7 3.08e-07 - 1.00e+00 1.00e+0 0h 1 7 6.9078602e+03 5.39e-14 5.52e-11 -8.6 1.66e-07 - 1.00e+00 1.00e+0 0h 1 8 6.9078601e+03 7.34e-10 1.38e-08 -12.9 1.96e-05 - 9.95e-01 1.00e+0 0h 1 9 6.9078601e+03 4.60e-11 1.48e-11 -12.9 4.90e-06 - 1.00e+00 1.00e+0 0h 1

Number of Iterations….: 9

                               (scaled)                 (unscaled)

Objective……………: 6.9078600966979134e-05 6.9078600966979129e+0 3 Dual infeasibility……: 1.4813050661395008e-11 1.4813050661395008e-0 3 Constraint violation….: 4.5974868335095509e-11 4.5974868335095509e-1 1 Complementarity………: 1.2668033762555648e-13 1.2668033762555648e-0 5 Overall NLP error…….: 4.5974868335095509e-11 1.4813050661395008e-0 3

Number of objective function evaluations = 10 Number of objective gradient evaluations = 10 Number of equality constraint evaluations = 10 Number of inequality constraint evaluations = 0 Number of equality constraint Jacobian evaluations = 10 Number of inequality constraint Jacobian evaluations = 0 Number of Lagrangian Hessian evaluations = 9 Total CPU secs in IPOPT (w/o function evaluations) = 0.173 Total CPU secs in NLP function evaluations = 0.028

EXIT: Optimal Solution Found.

@show getvalue(getindex(m,:θ) )

getvalue(getindex(m, :θ)) = [0.993265, 0.977254]

@show getvalue(getindex(m,:p) )

getvalue(getindex(m, :p)) = [0.000999579, 0.001001, 0.0010052, 0.00099616, 0.000990742, 0.0010011, 0.00098992, 0.000990238, 0.00100151, 0.000992317, 0 .000989084, 0.000998148, 0.00101457, 0.000997146, 0.0010037, 0.00100347, 0. 00101211, 0.00100065, 0.00100236, 0.000996729, 0.000999205, 0.000995715, 0. 000981385, 0.000997931, 0.00101592, 0.000912492, 0.00100569, 0.000954694, 0 .00100369, 0.00100766, 0.000998258, 0.000999524, 0.00101471, 0.000986877, 0 .00098182, 0.00101413, 0.000997771, 0.00100061, 0.00100171, 0.00101518, 0.0 0100549, 0.000995796, 0.000999688, 0.000996205, 0.00101358, 0.000999319, 0. 00100452, 0.00100122, 0.00101472, 0.0010026, 0.0010048, 0.000999952, 0.0010 0382, 0.000999883, 0.000991858, 0.000987051, 0.00100783, 0.000996135, 0.001 00324, 0.00101557, 0.00101007, 0.000999776, 0.000976365, 0.000994499, 0.001 00081, 0.00101514, 0.00100982, 0.000996065, 0.00100964, 0.000987885, 0.0009 99787, 0.00100987, 0.00102219, 0.000989002, 0.00100731, 0.00100102, 0.00099 9618, 0.00100044, 0.00100367, 0.00102295, 0.00100679, 0.000997983, 0.001007 11, 0.00099704, 0.000990827, 0.000995387, 0.00105951, 0.000997845, 0.000999 96, 0.000999571, 0.000992001, 0.000999758, 0.000997398, 0.00101006, 0.00101 41, 0.000985244, 0.000989646, 0.000991261, 0.00100002, 0.00100754, 0.000989 215, 0.000968021, 0.00100088, 0.00100992, 0.00099017, 0.00099147, 0.0009969 38, 0.00100318, 0.00100183, 0.00100152, 0.00100971, 0.00100037, 0.000972568 , 0.00100631, 0.000990139, 0.00100163, 0.00101576, 0.000998955, 0.00100869, 0.000978983, 0.000982131, 0.00100663, 0.000978668, 0.00101752, 0.00103939, 0.00103764, 0.000992057, 0.00100643, 0.00102319, 0.00101019, 0.000991725, 0.00100513, 0.00100241, 0.000996393, 0.00100189, 0.000998679, 0.00100453, 0 .00100091, 0.000985416, 0.000983434, 0.000990166, 0.0010009, 0.000998826, 0 .000967677, 0.000979264, 0.0010004, 0.00100127, 0.0010094, 0.000998864, 0.0 00982004, 0.000931135, 0.00100883, 0.000996369, 0.000977097, 0.00100598, 0. 00100905, 0.000999327, 0.000998302, 0.00103526, 0.000998257, 0.00100165, 0. 000988898, 0.000995233, 0.0010012, 0.000987345, 0.000986605, 0.00101382, 0. 00100906, 0.00098259, 0.0010044, 0.00100221, 0.000999068, 0.00100391, 0.001 00905, 0.00100419, 0.000999821, 0.00100097, 0.0010017, 0.000982241, 0.00099 9005, 0.00100331, 0.00102673, 0.00099002, 0.000995627, 0.0010035, 0.0010056 1, 0.00100002, 0.000988399, 0.000997987, 0.000982996, 0.00099615, 0.0009968 5, 0.000997772, 0.00102231, 0.000989627, 0.000999251, 0.0010076, 0.00099614 8, 0.000998877, 0.000992649, 0.00100142, 0.00100273, 0.000999824, 0.0009975 8, 0.00100095, 0.0010022, 0.000995069, 0.000989842, 0.000998808, 0.00097355 4, 0.0010177, 0.00099748, 0.00104028, 0.000995102, 0.0010031, 0.00100847, 0 .00100002, 0.0010018, 0.00100384, 0.000998201, 0.00102387, 0.000996548, 0.0 00984847, 0.000996849, 0.000995233, 0.00101295, 0.00100086, 0.000999889, 0. 00100788, 0.000996197, 0.00100444, 0.00102443, 0.00100207, 0.00100519, 0.00 0999701, 0.00102436, 0.00100035, 0.00101195, 0.00103835, 0.00101607, 0.0010 0757, 0.00092496, 0.00100835, 0.000958315, 0.00100111, 0.000978982, 0.00097 868, 0.00100004, 0.000999558, 0.000996593, 0.000993488, 0.00100454, 0.00096 9029, 0.000988582, 0.000999102, 0.00102095, 0.000998871, 0.00101019, 0.0010 1915, 0.000992052, 0.00098821, 0.00101527, 0.000995389, 0.000983722, 0.0009 94985, 0.000970853, 0.00103024, 0.000996164, 0.000986902, 0.00100084, 0.001 00176, 0.00100342, 0.00106773, 0.00100069, 0.000995473, 0.000999901, 0.0010 0262, 0.00101333, 0.000999781, 0.000972793, 0.000991225, 0.00102338, 0.0009 95767, 0.000996423, 0.00100014, 0.000994601, 0.00101726, 0.000992078, 0.001 00226, 0.000958525, 0.000996069, 0.000992591, 0.000991997, 0.00100227, 0.00 0997292, 0.00100166, 0.00101599, 0.000996781, 0.000998562, 0.00100084, 0.00 100578, 0.000999033, 0.00100132, 0.00102986, 0.000998906, 0.000993196, 0.00 0985741, 0.000985802, 0.00100025, 0.00102756, 0.00100193, 0.0010389, 0.0010 2006, 0.00100473, 0.00098935, 0.000996179, 0.000992496, 0.000972067, 0.0009 95801, 0.000998889, 0.0010027, 0.000999097, 0.00099973, 0.000995998, 0.0009 96746, 0.000979756, 0.000996921, 0.000963714, 0.000999955, 0.000999636, 0.0 0100564, 0.00100185, 0.000981755, 0.00099572, 0.00100385, 0.000994332, 0.00 0967636, 0.00099315, 0.00100671, 0.000992556, 0.000999185, 0.000999113, 0.0 0102065, 0.000999433, 0.00102761, 0.00100364, 0.00101914, 0.00100362, 0.001 02378, 0.000986652, 0.00100487, 0.00100195, 0.00100559, 0.00100642, 0.00100 556, 0.000996216, 0.00100312, 0.00100161, 0.000998858, 0.00100639, 0.001009 54, 0.000996513, 0.00100052, 0.000999056, 0.00102475, 0.00100023, 0.0009879 24, 0.00101995, 0.000991825, 0.00101149, 0.000998033, 0.000996081, 0.000992 263, 0.000997026, 0.000999748, 0.000986292, 0.000997486, 0.00100049, 0.0010 0011, 0.000999711, 0.00100008, 0.0010015, 0.000996922, 0.00101307, 0.000989 68, 0.00102237, 0.00100609, 0.000976434, 0.000990297, 0.00100235, 0.0009996 31, 0.00102101, 0.000997567, 0.000997097, 0.000998177, 0.00100889, 0.001022 93, 0.000994835, 0.00100829, 0.000996151, 0.000997298, 0.000992363, 0.00099 9691, 0.000997987, 0.000996693, 0.00100862, 0.00101454, 0.000977515, 0.0009 93234, 0.00101492, 0.00100201, 0.000996583, 0.00100131, 0.000984881, 0.0010 0793, 0.0010182, 0.000985866, 0.000998828, 0.00101149, 0.00098971, 0.000992 119, 0.00100172, 0.000997423, 0.000987682, 0.00100172, 0.00100253, 0.001002 25, 0.00100182, 0.000999799, 0.00101511, 0.00100016, 0.000999994, 0.0010069 9, 0.000999983, 0.000999185, 0.000992491, 0.00106681, 0.00101708, 0.0009999 66, 0.00100177, 0.00100008, 0.00100206, 0.00100711, 0.00100801, 0.000978201 , 0.00100013, 0.00100662, 0.00100129, 0.00102363, 0.000999317, 0.00100049, 0.00100797, 0.000967998, 0.00100242, 0.000970022, 0.00099562, 0.00100495, 0 .000997382, 0.00100567, 0.00100323, 0.00099476, 0.00104427, 0.00100623, 0.0 0100264, 0.000974404, 0.00102137, 0.00100277, 0.00103073, 0.00100401, 0.000 969464, 0.000997903, 0.000957741, 0.00101513, 0.000984382, 0.000998095, 0.0 0100402, 0.000998871, 0.000999021, 0.000984615, 0.00100105, 0.000995723, 0. 000980773, 0.00100919, 0.00100068, 0.00101603, 0.00101145, 0.00098827, 0.00 100923, 0.0010102, 0.000999263, 0.00110507, 0.00100937, 0.000998721, 0.0010 0037, 0.000994635, 0.000992802, 0.000972059, 0.00100704, 0.000995525, 0.000 998155, 0.00099894, 0.000985325, 0.00100901, 0.000998763, 0.00100927, 0.000 995235, 0.000999758, 0.00101139, 0.000987863, 0.000998135, 0.0010097, 0.000 998154, 0.00100131, 0.00099753, 0.000967663, 0.000986516, 0.00100063, 0.000 999854, 0.000993385, 0.00100217, 0.000994578, 0.00097275, 0.000987496, 0.00 0998056, 0.000999483, 0.000999901, 0.00100099, 0.00100131, 0.000950117, 0.0 00999998, 0.00100856, 0.000999403, 0.0010022, 0.00100103, 0.00100353, 0.000 999906, 0.000999127, 0.00100091, 0.00100629, 0.00101485, 0.00100029, 0.0009 88283, 0.00100124, 0.000998263, 0.00100091, 0.000992639, 0.000993534, 0.000 999352, 0.00101413, 0.000999404, 0.000992041, 0.000999234, 0.00102388, 0.00 100208, 0.00099619, 0.000954278, 0.00100039, 0.000998805, 0.00100055, 0.001 0066, 0.00100089, 0.000997242, 0.00100788, 0.00101886, 0.000985496, 0.00103 73, 0.00101049, 0.00101007, 0.00100007, 0.000996848, 0.000998926, 0.0009794 63, 0.00102743, 0.000996831, 0.00100219, 0.00100868, 0.000998466, 0.0010011 , 0.000983647, 0.000994973, 0.000993529, 0.000997854, 0.000999856, 0.000996 046, 0.000991379, 0.00100013, 0.00100568, 0.000998466, 0.0010011, 0.0009805 81, 0.00100678, 0.00100562, 0.00099318, 0.000993936, 0.000991791, 0.0009944 33, 0.00101472, 0.000996209, 0.0010037, 0.000987543, 0.000996507, 0.0009965 74, 0.00101339, 0.000981223, 0.000959346, 0.00100026, 0.00100123, 0.0009897 25, 0.00101172, 0.000998255, 0.000997, 0.000996832, 0.000999607, 0.00099889 3, 0.000969592, 0.00101518, 0.000999878, 0.00101112, 0.00100165, 0.00100426 , 0.000998652, 0.00100509, 0.000976772, 0.00100793, 0.00100012, 0.000999692 , 0.000999563, 0.0010055, 0.000987032, 0.00100732, 0.000995264, 0.000996141 , 0.00100613, 0.00100109, 0.00099272, 0.000999644, 0.001006, 0.00100037, 0. 00100299, 0.00100209, 0.000995505, 0.00100009, 0.000996364, 0.00101332, 0.0 0100003, 0.000998264, 0.00100189, 0.000990451, 0.000998073, 0.000999018, 0. 00100066, 0.00100042, 0.00100611, 0.000998921, 0.00100352, 0.00104529, 0.00 0999766, 0.00099695, 0.000992672, 0.000969217, 0.0010103, 0.000997474, 0.00 0998345, 0.0010292, 0.00103199, 0.00102398, 0.00098659, 0.00100219, 0.00100 338, 0.000999858, 0.00101274, 0.000999993, 0.000999794, 0.000999121, 0.0009 99332, 0.00100615, 0.000999821, 0.00102796, 0.001005, 0.00099736, 0.0010068 2, 0.00100285, 0.000998313, 0.000981736, 0.000999045, 0.000993601, 0.000975 443, 0.0010001, 0.000991091, 0.00100489, 0.00100778, 0.000985412, 0.0010031 4, 0.000990977, 0.000999558, 0.00099008, 0.000999941, 0.00100362, 0.0010079 8, 0.00100034, 0.000989814, 0.000997714, 0.000997094, 0.000982219, 0.001003 1, 0.000996273, 0.000999286, 0.0010192, 0.00102132, 0.00098091, 0.00100094, 0.000998621, 0.00100641, 0.0010082, 0.000997637, 0.000953782, 0.000979934, 0.000997979, 0.000996817, 0.00100148, 0.000988524, 0.00101334, 0.00097686, 0.000977804, 0.000990614, 0.000993291, 0.000991747, 0.000949019, 0.0009768 49, 0.00099382, 0.000990151, 0.00102611, 0.00100362, 0.000997519, 0.0010020 6, 0.00100023, 0.000977629, 0.00100138, 0.00100208, 0.00100252, 0.000998927 , 0.00100923, 0.00101064, 0.00100678, 0.000999287, 0.00100442, 0.000996126, 0.00100864, 0.00100347, 0.000997935, 0.000995566, 0.00101783, 0.00102043, 0.00100754, 0.000983038, 0.000991466, 0.00100029, 0.00102503, 0.00100714, 0 .00100521, 0.000995159, 0.00100616, 0.00100125, 0.00100386, 0.00101415, 0.0 00992913, 0.00101511, 0.00100077, 0.00099819, 0.000993905, 0.00100461, 0.00 0998781, 0.000998152, 0.00101365, 0.000997012, 0.000979249, 0.00100083, 0.0 00999969, 0.000994952, 0.00100901, 0.000956523, 0.0010002, 0.00100005, 0.00 100724, 0.000997035, 0.00100916, 0.00100071, 0.000997538, 0.00101616, 0.001 0059, 0.0010084, 0.00100065, 0.000996286, 0.000997123, 0.00100395, 0.000955 818, 0.00100962, 0.000996784, 0.00100731, 0.00100871, 0.00100223, 0.0009931 3, 0.00100948, 0.00100432, 0.000999836, 0.00100052, 0.00100456, 0.000969254 , 0.000977766, 0.00098976, 0.000983431, 0.00100502, 0.000991988, 0.00103888 , 0.000993616, 0.00100668, 0.00102077, 0.00100001, 0.00100231, 0.00101301, 0.000990386, 0.00101333, 0.000995084, 0.00100519, 0.00101528, 0.00100239, 0 .000999397, 0.00097871, 0.000996062, 0.00100284, 0.00100314, 0.00099054, 0. 0010553, 0.000988872, 0.000966162, 0.00099369, 0.000996205, 0.00100334, 0.0 0100797, 0.00101293, 0.000999631, 0.00101162, 0.00100033, 0.00100301, 0.001 03034, 0.000965661, 0.000996692, 0.000977557, 0.000996987, 0.00100418, 0.00 100422, 0.00102121, 0.00100629, 0.000996749, 0.000982121, 0.000945065, 0.00 0997412, 0.00100107, 0.000997798, 0.000998587, 0.000995997, 0.000992113, 0. 00100092, 0.000980724, 0.000996769, 0.00100508, 0.000974906, 0.00100749, 0. 000999265, 0.0010099, 0.000995268, 0.00100176, 0.00100599, 0.00101265, 0.00 100836, 0.000986554, 0.000995884, 0.00101164, 0.00102114, 0.00100968, 0.001 01913, 0.00101209, 0.00101907, 0.00100824, 0.00095917, 0.00100538, 0.001000 22, 0.00101255, 0.00100017, 0.00100033, 0.00100791, 0.000994424, 0.00099920 7, 0.000982549, 0.00100023, 0.00100564, 0.00100483, 0.000994677, 0.0010036, 0.000999589, 0.00100837, 0.000999956, 0.00100235, 0.000999236, 0.00100432, 0.0010009, 0.00101708, 0.000991521, 0.000975949, 0.00100377, 0.00100119, 0 .00100541, 0.00100014, 0.00100224, 0.00103173, 0.00099033, 0.00100065, 0.00 0999436, 0.00102088, 0.000966687, 0.00100062, 0.000987211, 0.00099391, 0.00 100491, 0.00100328, 0.000970584, 0.00102084, 0.00100613, 0.000990691, 0.001 00728, 0.00100265, 0.0010057, 0.000989377, 0.00101042, 0.00103954, 0.000992 904, 0.000953314, 0.000982558, 0.00100146, 0.000995608, 0.00100852, 0.00100 29, 0.00102352, 0.000989508, 0.00099985, 0.000983183, 0.00100587, 0.0010039 5, 0.00101888, 0.00099754, 0.00100066, 0.00099981, 0.00100946, 0.000996384, 0.00100028, 0.00100054, 0.000997224, 0.000999701, 0.00100015, 0.000995859, 0.00099404, 0.000992882, 0.00103776, 0.00100715, 0.00100716, 0.00100007, 0 .000981372, 0.000968858, 0.00100046, 0.00100008, 0.000997447, 0.00101985, 0 .00104103, 0.00100351, 0.00100985, 0.00100749, 0.000998268, 0.000992976, 0. 0010508, 0.000998078, 0.00100003, 0.0009986, 0.000972138, 0.000995393, 0.00 104112, 0.00101326, 0.000994109, 0.00100495, 0.00100219, 0.000996534, 0.000 990901, 0.00100562, 0.00100019, 0.00101735, 0.000992731, 0.00101526, 0.0010 0049, 0.000999036, 0.00100872, 0.000991884]

using BenchmarkTools
m.solver=IpoptSolver(print_level=0)
@btime (()->(setvalue(getindex(m,:θ),zeros(length(β0)));
             setvalue(getindex(m,:p),fill(1/n,n));
             solve(m)))()

33.564 ms (245553 allocations: 4.67 MiB)

m.solver=IpoptSolver(print_level=5)

For comparison here is how long it takes JuMP + Ipopt to solve for the CUE-GMM estimator.

function gmmObj(θ,gi,W)
  g = gi(θ)
  m = mean(g,dims=1)
  (size(g)[1]*( m*W*m')[1]) # return scalar, not 1x1 array
end

function cue_jump(y,x,z)
  n,d = size(x)
  n,k = size(z)
  cueobj = (θ1,θ2)->gmmObj([θ1,θ2],β->gi_ivshare(β,y,x,z),
                           inv(cov(gi_ivshare([θ1,θ2],y,x,z))))
  m = Model(solver=IpoptSolver())
  JuMP.register(m, :cueobj, 2, cueobj, autodiff=true)
  @variable(m, θ[1:d])
  @NLobjective(m,Min,cueobj(θ[1],θ[2])) 
  return(m)
end
@show mcue = cue_jump(y,x,z)

mcue = cue_jump(y, x, z) = Minimization problem with: * 0 linear constraints * 2 variables Solver is Ipopt

solve(mcue)

This is Ipopt version 3.12.11, running with linear solver mumps. NOTE: Other linear solvers might be more efficient (see Ipopt documentation ).

Number of nonzeros in equality constraint Jacobian…: 0 Number of nonzeros in inequality constraint Jacobian.: 0 Number of nonzeros in Lagrangian Hessian………….: 0

Total number of variables……………………….: 2 variables with only lower bounds: 0 variables with lower and upper bounds: 0 variables with only upper bounds: 0 Total number of equality constraints……………..: 0 Total number of inequality constraints……………: 0 inequality constraints with only lower bounds: 0 inequality constraints with lower and upper bounds: 0 inequality constraints with only upper bounds: 0

iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_p r ls 0 3.8501216e+02 0.00e+00 5.16e+01 0.0 0.00e+00 - 0.00e+00 0.00e+0 0 0 1 2.5557029e+02 0.00e+00 3.11e+02 -11.0 5.16e+01 - 1.00e+00 3.12e-0 2f 6 2 2.0215557e+02 0.00e+00 3.01e+02 -11.0 1.38e+00 - 1.00e+00 5.00e-0 1f 2 3 9.5492269e+01 0.00e+00 2.24e+02 -11.0 3.09e-01 - 1.00e+00 1.00e+0 0f 1 4 4.7444578e+01 0.00e+00 5.35e+02 -11.0 2.60e-01 - 1.00e+00 1.00e+0 0f 1 5 1.2168399e+01 0.00e+00 3.12e+02 -11.0 5.97e-01 - 1.00e+00 2.50e-0 1f 3 6 1.1109557e+01 0.00e+00 2.44e+02 -11.0 1.14e-01 - 1.00e+00 1.00e+0 0f 1 7 3.7073052e-01 0.00e+00 3.48e+01 -11.0 4.38e-02 - 1.00e+00 1.00e+0 0f 1 8 2.2362585e-01 0.00e+00 9.38e+00 -11.0 1.31e-02 - 1.00e+00 1.00e+0 0f 1 9 2.0888379e-01 0.00e+00 5.12e-01 -11.0 3.25e-03 - 1.00e+00 1.00e+0 0f 1 iter objective inf_pr inf_du lg(mu) ||d|| lg(rg) alpha_du alpha_p r ls 10 2.0884334e-01 0.00e+00 9.05e-02 -11.0 1.75e-04 - 1.00e+00 1.00e+0 0f 1 11 2.0884245e-01 0.00e+00 2.16e-03 -11.0 1.89e-05 - 1.00e+00 1.00e+0 0f 1 12 2.0884245e-01 0.00e+00 6.10e-05 -11.0 5.24e-07 - 1.00e+00 1.00e+0 0f 1 13 2.0884245e-01 0.00e+00 4.91e-07 -11.0 2.16e-08 - 1.00e+00 1.00e+0 0f 1 14 2.0884245e-01 0.00e+00 1.86e-08 -11.0 1.48e-10 - 1.00e+00 1.00e+0 0f 1 15 2.0884245e-01 0.00e+00 1.55e-10 -11.0 3.84e-12 - 1.00e+00 1.00e+0 0f 1

Number of Iterations….: 15

                               (scaled)                 (unscaled)

Objective……………: 2.0884244652851991e-01 2.0884244652851991e-0 1 Dual infeasibility……: 1.5482407023093003e-10 1.5482407023093003e-1 0 Constraint violation….: 0.0000000000000000e+00 0.0000000000000000e+0 0 Complementarity………: 0.0000000000000000e+00 0.0000000000000000e+0 0 Overall NLP error…….: 1.5482407023093003e-10 1.5482407023093003e-1 0

Number of objective function evaluations = 36 Number of objective gradient evaluations = 16 Number of equality constraint evaluations = 0 Number of inequality constraint evaluations = 0 Number of equality constraint Jacobian evaluations = 0 Number of inequality constraint Jacobian evaluations = 0 Number of Lagrangian Hessian evaluations = 0 Total CPU secs in IPOPT (w/o function evaluations) = 0.008 Total CPU secs in NLP function evaluations = 0.005

EXIT: Optimal Solution Found.

@show getvalue(getindex(mcue,:θ) )

getvalue(getindex(mcue, :θ)) = [0.993236, 0.977304]

mcue.solver=IpoptSolver(print_level=0)
@btime (()->(setvalue(getindex(mcue,:θ),zeros(length(β0)));
             solve(mcue)))()

12.425 ms (2803 allocations: 10.54 MiB)

mcue.solver=IpoptSolver(print_level=5)

Bootstrap for EL

For bootstrapping GMM, we discussed how it is important that the null hypothesis holds in the bootstrapped data. In GMM we did this by substracting the sample averages of the moments. In GEL, an alternative way to impose the null, is to sample the data with probabilities \(\hat{\pi}_i\) instead of with equal proability. See Brown and Newey (2002) for more information.

Brown, Bryan W, and Whitney K Newey. 2002. “Generalized Method of Moments, Efficient Bootstrapping, and Improved Inference.” Journal of Business & Economic Statistics 20 (4): 507–17. https://doi.org/10.1198/073500102288618649.

Guggenberger, Patrik, and Richard J. Smith. 2005. “Generalized Empirical Likelihood Estimators and Tests Under Partial, Weak, and Strong Identification.” Econometric Theory 21 (4): 667–709. https://doi.org/10.1017/S0266466605050371.

Newey, Whitney K, and Richard J Smith. 2004. “Higher Order Properties of Gmm and Generalized Empirical Likelihood Estimators.” Econometrica 72 (1): 219–55. https://www.jstor.org/stable/pdf/3598854.pdf?casa_token=m7RohRNDzqcAAAAA:rOXGanSJq8u2gpjAg1T-K_dJxhE_PoNwtFNQGV6YHwZ_0WlTnc_9jj5qbCeGpo6ekF1WGEcBPG9wLelRbrBziziFD6inZ_W_wRuza_UZKJzJSYnwtw7vOA.