20.7 μs

This notebook will demonstrate some demand estimation methods. It focuses on the random coefficients logit model of demand for differentiated products of Berry, Levinsohn, & Pakes (1995).

9.6 μs

Simulation

Preferences

There are J products with K characteristics. Consumer i's utility from product j in market t is:

uijt=xjtβ+xjtΣνit+ξjt+ϵijt

where xjt is a vector product characteristics, νit is a vector of unobserved idiosyncratic tastes for characteristics, ξjt is an unobserved product demand shock, and ϵijt is an independent Type-I extreme value shock.

Market Shares

The market share of product j in market t is given by an integral over ν of logit probabilities:

σj(xt,ξt)=exjtβ+xjtΣν+ξjt1+k=1Jexktβ+xktΣνit+ξktdFν(ν)

8.8 μs

Profit Maximization

We will assume that firm's choose price, which will be the first element of x, (xjt[1]), while other product characteristics are fixed. Firms know the characteristics of all products and the whole vector ξt when choosing xjt[1]. Firms' profit maximization problem is:

maxppMtσj(xt(p),ξt)Cj(Mtσj(xt(p),ξt))

where xt(p) denotes xt with xjt[1] set to p and all other elements held fixed, Cj is the cost function of the firm, and Mt is the market size.

The firm's first order condition is

σj(xt(p),ξt)+pσp(xt(p),ξt)=Cj(Mtσj(xt(p),ξt))σp(xt(p),ξt)

Note that rearranging this equation gives the familiar price equals marginal cost plus markup formula:

pjCj=σjσj/pj

This is a markup since σj/pj is generally negative.

Marginal Costs

We will assume that marginal costs are log linear:

Cj(Ms)=exp(wjtγ+ωjt)

where wjt are observed firm or product characteristics and ωjt is an unobserved cost shock.

Equilibrium

In equilibrium, each firm's price much satisfy the above first order condition given the other J1 prices. Thus, equilibrium prices satisfy as system of J nonlinear equations.

15.3 μs

Simulating Data

We will use the BLPDemand.jl package for computing and simulating the equilibrium. See the docs for that package for more information.

In the simulation, exogenous product characteristics and cost shifters are U(0,1). ξ and ω are normally distributed.

To ensure that finite equilibrium prices exist, it is important that all consumers dislike higher prices, i.e. β[1]+ν[i,1]σ[1]<0 for all i. Otherwise, the firm could charge an infinite price and just sell to the price loving consumers. Therefore, the first component of ν (the random coefficient on price) is U(0,1), and the other components of ν are normally distributed.

10.9 μs

To see the source for functions in BLPDemand.jl or any other package, you can use the CodeTracking package.

3.5 μs
2.4 s
---
function simulateBLP(J, T, β::AbstractVector, σ::AbstractVector, γ::AbstractVector, S;
                     varξ=1, varω=1, randj=1.0, firmid=1:J, costf=:log)
  
  K = length(β)
  L = length(γ)
  dat = Array{MarketData,1}(undef, T)
  Ξ = Array{Vector,1}(undef,T)
  Ω = Array{Vector,1}(undef,T)
  for t in 1:T
    Jt, fid = if randj < 1.0
      inc = falses(J)
      while sum(inc)==0
        inc .= rand(J) .< randj
      end
      sum(inc), firmid[inc]
    else
      J, firmid
    end
    x = rand(K, Jt)
    ξ = randn(Jt)*varξ
    w = rand(L, Jt)
    ν = randn(K, S)
    ν[1,:] .= -rand(S) # make sure individuals' price coefficients are negative
    ω = randn(Jt)*varω

    c = costf == :log ? exp.(w'*γ + ω) : w'*γ + ω
    p = eqprices(c, β, σ, ξ, x[2:end,:], ν, firmid=fid)

    x[1,:] .= p
    s = share(x'*β+ξ, σ, x, ν)
    z = makeivblp(cat(x[2:end,:],w, dims=1), firmid=fid,
                  forceown=(length(firmid)!=length(unique(firmid))))
    dat[t] = MarketData(s, x, w, fid, z, z, ν)
    Ξ[t] = ξ
    Ω[t] = ω
  end

  return(dat=dat, ξ=Ξ, ω=Ω)
  
end
13.5 s

The hardest part of simulating the model is solving for equilibrium prices. This is done by eqprices function, which uses the approach of Morrow and Skerlos (2011).

4.8 μs
function eqprices(mc::AbstractVector,
                  β::AbstractVector, σ::AbstractVector,
                  ξ::AbstractVector,
                  x::AbstractMatrix, ν::AbstractMatrix;
                  firmid= 1:length(mc),
                  tol=sqrt(eps(eltype(mc))),
                  maxiter=10000, verbose=false)

  iter = 0
  dp = 10*tol
  focnorm = 10*tol
  p = mc*1.1
  pold = copy(p)
  samefirm = firmid.==firmid'  
  while (iter < maxiter) && ((dp > tol) || (focnorm > tol))
    s, ds, Λ, Γ = dsharedp(β, σ, p, x, ν, ξ)    
    ζ = inv(Λ)*(samefirm.*Γ)*(p - mc) - inv(Λ)*s
    focnorm = norm(Λ*(p-mc - ζ))
    pold, p = p, pold
    p .= mc .+ ζ
    dp = norm(p-pold)
    if verbose && (iter % 100 == 0)
      @show iter, p, focnorm
    end
    iter += 1    
  end
  if verbose
    @show iter, p, focnorm
  end
  return(p)  
end
29.0 ms

We use the following parameters for simulating the data.

3.6 μs
4.0 μs
2.6 s
NamedTuple{(:dat, :ξ, :ω),Tuple{Array{MarketData,1},Array{Array{T,1} where T,1},Array{Array{T,1} where T,1}}}
69.0 ns

We can see that simulateBLP returns a named tuple. The first element of the tuple is an array of MarketData which represents data that we would observe. The next two are the unobserved demand and cost shocks.

MarketData is a struct defined in BLPDemand.jl.

Let's look at some summary statistics and figures describing the simulated data.

6.0 μs
954 ms
minmaxmeanstd dev
StringRealRealFloat64Float64
1
"Number of Products"
5
5
5.0
0.0
2
"Shares"
0.0274241
0.312975
0.132071
0.0505714
3
"Prices"
3.39449
9.52479
5.4538
0.88458
4
"Other Characteristics"
0.00183849
0.998827
0.509104
0.282729
5
"Cost Shifters"
0.0096932
0.999206
0.490112
0.281802
6
"Demand Instruments"
0.00183849
3.9834
1.99203
1.27632
7
"Supply Instruments"
0.00183849
3.9834
1.99203
1.27632
2.4 s
10.9 s
8.2 s
105 ms

Estimation

It is conventional to group together all product and market specific terms entering consumers' utility as

δjt=xjtβ+ξjt

Let δt denote the vector J values of δjt in market t.

Define the share as a function of δt as

σjt(δt;θ)=eδjt+xjtΣν1+k=1Jeδkt+xktΣνitdFν(ν)

where θ denotes all the parameters to be estimated.

Let σt(δt;θ) denote the vector of J shares.

Define the inverse of this function as Δt(st;θ)=d where d satisfies σt(d;θ)=st.

The demand-side moment condition for this model can then be written as

E[(Δjt(st;θ)xjtβξjt)zjtD]=0

Using the log-linear marginal specification, we can rearrange the first order condition above to isolate the cost shock:

ωjt=log(pjt+σjt(Δt(st;θ);θ)σjtpj(Δt(st;θ);θ))wjtγ

and the supply side moments can be written as:

E[(log(pjt+σjt(Δt(st;θ);θ)σjtpj(Δt(st;θ);θ))wjtγ)ωjtzjtS]=0

To more succintly denote the moments, define:

gjt(δ,θ)=((δjtxjtβ)zjtD(log(pjt+σjt(δt;θ)σjtpj(δt;θ))wjtγ)zjtS)

Then the combined moments conditions can be written as

E[gjt(Δ(st;θ),θ)]=0

To esitmate θ we minimize a quadratic form in the empirical moments:

θ^argminθ(1JTj,tgjt(Δ(st;θ),θ))W(1JTj,tgjt(Δ(st;θ),θ))

11.8 μs
82.0 ns

Nested Fixed Point

The original BLP paper estimated θ using a "nested fixed point" (NFXP) approach. The function Δ(s;θ) has no closed form expression. Δ(s;θ) is the solution to σ(Δ(s;θ);θ)=s. This equation can only be solved numerically with some sort of iterative algorithm. BLP rearrange the equation to write δ as fixed point the point of a contraction mapping, Δ(s;θ)=F(Δ(s;θ),θ,s) and then compute Δ iteratively:

  • Choose d0 arbitrarily

  • Define di=F(di+1,θ,s) for i=1,2,...

  • Continue until didi1 is small enough

  • Use the final di as Δ(s,θ)

This simple repeated application of a contraction mapping is robust, but not neccessarily the most efficient way to compute Δ. Other methods can compute Δ more quickly, but can be less stable. All methods involve some of iteration until convergence. The function delta in BLPDemand.jl computes Δ.

The "nested" part of nested fixed point refers to the fact that minimizing the GMM objective function also involves some sort of iterative minimization algorithm. Each iteration of the minimization algorithm, we redo all the iterations needed to compute Δ(s,θ).

2.6 ms

Automatic Differentiation

We need to compute deriviatives of many of the functions above. σp is part of the moment conditions. Minimization algorithms can be much more efficient if we provide the gradient and/or hessian of the objective function. Finally, the asymptotic variance of θ involves the Jacobian of the moment conditions (i.e. Dθgjt(Δ(st;θ),θ)). While we could calculate these derivatives by hand and then write code to compute them, there is a less laborious and error-prone way. Automatic differentiation refers to taking code that computes some function, f(x) and algorithmically applying the chain rule to compute dfdx. One of Julia's nicest features is how well it supports automatic differentiation. The two leading Julia packages for automatic differentiation are ForwardDiff.jl and Zygote.jl. ForwardDiff.jl is usually a good choice for function from RnRk with n not too much larger than k. Zygote.jl is somewhat more restrictive in what code it is compatible with, but is more efficient if n is much much larger than k. You can read more in the Quantecon notes on automatic differentiation.

2.9 μs
Iter     Function value   Gradient norm 
     0     1.158102e+04     8.015509e+05
 * time: 0.009341955184936523
     1     1.924357e+02     1.126686e+05
 * time: 1.9586098194122314
     2     7.571640e+00     3.806780e+02
 * time: 1.974524974822998
     3     7.569459e+00     1.405256e+01
 * time: 1.998633861541748
     4     4.394585e+00     1.448938e+03
 * time: 2.0837879180908203
     5     4.364341e+00     3.530266e+01
 * time: 2.0992259979248047
     6     4.364101e+00     3.047149e+00
 * time: 2.1224489212036133
     7     4.363403e+00     2.152880e+02
 * time: 2.1677727699279785
     8     4.169330e+00     6.884843e+02
 * time: 2.213343858718872
     9     4.108017e+00     1.350131e+03
 * time: 2.2287039756774902
    10     4.038903e+00     4.962200e+02
 * time: 2.2513558864593506
    11     4.032905e+00     5.101109e+01
 * time: 2.2665657997131348
    12     4.031522e+00     3.735515e+01
 * time: 2.2816929817199707
    13     4.031358e+00     4.919446e+00
 * time: 2.3042078018188477
    14     4.031356e+00     6.030557e+00
 * time: 2.3707997798919678
    15     4.031171e+00     7.907360e+00
 * time: 2.4144129753112793
    16     3.944447e+00     8.525036e+02
 * time: 2.458922863006592
    17     3.911783e+00     1.384805e+03
 * time: 2.489116907119751
    18     3.855410e+00     2.188737e+03
 * time: 2.504194974899292
    19     3.597449e+00     4.505988e+03
 * time: 2.5271029472351074
    20     3.345880e+00     3.317219e+03
 * time: 2.588615894317627
    21     3.304821e+00     2.178381e+03
 * time: 2.603888988494873
    22     3.235112e+00     6.233731e+00
 * time: 3.4163057804107666
    23     3.235005e+00     8.420931e+01
 * time: 3.4542877674102783
    24     1.595482e+00     1.829725e+03
 * time: 3.515472888946533
    25     1.487061e+00     5.183353e+02
 * time: 3.5388197898864746
    26     1.481471e+00     5.661871e+00
 * time: 3.554438829421997
    27     1.481462e+00     5.655291e+00
 * time: 3.5779268741607666
    28     6.369042e-01     2.907135e+02
 * time: 3.639993906021118
    29     4.882034e-01     9.423588e+02
 * time: 3.6556828022003174
    30     4.722599e-01     2.181082e+02
 * time: 3.6712148189544678
    31     4.692708e-01     1.569599e+01
 * time: 3.68691086769104
    32     4.692615e-01     1.957743e+00
 * time: 3.7025837898254395
    33     4.690967e-01     8.172367e+01
 * time: 3.795501947402954
    34     4.170150e-01     4.707579e+02
 * time: 3.842383861541748
    35     3.874091e-01     8.745371e+02
 * time: 3.858219861984253
    36     3.660467e-01     1.680291e+02
 * time: 3.8745369911193848
    37     3.565735e-01     5.990861e+02
 * time: 3.8904759883880615
    38     3.393281e-01     6.071597e+02
 * time: 3.914761781692505
    39     3.361840e-01     5.630346e+02
 * time: 3.930722951889038
    40     3.315947e-01     3.883355e+01
 * time: 3.9465367794036865
    41     3.270270e-01     1.243660e+02
 * time: 3.9702279567718506
    42     3.260627e-01     7.921322e+01
 * time: 3.993623971939087
    43     3.259680e-01     9.265922e-01
 * time: 4.009253978729248
    44     3.259491e-01     6.831564e+00
 * time: 4.032655954360962
    45     3.257824e-01     1.187813e+01
 * time: 4.071793794631958
    46     3.239176e-01     5.372098e+01
 * time: 4.103302001953125
    47     3.233085e-01     9.303561e+01
 * time: 4.134528875350952
    48     3.218731e-01     1.392104e+02
 * time: 4.150173902511597
    49     3.202316e-01     2.240442e+02
 * time: 4.221021890640259
    50     3.199122e-01     9.829724e+01
 * time: 4.236719846725464
    51     3.189274e-01     1.641617e+02
 * time: 4.259667873382568
    52     3.178248e-01     2.557791e+02
 * time: 4.274986982345581
    53     3.153498e-01     1.790588e+02
 * time: 4.298233985900879
    54     3.137514e-01     7.463333e+01
 * time: 4.321682929992676
    55     3.132983e-01     1.441394e+02
 * time: 4.337328910827637
    56     3.117996e-01     1.723170e+02
 * time: 4.353243827819824
    57     3.100110e-01     1.656003e+02
 * time: 4.376643896102905
    58     3.097470e-01     2.063646e+02
 * time: 4.392545938491821
    59     3.087138e-01     3.113353e+01
 * time: 4.408326864242554
    60     3.077516e-01     5.728675e+01
 * time: 4.423904895782471
    61     3.074488e-01     8.128606e+01
 * time: 4.439549922943115
    62     3.065080e-01     1.423380e+02
 * time: 4.455223798751831
    63     3.055215e-01     2.248325e+02
 * time: 4.470853805541992
    64     3.051409e-01     9.599818e+01
 * time: 4.486663818359375
    65     3.042253e-01     1.544814e+02
 * time: 4.509979963302612
    66     3.031998e-01     2.342677e+02
 * time: 4.5255959033966064
    67     3.011996e-01     6.464122e+01
 * time: 4.548874855041504
    68     3.007170e-01     1.480817e+02
 * time: 4.564445972442627
    69     2.992811e-01     2.081977e+02
 * time: 4.579939842224121
    70     2.986292e-01     1.651863e+01
 * time: 4.595477819442749
    71     2.975897e-01     1.756935e+02
 * time: 4.654794931411743
    72     2.966141e-01     1.860799e+02
 * time: 4.678025960922241
    73     2.961896e-01     2.527389e+01
 * time: 4.693364858627319
    74     2.953896e-01     1.213568e+02
 * time: 4.716164827346802
    75     2.949674e-01     1.823298e+02
 * time: 4.7317869663238525
    76     2.942106e-01     5.112320e+01
 * time: 4.755225896835327
    77     2.938091e-01     3.383827e+01
 * time: 4.770978927612305
    78     2.933637e-01     1.121973e+02
 * time: 4.78657078742981
    79     2.930515e-01     5.772480e+01
 * time: 4.802248001098633
    80     2.927370e-01     1.379276e+02
 * time: 4.825741767883301
    81     2.919313e-01     1.604749e+02
 * time: 4.849093914031982
    82     2.909408e-01     8.290502e+01
 * time: 4.872331857681274
    83     2.906202e-01     2.313698e+01
 * time: 4.8879358768463135
    84     2.904082e-01     2.540116e+01
 * time: 4.903480768203735
    85     2.902697e-01     7.055657e+01
 * time: 4.918900966644287
    86     2.902208e-01     2.897104e+01
 * time: 4.942051887512207
    87     2.899351e-01     7.757287e+01
 * time: 4.972864866256714
    88     2.891741e-01     1.095429e+02
 * time: 4.99612283706665
    89     2.873303e-01     1.093396e+02
 * time: 5.019243001937866
    90     2.871501e-01     8.660222e+01
 * time: 5.561618804931641
    91     2.832469e-01     1.004631e+02
 * time: 5.633150815963745
    92     2.828676e-01     1.803296e+02
 * time: 5.655808925628662
    93     2.801536e-01     9.969537e+01
 * time: 5.678155899047852
    94     2.793153e-01     1.709469e+02
 * time: 5.701012849807739
    95     2.774047e-01     3.129373e+01
 * time: 5.724222898483276
    96     2.772003e-01     3.137401e+01
 * time: 5.739959001541138
    97     2.766189e-01     4.000743e+01
 * time: 5.755527973175049
    98     2.762675e-01     8.963176e+01
 * time: 5.77103590965271
    99     2.757565e-01     1.249678e+02
 * time: 5.794271945953369
   100     2.749841e-01     4.656272e+01
 * time: 5.8254148960113525
   101     2.747144e-01     1.365616e+01
 * time: 5.856243848800659
   102     2.744100e-01     5.668909e+01
 * time: 5.871713876724243
   103     2.738863e-01     1.261332e+02
 * time: 5.8871989250183105
   104     2.735526e-01     8.584698e+01
 * time: 5.902673959732056
   105     2.733853e-01     1.487574e+02
 * time: 5.928653955459595
   106     2.726635e-01     1.475121e+02
 * time: 5.951898813247681
   107     2.726540e-01     1.478423e+02
 * time: 5.967432975769043
   108     2.718231e-01     1.674944e+02
 * time: 5.9983909130096436
   109     2.713714e-01     5.827077e+01
 * time: 6.082246780395508
   110     2.704240e-01     8.869285e+01
 * time: 6.097794771194458
   111     2.696814e-01     5.874190e+01
 * time: 6.1130828857421875
   112     2.695847e-01     1.109061e+02
 * time: 6.13582181930542
   113     2.692614e-01     1.373914e+02
 * time: 6.158984899520874
   114     2.685665e-01     1.016962e+02
 * time: 6.1823248863220215
   115     2.674801e-01     3.186458e+01
 * time: 6.205618858337402
   116     2.672910e-01     3.939303e+01
 * time: 6.236764907836914
   117     2.671380e-01     8.641395e+01
 * time: 6.2525389194488525
   118     2.669469e-01     1.401641e+02
 * time: 6.268186807632446
   119     2.659307e-01     9.682655e+01
 * time: 6.299004793167114
   120     2.657933e-01     2.445027e+00
 * time: 6.314568996429443
   121     2.653830e-01     8.146154e+01
 * time: 6.337719917297363
   122     2.652376e-01     1.213177e+02
 * time: 6.353096961975098
   123     2.643754e-01     6.505866e+00
 * time: 6.376199960708618
   124     2.643416e-01     3.507493e+01
 * time: 6.391617774963379
   125     2.639139e-01     1.085753e+02
 * time: 6.414681911468506
   126     2.634703e-01     5.002405e+01
 * time: 6.437814950942993
   127     2.630736e-01     4.175424e+01
 * time: 6.512578964233398
   128     2.628705e-01     6.504037e+01
 * time: 6.528037786483765
   129     2.625408e-01     6.599695e+00
 * time: 6.5431458950042725
   130     2.624482e-01     4.485678e+01
 * time: 6.573255777359009
   131     2.623290e-01     6.795676e+01
 * time: 6.588785886764526
   132     2.621349e-01     9.947932e+01
 * time: 6.60447096824646
   133     2.618118e-01     5.712859e+01
 * time: 6.627702951431274
   134     2.615590e-01     7.188437e+01
 * time: 6.6508469581604
   135     2.609004e-01     1.708055e+01
 * time: 6.674138784408569
   136     2.608418e-01     9.544046e+01
 * time: 6.6975319385528564
   137     2.604568e-01     6.821765e+01
 * time: 6.720704793930054
   138     2.602313e-01     3.181413e+00
 * time: 6.751471996307373
   139     2.601784e-01     2.324914e+01
 * time: 6.766927003860474
   140     2.600523e-01     7.148502e+01
 * time: 6.782321929931641
   141     2.599336e-01     9.816165e+01
 * time: 6.805563926696777
   142     2.598222e-01     2.320962e+01
 * time: 6.828610897064209
   143     2.595957e-01     5.797722e+01
 * time: 6.851655960083008
   144     2.594819e-01     7.526668e+01
 * time: 6.867009878158569
   145     2.591757e-01     1.099213e+02
 * time: 6.882444858551025
   146     2.589763e-01     1.645011e+00
 * time: 6.9502928256988525
   147     2.586469e-01     8.395346e+01
 * time: 6.965851783752441
   148     2.584391e-01     1.270971e+02
 * time: 6.981046915054321
   149     2.582333e-01     2.457122e+01
 * time: 6.996275901794434
   150     2.579269e-01     6.465468e+01
 * time: 7.019422769546509
   151     2.577715e-01     8.510171e+01
 * time: 7.035075902938843
   152     2.573741e-01     9.996391e+01
 * time: 7.058332920074463
   153     2.571380e-01     2.648307e+01
 * time: 7.073880910873413
   154     2.565805e-01     4.614501e+01
 * time: 7.089401960372925
   155     2.564930e-01     6.083702e+01
 * time: 7.105105876922607
   156     2.560428e-01     1.055427e+02
 * time: 7.121107816696167
   157     2.557200e-01     8.363318e+00
 * time: 7.1442718505859375
   158     2.555465e-01     4.675574e+01
 * time: 7.15979790687561
   159     2.552150e-01     8.612866e+01
 * time: 7.182952880859375
   160     2.551461e-01     1.961896e+01
 * time: 7.198385000228882
   161     2.550703e-01     2.241686e+01
 * time: 7.2214179039001465
   162     2.550081e-01     4.096941e+01
 * time: 7.252126932144165
   163     2.549200e-01     5.745625e+01
 * time: 7.267636775970459
   164     2.547861e-01     7.265191e+01
 * time: 7.290639877319336
   165     2.547138e-01     3.328488e+00
 * time: 7.306104898452759
   166     2.545572e-01     6.223203e+01
 * time: 7.365140914916992
   167     2.544506e-01     6.534258e+01
 * time: 7.388667821884155
   168     2.543978e-01     5.660676e+00
 * time: 7.403896808624268
   169     2.542638e-01     5.726138e+01
 * time: 7.4265899658203125
   170     2.541786e-01     7.346500e+01
 * time: 7.449765920639038
   171     2.540909e-01     2.711530e-01
 * time: 7.465380907058716
   172     2.539917e-01     5.472818e+01
 * time: 7.488641977310181
   173     2.539074e-01     6.552397e+01
 * time: 7.51191782951355
   174     2.538474e-01     6.253597e+00
 * time: 7.527484893798828
   175     2.537567e-01     4.882992e+01
 * time: 7.55114483833313
   176     2.536947e-01     7.286973e+01
 * time: 7.574304819107056
   177     2.536031e-01     4.886872e+00
 * time: 7.589815855026245
   178     2.535288e-01     4.883026e+01
 * time: 7.613020896911621
   179     2.534580e-01     5.921381e+01
 * time: 7.636051893234253
   180     2.534142e-01     7.817139e+00
 * time: 7.651554822921753
   181     2.533358e-01     3.895045e+01
 * time: 7.674715995788574
   182     2.532965e-01     5.555704e+01
 * time: 7.690222978591919
   183     2.532070e-01     1.414783e+01
 * time: 7.713343858718872
   184     2.531741e-01     3.861907e+01
 * time: 7.728725910186768
   185     2.530980e-01     5.127703e+01
 * time: 7.794784784317017
   186     2.530707e-01     8.024838e+00
 * time: 7.81061577796936
   187     2.530057e-01     2.296540e+01
 * time: 7.833395957946777
   188     2.529880e-01     2.947937e+01
 * time: 7.848516941070557
   189     2.529383e-01     4.144680e+01
 * time: 7.863749980926514
   190     2.528743e-01     1.430139e+00
 * time: 7.887034893035889
   191     2.528648e-01     7.833023e+00
 * time: 7.9027509689331055
   192     2.528002e-01     3.526563e+01
 * time: 7.918940782546997
   193     2.527776e-01     1.263031e+01
 * time: 7.934583902359009
   194     2.527621e-01     3.258094e+01
 * time: 7.950143814086914
   195     2.527274e-01     3.607939e+01
 * time: 7.973707914352417
   196     2.526942e-01     3.277102e+01
 * time: 7.997054815292358
   197     2.526658e-01     8.506579e+00
 * time: 8.035525798797607
   198     2.526525e-01     2.053922e+00
 * time: 8.05101990699768
   199     2.526220e-01     2.454851e+01
 * time: 8.066417932510376
   200     2.526176e-01     1.306052e+01
 * time: 8.08192777633667
   201     2.526100e-01     2.133432e+01
 * time: 8.105268955230713
   202     2.525970e-01     2.660964e+01
 * time: 8.128410816192627
   203     2.525681e-01     1.419395e+01
 * time: 8.15154480934143
   204     2.525510e-01     6.958683e+00
 * time: 8.174622774124146
   205     2.525473e-01     1.229533e+01
 * time: 8.234369993209839
   206     2.525381e-01     7.349240e+00
 * time: 8.249811887741089
   207     2.525330e-01     1.134161e+01
 * time: 8.272371768951416
   208     2.525287e-01     7.165329e+00
 * time: 8.28751277923584
   209     2.525275e-01     3.964161e+00
 * time: 8.303059816360474
   210     2.525269e-01     1.833966e+00
 * time: 8.318589925765991
   211     2.525264e-01     8.726435e-01
 * time: 8.33425498008728
   212     2.525262e-01     2.815254e+00
 * time: 8.349865913391113
   213     2.525260e-01     1.866213e+00
 * time: 8.38074278831482
   214     2.525257e-01     2.020357e+00
 * time: 8.404403924942017
   215     2.525253e-01     2.527619e-01
 * time: 8.435282945632935
   216     2.525238e-01     1.759484e+00
 * time: 8.45849084854126
   217     2.525182e-01     3.484391e+00
 * time: 8.481624841690063
   218     2.525171e-01     2.505418e-01
 * time: 8.496988773345947
   219     2.525129e-01     3.349608e+00
 * time: 8.512557983398438
   220     2.524942e-01     1.454801e+01
 * time: 8.5433349609375
   221     2.524651e-01     3.527870e+00
 * time: 8.558886766433716
   222     2.524624e-01     4.425320e+00
 * time: 8.574318885803223
   223     2.524616e-01     4.332724e+00
 * time: 8.597463846206665
   224     2.524607e-01     1.594704e+01
 * time: 8.6648588180542
   225     2.524555e-01     2.584182e+00
 * time: 8.68783688545227
   226     2.524503e-01     3.406157e+00
 * time: 8.711942911148071
   227     2.524496e-01     4.542937e+00
 * time: 8.727239847183228
   228     2.524469e-01     7.607586e+00
 * time: 8.742854833602905
   229     2.524437e-01     1.230285e+01
 * time: 8.758599996566772
   230     2.524366e-01     1.306009e+01
 * time: 8.781839847564697
   231     2.524324e-01     8.123500e+00
 * time: 8.797380924224854
   232     2.524061e-01     4.126091e+00
 * time: 8.820743799209595
   233     2.523856e-01     1.850825e+01
 * time: 8.851924896240234
   234     2.523527e-01     2.140838e+01
 * time: 8.882749795913696
   235     2.522392e-01     3.420429e+01
 * time: 8.898308992385864
   236     2.521573e-01     4.593372e+01
 * time: 8.913831949234009
   237     2.520087e-01     6.422411e+00
 * time: 8.944560766220093
   238     2.519650e-01     3.861112e+01
 * time: 8.9753999710083
   239     2.517994e-01     7.376328e+01
 * time: 8.998568773269653
   240     2.515322e-01     5.302435e+00
 * time: 9.021621942520142
   241     2.513456e-01     6.196495e+01
 * time: 9.037197828292847
   242     2.510036e-01     5.381627e+01
 * time: 9.104031801223755
   243     2.506111e-01     1.177536e+02
 * time: 9.119385004043579
   244     2.502757e-01     1.337410e+01
 * time: 9.14196491241455
   245     2.502440e-01     1.638485e+01
 * time: 9.157287836074829
   246     2.502344e-01     1.799992e+00
 * time: 9.172859907150269
   247     2.502319e-01     9.173540e+00
 * time: 9.196141958236694
   248     2.502276e-01     1.419444e+01
 * time: 9.219385862350464
   249     2.502211e-01     7.141728e+00
 * time: 9.24256682395935
   250     2.502169e-01     1.463395e+01
 * time: 9.266063928604126
   251     2.502117e-01     9.830643e-01
 * time: 9.281617879867554
   252     2.502081e-01     1.087000e+01
 * time: 9.298495769500732
   253     2.502042e-01     1.121336e+01
 * time: 9.321640968322754
   254     2.502041e-01     1.205783e+01
 * time: 9.344727993011475
   255     2.502017e-01     1.217670e-01
 * time: 9.360190868377686
   256     2.501985e-01     7.672462e+00
 * time: 9.3833909034729
   257     2.501967e-01     7.559002e+00
 * time: 9.406499862670898
   258     2.501966e-01     7.072219e+00
 * time: 9.421987771987915
   259     2.501957e-01     9.319400e-01
 * time: 9.437367916107178
   260     2.501943e-01     3.334921e+00
 * time: 9.452796936035156
   261     2.501938e-01     5.334688e+00
 * time: 9.51064395904541
   262     2.501932e-01     2.740513e-01
 * time: 9.534624814987183
   263     2.501928e-01     3.099231e+00
 * time: 9.557366847991943
   264     2.501926e-01     7.963243e-01
 * time: 9.5726318359375
   265     2.501925e-01     1.308775e+00
 * time: 9.595669984817505
   266     2.501925e-01     5.048317e-02
 * time: 9.619057893753052
   267     2.501924e-01     6.794882e-02
 * time: 9.642277956008911
   268     2.501924e-01     7.474938e-01
 * time: 9.665480852127075
   269     2.501924e-01     1.442685e-01
 * time: 9.688922882080078
   270     2.501924e-01     3.981050e-01
 * time: 9.720008850097656
   271     2.501922e-01     1.463974e+00
 * time: 9.750826835632324
   272     2.501921e-01     5.028560e-02
 * time: 9.77402377128601
   273     2.501919e-01     2.087718e+00
 * time: 9.804776906967163
   274     2.501894e-01     1.674360e+00
 * time: 9.83558988571167
   275     2.501829e-01     9.066367e+00
 * time: 9.851081848144531
   276     2.501674e-01     1.101074e+01
 * time: 9.866505861282349
   277     2.501642e-01     1.411714e+00
 * time: 9.881941795349121
   278     2.501636e-01     7.196715e+00
 * time: 9.955275774002075
   279     2.501610e-01     8.254154e-01
 * time: 9.979021787643433
   280     2.501608e-01     2.190481e+00
 * time: 9.994337797164917
   281     2.501603e-01     4.962324e+00
 * time: 10.009703874588013
   282     2.501598e-01     5.182638e-01
 * time: 10.032745838165283
   283     2.501595e-01     2.884059e+00
 * time: 10.056116819381714
   284     2.501594e-01     1.067690e+00
 * time: 10.071696996688843
   285     2.501594e-01     1.468252e+00
 * time: 10.094895839691162
   286     2.501592e-01     9.911741e-01
 * time: 10.118231773376465
   287     2.501591e-01     9.684999e-01
 * time: 10.141735792160034
   288     2.501590e-01     9.798744e-01
 * time: 10.157241821289062
   289     2.501590e-01     2.494077e-02
 * time: 10.172800779342651
   290     2.501590e-01     7.844264e-02
 * time: 10.195939779281616
   291     2.501590e-01     5.567220e-02
 * time: 10.21941590309143
   292     2.501590e-01     1.027418e-02
 * time: 10.25025486946106
   293     2.501590e-01     2.178961e-02
 * time: 10.273340940475464
   294     2.501590e-01     1.242358e-01
 * time: 10.304064989089966
   295     2.501585e-01     3.346181e+00
 * time: 10.39514684677124
   296     2.501416e-01     2.509670e+00
 * time: 10.425709962844849
   297     2.484456e-01     4.580117e+02
 * time: 10.48698091506958
   298     2.483429e-01     4.915112e+02
 * time: 10.526407957077026
   299     2.474250e-01     2.176344e+02
 * time: 10.541956901550293
   300     2.437584e-01     1.470661e+02
 * time: 10.56555700302124
   301     2.426810e-01     1.988756e+02
 * time: 10.596614837646484
   302     2.416667e-01     1.075403e+02
 * time: 10.612271785736084
   303     2.413484e-01     1.267112e+01
 * time: 10.627819776535034
   304     2.413441e-01     3.633878e+00
 * time: 10.650941848754883
   305     2.413380e-01     4.585203e+00
 * time: 10.674152851104736
   306     2.413365e-01     6.447549e-01
 * time: 10.689729928970337
   307     2.413365e-01     1.932337e-01
 * time: 10.712978839874268
   308     2.413364e-01     2.893140e-01
 * time: 10.736071825027466
   309     2.413364e-01     2.362366e-02
 * time: 10.759217977523804
   310     2.413364e-01     1.723668e-01
 * time: 10.825915813446045
   311     2.413364e-01     4.426027e-01
 * time: 10.86402678489685
   312     2.413364e-01     4.186628e-01
 * time: 10.886993885040283
   313     2.413364e-01     3.620181e-02
 * time: 10.902659893035889
   314     2.413364e-01     5.273786e-02
 * time: 10.926074981689453
   315     2.413364e-01     1.803126e-03
 * time: 10.94936990737915
   316     2.413364e-01     4.714602e-04
 * time: 10.972757816314697
   317     2.413364e-01     9.599359e-05
 * time: 10.996540784835815
   318     2.413364e-01     8.719291e-05
 * time: 11.019848823547363
   319     2.413364e-01     2.689496e-04
 * time: 11.050747871398926
   320     2.413364e-01     3.159250e-04
 * time: 11.073993921279907
   321     2.413364e-01     4.748693e-05
 * time: 11.089479923248291
   322     2.413364e-01     1.084864e-03
 * time: 11.112883806228638
   323     2.413364e-01     1.286809e-05
 * time: 11.128447771072388
   324     2.413364e-01     2.470713e-05
 * time: 11.151625871658325
   325     2.413364e-01     1.910783e-05
 * time: 11.167285919189453
   326     2.413364e-01     7.352720e-06
 * time: 11.182723999023438
   327     2.413364e-01     4.334510e-05
 * time: 11.249237775802612
   328     2.413364e-01     4.994040e-05
 * time: 11.272461891174316
   329     2.413364e-01     7.901246e-06
 * time: 11.295106887817383
   330     2.413364e-01     2.450247e-05
 * time: 11.31037187576294
   331     2.413364e-01     2.791747e-05
 * time: 11.326072931289673
   332     2.413364e-01     1.994570e-05
 * time: 11.364902973175049
   333     2.413364e-01     7.422739e-05
 * time: 11.388206958770752
   334     2.413364e-01     2.016650e-05
 * time: 11.403850793838501
   335     2.413364e-01     3.898515e-05
 * time: 11.465949773788452
   336     2.413364e-01     5.813111e-05
 * time: 11.496924877166748
   337     2.413364e-01     5.737829e-05
 * time: 11.512472867965698
   338     2.413364e-01     2.258345e-05
 * time: 11.527966976165771
   339     2.413364e-01     6.517463e-05
 * time: 11.558870792388916
   340     2.413364e-01     1.174185e-05
 * time: 11.582137823104858
   341     2.413364e-01     8.400931e-06
 * time: 11.60533094406128
   342     2.413364e-01     7.217852e-06
 * time: 11.62084698677063
   343     2.413364e-01     1.826584e-05
 * time: 12.1454598903656
   344     2.413364e-01     1.444287e-05
 * time: 12.168283939361572
   345     2.413364e-01     1.275383e-05
 * time: 12.183957815170288
   346     2.413364e-01     1.463366e-03
 * time: 12.67297077178955
   347     2.413364e-01     6.977897e-04
 * time: 12.696410894393921
   348     2.413364e-01     2.884721e-05
 * time: 12.720035791397095
   349     2.413364e-01     5.512581e-04
 * time: 12.758677959442139
   350     2.413364e-01     8.555417e-07
 * time: 12.781937837600708
   351     2.413364e-01     7.524316e-05
 * time: 12.828342914581299
   352     2.413364e-01     5.337661e-05
 * time: 12.843871831893921
   353     2.413364e-01     2.534011e-05
 * time: 12.905558824539185
   354     2.413364e-01     4.205067e-06
 * time: 12.97214388847351
   355     2.413364e-01     3.500206e-07
 * time: 12.987884998321533
13.9 s

MPEC

MPEC is an alternative approach to computing estimates promoted by Judd and Su and applied to this model by Dube, Fox, and Su (2012). MPEC stands for "Mathematical Programming with Equilibrium Constraints." It is based on the observation that the estimation problem can be expressed as a constrained minimization problem

(θ^,δ^)=argminθ,δ(1JTj,tgjt(δt,θ))W(1JTj,tgjt(δt,θ)) s.t. st=σt(δt;θ)t

Note that this problem no longer requires compute Δ(s,θ), so there is no more nesting of iterative computations. However, there are some tradeoffs. We now have a constrained minimization problem, which are slightly more complicated to solve than unconstrained problems. More importantly, the minimization problem is now over a much higher dimensional space (θ and δ instead of just θ). Nonetheless, the MPEC formulation of the problem can sometimes lead to more efficient and/or more robust computation. The relative performance of MPEC and NFXP is very dependent on the implementation details (the minization algorithms, the way derivatives are calculated, and whether any sparsity patterns are taken advantage of).

6.2 μs
This is Ipopt version 3.13.2, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).

Number of nonzeros in equality constraint Jacobian...:   155509
Number of nonzeros in inequality constraint Jacobian.:        0
Number of nonzeros in Lagrangian Hessian.............:    59009

Total number of variables............................:    17516
                     variables with only lower bounds:     5003
                variables with lower and upper bounds:        0
                     variables with only upper bounds:        0
Total number of equality constraints.................:    17509
Total number of inequality constraints...............:        0
        inequality constraints with only lower bounds:        0
   inequality constraints with lower and upper bounds:        0
        inequality constraints with only upper bounds:        0

iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
   0  0.0000000e+00 5.38e+00 2.59e-03  -1.0 0.00e+00    -  0.00e+00 0.00e+00   0
   1  2.4902150e+01 9.87e+00 1.94e+02  -1.0 3.70e+00   0.0 5.96e-01 1.00e+00h  1
   2  7.0778252e+00 3.72e+00 2.65e+02  -1.0 2.65e+00    -  8.01e-01 1.00e+00f  1
   3  5.7238522e+00 1.80e+00 8.38e+01  -1.0 2.45e+00    -  1.00e+00 6.89e-01h  1
   4  4.9730437e+00 1.65e+00 8.03e+01  -1.0 4.78e+00    -  1.00e+00 8.36e-02h  1
   5  4.4563632e+00 2.71e-01 3.80e+01  -1.0 1.01e+00    -  1.00e+00 1.00e+00f  1
   6  4.1119628e+00 4.64e-02 8.11e+01  -1.0 3.97e-01    -  4.59e-01 1.00e+00h  1
   7  4.0797294e+00 2.28e-03 5.14e-02  -1.0 1.03e-01  -0.5 1.00e+00 1.00e+00h  1
   8  4.0300757e+00 7.91e-04 6.33e-01  -2.5 5.11e-02  -1.0 8.51e-01 1.00e+00h  1
   9  3.3007868e+00 2.49e-01 2.86e-01  -2.5 8.45e-01    -  7.22e-01 1.00e+00f  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  10  2.0409823e+00 8.57e-01 7.93e-01  -2.5 2.12e+01    -  9.03e-02 8.05e-02f  1
  11  1.3077361e+00 8.14e-01 2.40e+00  -2.5 5.54e+00    -  6.76e-01 3.03e-01h  1
  12  2.7037103e-01 4.10e+00 7.95e-01  -2.5 6.87e+00    -  7.13e-01 8.33e-01h  1
  13  1.0600200e+00 4.68e+00 5.89e+01  -2.5 1.33e+01    -  8.28e-02 1.00e+00h  1
  14  7.7575740e-01 2.59e-01 1.47e+00  -2.5 7.21e-01  -1.4 5.19e-01 1.00e+00h  1
  15  6.9478453e-01 2.72e-02 6.32e-02  -2.5 9.22e-01  -1.9 1.00e+00 1.00e+00h  1
  16  5.2893994e-01 1.23e-01 2.86e-02  -2.5 2.44e+00  -2.4 1.00e+00 1.00e+00h  1
  17  3.3163234e-01 3.80e-01 8.92e-02  -2.5 4.07e+00  -2.9 1.00e+00 1.00e+00h  1
  18  2.7053490e-01 3.12e-01 1.44e-01  -2.5 3.07e+00  -3.3 4.63e-01 1.00e+00h  1
  19  2.5994722e-01 5.40e-02 1.27e-02  -2.5 1.80e+00  -2.9 1.00e+00 1.00e+00h  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  20  2.3970933e-01 4.86e-01 1.44e-01  -2.5 8.91e+00  -3.4 6.86e-01 1.00e+00H  1
  21  2.3101273e-01 5.22e-02 1.70e-01  -2.5 1.29e+02  -3.9 1.93e-01 4.11e-02h  3
  22  2.1548498e-01 8.06e-01 5.38e-01  -2.5 2.21e+01  -3.4 1.00e+00 1.00e+00H  1
  23  2.8727140e-01 6.87e-01 7.96e-01  -2.5 9.06e+01  -3.9 1.88e-01 1.13e-01h  1
  24  1.3111627e-01 1.98e+01 4.67e+00  -2.5 4.99e+01    -  1.00e+00 9.95e-01h  1
  25  3.8830274e-01 1.38e+01 3.16e+01  -2.5 5.07e+01    -  6.15e-02 3.22e-01h  2
  26  6.5921831e-01 1.06e+01 1.22e+02  -2.5 4.72e+01  -4.4 1.53e-01 1.00e+00H  1
  27  1.3625812e+00 3.47e+00 8.93e+01  -2.5 1.35e+01    -  5.42e-01 1.00e+00h  1
  28  1.6239477e+00 1.66e-01 7.13e+00  -2.5 1.56e+00  -1.3 1.00e+00 1.00e+00h  1
  29  1.5611908e+00 2.64e-02 2.90e+00  -2.5 2.16e+00  -1.7 1.00e+00 1.00e+00h  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  30  1.0878528e+00 1.78e+00 2.26e+01  -2.5 2.64e+01    -  1.00e+00 5.00e-01f  2
  31  5.9471167e-01 5.47e+00 8.52e+00  -2.5 4.95e+01    -  1.00e+00 4.93e-01h  1
  32  3.9199789e-01 1.99e+00 4.89e+00  -2.5 2.44e+01    -  1.00e+00 1.00e+00h  1
  33  1.9504844e-01 4.58e+00 3.63e+00  -2.5 6.01e+01    -  8.57e-01 4.76e-01h  1
  34  9.9082472e-02 6.70e+00 3.99e+00  -2.5 4.70e+01    -  1.00e+00 1.00e+00h  1
  35  1.1646569e-01 4.58e-02 1.27e+01  -2.5 2.43e+00    -  8.25e-02 1.00e+00h  1
  36  1.1728633e-01 5.55e-01 2.58e-01  -2.5 1.48e+01    -  1.00e+00 1.00e+00h  1
  37  1.3111244e-01 1.85e-01 1.29e-01  -2.5 7.65e+00    -  1.00e+00 1.00e+00h  1
  38  1.3322783e-01 7.96e-04 1.03e-03  -2.5 3.36e-01    -  1.00e+00 1.00e+00h  1
  39  6.3105600e-02 8.04e-01 1.22e+00  -3.8 8.68e+00    -  8.45e-01 1.00e+00h  1
iter    objective    inf_pr   inf_du lg(mu)  ||d||  lg(rg) alpha_du alpha_pr  ls
  40  5.8024469e-02 3.67e-01 2.18e-01  -3.8 1.08e+01    -  1.00e+00 1.00e+00h  1
  41  5.8878589e-02 6.15e-03 7.51e-03  -3.8 1.00e+00    -  1.00e+00 1.00e+00h  1
  42  5.8895170e-02 3.77e-07 4.33e-07  -3.8 1.68e-02    -  1.00e+00 1.00e+00h  1
  43  5.7983183e-02 2.97e-02 3.38e-02  -5.7 3.00e+00    -  9.77e-01 1.00e+00h  1
  44  5.8172630e-02 6.32e-04 1.01e-04  -5.7 4.02e-01    -  1.00e+00 1.00e+00h  1
  45  5.8173082e-02 1.17e-08 4.42e-09  -5.7 1.22e-03    -  1.00e+00 1.00e+00h  1
  46  5.8172911e-02 7.84e-06 6.78e-06  -8.6 4.80e-02    -  1.00e+00 1.00e+00h  1
  47  5.8172953e-02 2.90e-11 4.83e-12  -8.6 8.44e-05    -  1.00e+00 1.00e+00h  1

Number of Iterations....: 47

                                   (scaled)                 (unscaled)
Objective...............:   5.8172953363477609e-02    5.8172953363477609e-02
Dual infeasibility......:   4.8316906031686813e-12    4.8316906031686813e-12
Constraint violation....:   2.8961721909581684e-11    2.8961721909581684e-11
Complementarity.........:   2.5059049512274208e-09    2.5059049512274208e-09
Overall NLP error.......:   2.5059049512274208e-09    2.5059049512274208e-09


Number of objective function evaluations             = 62
Number of objective gradient evaluations             = 48
Number of equality constraint evaluations            = 62
Number of inequality constraint evaluations          = 0
Number of equality constraint Jacobian evaluations   = 48
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations             = 47
Total CPU secs in IPOPT (w/o function evaluations)   =     20.889
Total CPU secs in NLP function evaluations           =      4.698

EXIT: Optimal Solution Found.
6.8 s
83.0 ns