Package {sfa}


Version: 1.1.5
Date: 2026-08-23
Title: Stochastic Frontier Analysis
Type: Package
Maintainer: David Bernstein <davebernstein1@gmail.com>
Description: Provides a user-friendly framework for estimating a wide variety of cross-sectional and panel stochastic frontier models. Suitable for a broad range of applications, the implementation offers extensive flexibility in specification and estimation techniques.
Suggests: knitr, rmarkdown, testthat, sn, np, Benchmarking
Imports: pso, numDeriv, gsl, plm, minqa, randtoolbox, mnormt, truncnorm, tmvtnorm, Formula, methods, pbapply, pracma, MASS, parallel, graphics, grDevices
Depends: R (≥ 4.0.0)
License: GPL-2 | GPL-3 [expanded from: GPL (≥ 2)]
Language: en-US
URL: https://www.davidharrybernstein.com/software, https://github.com/davidhbernstein/sfa
BugReports: https://github.com/davidhbernstein/sfa/issues
LazyLoad: yes
NeedsCompilation: no
VignetteBuilder: knitr
Packaged: 2026-08-23 12:05:32 UTC; davidbernstein
Author: David Bernstein ORCID iD [aut, cre], Christopher Parmeter [aut], Alexander Stead [aut]
Repository: CRAN
Date/Publication: 2026-08-23 18:30:02 UTC

Stochastic Frontier Analysis

Description

Provides a user-friendly framework for estimating a wide variety of cross-sectional and panel stochastic frontier models. Suitable for a broad range of applications, the implementation offers extensive flexibility in specification and estimation techniques.

Details

The DESCRIPTION file:

Package: sfa
Version: 1.1.5
Date: 2026-08-23
Title: Stochastic Frontier Analysis
Type: Package
Authors@R: c(person("David", "Bernstein", email = "davebernstein1@gmail.com", role = c("aut", "cre"), comment = c(ORCID = "0000-0002-2267-5741")), person("Christopher", "Parmeter", role = c("aut")), person("Alexander", "Stead", role = c("aut")))
Maintainer: David Bernstein <davebernstein1@gmail.com>
Description: Provides a user-friendly framework for estimating a wide variety of cross-sectional and panel stochastic frontier models. Suitable for a broad range of applications, the implementation offers extensive flexibility in specification and estimation techniques.
Suggests: knitr, rmarkdown, testthat, sn, np, Benchmarking
Imports: pso, numDeriv, gsl, plm, minqa, randtoolbox, mnormt, truncnorm, tmvtnorm, Formula, methods, pbapply, pracma, MASS, parallel, graphics, grDevices
Depends: R (>= 4.0.0)
License: GPL (>= 2)
Language: en-US
URL: https://www.davidharrybernstein.com/software, https://github.com/davidhbernstein/sfa
BugReports: https://github.com/davidhbernstein/sfa/issues
LazyLoad: yes
NeedsCompilation: no
VignetteBuilder: knitr
Author: David Bernstein [aut, cre] (ORCID: <https://orcid.org/0000-0002-2267-5741>), Christopher Parmeter [aut], Alexander Stead [aut]

Index of help topics:

FinnishElec             FinnishElec
Indian                  Indian
PL80_MVTN               Pitt and Lee (1981) Model III: the multivariate
                        truncated normal panel likelihood
USUtilities             USUtilities
data_gen_cs             Generate Cross-Sectional Data for Stochastic
                        Frontier Analysis
data_gen_p              Generate Panel Data for Stochastic Frontier
                        Analysis
npsfm                   Nonparametric Stochastic Frontier Models
panel89                 Panel89
predict.sfareg          Predictions, fitted values and residuals for
                        stochastic frontier fits
print.sfareg            Print method for "sfareg" objects
psfm                    psfm
psfm_bootstrap          Parametric Bootstrap for 'psfm()' Panel
                        Stochastic Frontier Models
sfa-package             Stochastic Frontier Analysis
sfa_diagnostics         Optimizer Diagnostics for Stochastic Frontier
                        Fits
sfareg-methods          Extractor methods for "sfareg" objects
sfm                     sfm
summary.sfareg          Summary method for "sfareg" objects
ttsfm                   Two-Tier Stochastic Frontier Model
zsfm                    Zero-Inefficiency Stochastic Frontier Models

See Also

https://www.davidharrybernstein.com/software

https://github.com/davidhbernstein/sfa

Examples


## Simple application of the generalized true random effects estimator.
library(sfa)

data_trial <- data_gen_p(t=10,N=100,  rand = 100, 
                         sig_u = 1,   sig_v = 0.3, 
                         sig_r = .2,  sig_h = .4, 
                         cons  = 0.5, beta1 = 0.5,
                         beta2 = 0.5)

psfm(formula    = y_gtre ~ x1 + x2,    
     model_name = "GTRE", 
     data       = data_trial,
     individual = "name",
     PSopt      = FALSE)
               

FinnishElec

Description

Cross-sectional data on Finnish electricity distribution firms, including annual averages of expenditure and output measures over a four-year regulatory period.

Usage

data("FinnishElec")

Format

A data frame with 89 observations on the following 6 variables.

id

a character vector containing a unique identifier for each distribution firm

x

a numeric vector containing total expenditure (TOTEX*) (1000 Euros)

y1

a numeric vector containing weighted energy transmitted through the network (GWh of 0.4 kV equivalents)

y2

a numeric vector containing total length of the network (km)

y3

a numeric vector containing total number of customers connected to the network

z

a numeric vector containing the proportion of underground cables in the total network length.

Details

*TOTEX includes capital expenditure (CAPEX), controllable operational expenditure (OPEX), and estimated external cost of interruptions.

Source

Kuosmanen, T. (2012). 'Stochastic semi-nonparametric frontier estimation of electricity distribution networks: Application of the StoNED method in the Finnish regulatory model.' Energy Economics, 34(6), pp. 2189-2199. doi:10.1016/j.eneco.2012.03.005

Examples

data(FinnishElec)
plot(FinnishElec)

Indian

Description

Panel data on 34 paddy farmers from Aurepalle, India, collected over ten years (1975-76 to 1984-85). The panel is unbalanced: farmers are observed between 2 and 10 times. Includes farmer characteristics (age, schooling) and production variables (output, land, labor, bullocks, input costs).

Usage

data("Indian")

Format

A data frame with 273 observations (an unbalanced panel of 34 farmers over 10 years) on the following 10 variables.

id

a numeric vector containing a unique identifier for each farmer

yr

a numeric vector containing the year of the observation

age

a numeric vector containing the age of the primary decision maker

school

a numeric vector containing the number of years of schooling of the primary decision maker

yvar

a numeric vector containing the natural logarithm of the total value of output (rupees)

Lland

a numeric vector containing the natural logarithm of the total area of land operated (ha)

PIland

a numeric vector containing the proportion of land that is irrigated

Llabor

a numeric vector containing the natural logarithm of the total number of hours of hired and family labour used

Lbull

a numeric vector containing the natural logarithm of the number of hours of bullock labour used

Lcost

a numeric vector containing the natural logarithm of the value of inputs including fertilizer, manure, pesticides, machinery, etc.

Source

Battese, G.E. and Coelli, T.J. (1995) 'A model for technical inefficiency effects in a stochastic frontier production function for panel data', Empirical Economics, 20(2), pp. 325-332. doi:10.1007/BF01205442.

References

Battese, G.E. and Coelli, T.J. (1992) 'Frontier production functions, technical efficiency and panel data: With application to paddy farmers in India', Journal of Productivity Analysis, 3(1-2), pp. 153-169. doi:10.1007/BF00158774.

Examples

data(Indian)

Pitt and Lee (1981) Model III: the multivariate truncated normal panel likelihood

Description

Notes on psfm(model_name = "PL80_MVTN"), which fits the likelihood derived in Appendix 2 of Pitt and Lee (1981) but never used there.

Details

The model is

y_{it} = x_{it}'\beta + u_{it} + v_{it}, \qquad u_{it} \le 0,

with the firm's inefficiency vector u_i = (u_{i1},\dots,u_{iT})' drawn from a T-variate normal N(0,\Sigma) truncated to the negative orthant, and v_{it} iid N(0,\sigma_v^2) independent of u. Unlike model_name = "PL80", which holds inefficiency fixed over time, here it varies across periods and is correlated within a firm – \Sigma carries that dependence.

Why it was never used. Pitt and Lee derived this likelihood and then set it aside, writing that it “is difficult to evaluate since the quantities P_0 and P(y_i - x_i\beta) involve T-dimensional numerical integrals”, and estimating Model III by Zellner seemingly-unrelated regression instead. Those quantities are orthant probabilities of a multivariate normal. mnormt::sadmvn() evaluates one in about 3 milliseconds at T = 6, so a likelihood evaluation costs roughly N+1 of them – about 0.3 s at N = 100, and about 12 s for a whole fit at N = 80, T = 4. What was intractable in 1981 is merely slow now.

Writing Q^{-1} = \Sigma^{-1} + I/\sigma_v^2 and \mu_i = Q\varepsilon_i/\sigma_v^2, the per-firm log density is

\log f(\varepsilon_i) = -\tfrac{T}{2}\log 2\pi - T\log\sigma_v - \tfrac{1}{2}\log|\Sigma| + \tfrac{1}{2}\log|Q| - \log P_0 - \tfrac{1}{2}\left(\tfrac{\varepsilon_i'\varepsilon_i}{\sigma_v^2} - \tfrac{\varepsilon_i'Q\varepsilon_i}{\sigma_v^4}\right) + \log P_i,

where P_0 = \Pr(w \le 0) for w \sim N(0,\Sigma) is the truncation constant and P_i = \Pr(w \le 0) for w \sim N(\mu_i, Q).

\Sigma is equicorrelated, not unrestricted. It is parameterized as \Sigma = \sigma_u^2[(1-\rho)I + \rho \mathbf{1}\mathbf{1}'], costing two parameters. An unrestricted \Sigma costs T(T+1)/2 – 21 at T=6, 55 at T=10 – on top of \beta and \sigma_v, every one identified only through orthant probabilities. The equicorrelated form captures what the general \Sigma was introduced for: dependence of a firm's inefficiency across periods. \rho = 0 gives inefficiency independent over time; \rho \to 1 approaches the time-invariant "PL80" case. Because the form is equicorrelated every matrix quantity above is closed form (Sherman-Morrison), so only the orthant probabilities are numerical.

Requirements and limits. A balanced panel with T \ge 2: \Sigma is a single T \times T matrix shared by every firm, and with T = 1 there is no cross-period dependence for it to describe. An unbalanced panel is an error pointing at "PL80". Reported parameters are sigv, sigu, rho and the frontier coefficients.

u_hat is the posterior mean of u from the Gaussian part, floored at zero, not E[u_{it}\mid\varepsilon_i]: the exact conditional mean of a truncated multivariate normal is another T-dimensional integral. It is exact where the truncation does not bind and an approximation where it does.

Author(s)

David H. Bernstein

References

Pitt, M.M. and Lee, L.-F. (1981) 'The measurement and sources of technical inefficiency in the Indonesian weaving industry', Journal of Development Economics, 9(1), pp. 43-64. See Appendix 2 for the likelihood.

See Also

psfm, data_gen_p for the y_pl_mvtn column that tests it.

Examples


library(sfa)
d <- data_gen_p(t = 4, N = 60, rand = 5, sig_u = 1, sig_v = 0.3, sig_r = 0.2,
                sig_h = 0.4, cons = 0.5, beta1 = 0.5, beta2 = 0.5)
f <- psfm(y_pl_mvtn ~ x1 + x2, model_name = "PL80_MVTN",
          data = as.data.frame(d), individual = "name")
f$out


USUtilities

Description

Panel data on U.S. investor-owned fossil fuel-fired steam electric utilities for the period 1986-1997. These data include measures of output, capital, labour and maintenance, and fuel.

Usage

data("USUtilities")

Format

A data frame with 972 observations (a balanced panel of observations on 81 utilities over 12 years) on the following 7 variables.

firmID

a numeric vector containing a unique firm identifier

year

a numeric vector containing the year of the observation

q

a numeric vector containing net steam electric power generation (MWh)

K

a numeric vector containing capital stock, calculated using a method described by Christensen and Jorgenson (1970)

L

a numeric vector containing quantity of labor and maintenance, calculated as cost divided by price index

F

a numeric vector containing quantity of fuel used, calculated as fuel costs divided by fuel price index

trend

a numeric vector containing an annual time trend (1992=100)

Details

The dataset covers 81 investor-owned utilities after aggregating subsidiaries and excluding plants in states with partial deregulation plans. Data sources include the Energy Information Administration (EIA), Federal Energy Regulatory Commission (FERC), and Bureau of Labor Statistics (BLS). Output is net steam electric generation from fossil fuel-fired boilers.

Source

Rungsuriyawiboon, S. and Stefanou, S.E. (2007). 'Dynamic Efficiency Estimation: An Application to U.S. Electric Utilities.' Journal of Business & Economic Statistics, 25(2), pp. 226-238. doi:10.1198/073500106000000288

References

Christensen, L.R. and Jorgenson, D.W. (1970). 'U.S. Real Product and Real Factor Input, 1928-1967.' Review of Income and Wealth, 16(1), pp. 19-50. doi: 10.1111/j.1475-4991.1970.tb00695.x

Examples

data(USUtilities)

Generate Cross-Sectional Data for Stochastic Frontier Analysis

Description

data_gen_cs generates simulated cross-sectional data based on the stochastic frontier model, allowing for different distributional assumptions for the one-sided technical inefficiency error term (u) and the two-sided idiosyncratic error term (v). The model has the general form: Y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + v - u where u \geq 0 and represents inefficiency. All variants are produced so that the user can select those that they want.

Usage

data_gen_cs(N, rand, sig_u, sig_v, cons, beta1, beta2, a, mu, sig_w = sig_u,
            shape_g = 2, m_nak = 1, mu_ln = -0.5, k_w = 1.5)

Arguments

N

A single integer specifying the number of observations (cross-sectional units).

rand

A single integer to set the seed for the random number generator, ensuring reproducibility.

sig_u

The standard deviation parameter (\sigma_u) for the base distribution of the one-sided error term u.

sig_v

The standard deviation parameter (\sigma_v) for the base distribution of the two-sided error term v.

cons

The value of the constant term (intercept) in the model.

beta1

The coefficient for the x_1 variable.

beta2

The coefficient for the x_2 variable.

a

The degrees of freedom parameter for the t half-t distribution (u_t and v_t, respectively). Requires the rt function.

mu

The mean parameter (\mu) for the normal truncated normal distribution (u_tn). Requires the rtruncnorm function.

sig_w

The standard deviation/scale parameter (\sigma_w) for the second one-sided error component w, used to generate two-tier model columns (see ttsfm). Defaults to sig_u.

shape_g

Shape of the gamma inefficiency draw behind y_pcs_g, which targets sfm(model_name = "NG"). The scale is set to sig_u/shape_g, so E[u] = \sigma_u regardless of the shape chosen. sfm()'s NG likelihood reports the shape as mu and the scale as sigu, so the values to recover are shape_g and sig_u/shape_g.

m_nak

Nakagami shape m behind y_pcs_nak, which targets sfm(model_name = "NNAK"). The spread is \Omega = \sigma_u^2, so \sigma_u is the root-mean-square of u. The default 1 is the Rayleigh case; 0.5 would collapse the Nakagami onto the half-normal and so duplicate y_pcs.

mu_ln

Log-scale mean of the lognormal inefficiency draw behind y_pcs_ln, which targets sfm(model_name = "NLN"). The log-scale standard deviation is sig_u. The default -0.5 with sig_u = 1 gives E[u] = 1.

k_w

Weibull shape behind y_pcs_wb, which targets sfm(model_name = "NW"). The scale is sig_u.

Details

The function simulates two explanatory variables, x_1 and x_2, as transformations of uniform random variables.

The function generates several different frontier models by combining various distributions for u and v:

**Specific Model Outputs (y_pcs variants):**

**Note:** The rtruncnorm function is required for y_pcs_tn and loads with the package. In isolation it could be loaded by using library(truncnorm).

Value

A data frame containing N observations with the following columns:

name

Individual identifier (simply 1 to N).

cons

The constant term value.

x1

Simulated explanatory variable x_1.

x2

Simulated explanatory variable x_2.

u, uz, u_t, u_c, u_e, u_u, u_tn, u_st, u_thn

The simulated one-sided error terms under different distributions.

u_r

Rayleigh one-sided error term, for y_pcs_r. Drawn on the convention sfm's "NR" uses, E[u^2] = \sigma_u^2, so the Rayleigh scale is \sigma_u/\sqrt{2}, E[u] = \sigma_u\sqrt{\pi}/2 and Var(u) = (1-\pi/4)\sigma_u^2.

v, v_t, v_c, v_st, v_thn

The simulated two-sided error terms under different distributions.

lam_st

The common Gamma(a/2, a/2) scale-mixing variable behind y_pcs_st; both of that column's components are divided by its square root, which is what makes the composed error skew-t.

y_pcs, y_pcs_t, y_pcs_st, y_pcs_thn, y_pcs_e, y_pcs_ez, y_pcs_c, y_pcs_u, y_pcs_z, y_pcs_w, y_pcs_tn

The dependent variable Y under the corresponding SFA model distributions.

y_pcs_r

Normal-Rayleigh dependent variable, for sfm's "NR". Use this column, not y_pcs: the normal-Rayleigh and normal-half-normal composed errors are different families, not reparameterizations of one another, and "NR" cannot recover y_pcs's parameters. True values are (\sigma_v, \sigma_u) = (sig_v, sig_u).

z

The auxiliary variable used for heteroskedasticity in y_pcs_z, y_pcs_ez, y_tthn_z, and y_zisf_z.

uz_e

Exponential one-sided error term with heteroskedastic scale, for y_pcs_ez.

w_tt, w_tt_hn, wz_hn

The second one-sided error component (w) for the two-tier columns below, under exponential, homoskedastic half-normal, and heteroskedastic half-normal distributions respectively.

zp

A second auxiliary variable used for heteroskedasticity of \sigma_w in y_tthn_z.

y_ttne

Homoskedastic two-tier dependent variable (v + w - u, both one-sided components exponential), for ttsfm's "TTNE"/"TTNLS".

y_tthn

Homoskedastic two-tier dependent variable (both one-sided components half-normal), for ttsfm's "TTHN" with no pipes.

y_tthn_z

Heteroskedastic two-tier dependent variable (\sigma_u a function of z, \sigma_w a function of zp), for ttsfm's "TTHN" with formula ~x1+x2|z|zp.

eff_ind, eff_ind_z

Indicator (1 = efficient, u forced to 0) used to build the zero-inefficiency columns below.

prob_z_true

The true heteroskedastic "efficient regime" probability used to draw eff_ind_z.

y_zisf

Zero-inefficiency dependent variable with constant efficient-regime probability, for zsfm's "ZISF".

y_zisf_z

Zero-inefficiency dependent variable with z-dependent efficient-regime probability, for zsfm's "ZISF_Z".

u_g, y_pcs_g

Gamma one-sided error term and the corresponding dependent variable, for sfm's "NG". Shape shape_g, scale sig_u/shape_g, so E[u] = \sigma_u whatever shape is chosen.

u_nak, y_pcs_nak

Nakagami one-sided error term and dependent variable, for sfm's "NNAK". Shape m_nak, spread \Omega = \sigma_u^2. The default m_nak = 1 is the Rayleigh case, so this column coincides in distribution with y_pcs_r at the default.

u_ge, y_pcs_ge

Generalized-exponential one-sided error term and dependent variable, for sfm's "NGE", drawn as u = -\sigma_u\log(1 - \sqrt{U}) with U \sim \mathrm{Unif}(0,1).

u_ln, y_pcs_ln

Lognormal one-sided error term and dependent variable, for sfm's "NLN". Log-scale mean mu_ln and log-scale standard deviation sig_u.

u_w, y_pcs_wb

Weibull one-sided error term and dependent variable, for sfm's "NW". Shape k_w, scale sig_u.

con

A constant column set to 1, potentially for use in estimation.

Author(s)

David H. Bernstein

See Also

data_gen_p for the panel generator, and sfm, zsfm and ttsfm for the estimators these columns are built for.

rnorm, runif, rt, rexp, rcauchy, rtruncnorm (if available).

Examples


# Generate 100 observations of SFA data
data_sfa <- data_gen_cs(
  N     = 100,
  rand  = 123,
  sig_u = 0.5,
  sig_v = 0.2,
  cons  = 5,
  beta1 = 1.5,
  beta2 = 2.0,
  a     = 5,   # degrees of freedom for T/Half-T
  mu    = 0.1  # mean for Truncated Normal
)

# Display the first few rows of the generated data
head(data_sfa)

# Example of a Normal-Half Normal SFA model data
summary(data_sfa$y_pcs)
plot(density(data_sfa$y_pcs))

Generate Panel Data for Stochastic Frontier Analysis

Description

data_gen_p generates simulated panel data for estimating various panel stochastic frontier models, including the Generalized True Random Effects (GTRE), True Random Effects (TRE), Pooled Cross-Section (PCS), and True Fixed Effects (TFE) models. The function returns the data as a pdata.frame. All variants are produced so that the user can select those that they want.

Usage

data_gen_p(t, N, rand, sig_u, sig_v, sig_r, sig_h, cons, tau = 0.5,
                  mu = 0, beta1, beta2, eta = 0.1, b_k90 = 0.05, c_k90 = 0.01,
                  d_k90 = 0.05, e_k90 = -0.005, rho_mvtn = 0.5)

Arguments

t

The number of time periods.

N

The number of individuals.

rand

A seed for the random number generator to ensure reproducibility.

sig_u

The standard deviation (\sigma_u) for the one-sided error component (u_{it}).

sig_v

The standard deviation (\sigma_v) for the two-sided error component (v_{it}).

sig_r

The standard deviation (\sigma_r) for the two-sided individual effect (r_i).

sig_h

The standard deviation (\sigma_h) for the one-sided individual effect (h_i).

cons

The constant term (\beta_0) for the frontier models.

tau

The dependence parameter (\tau) used for the y_tfe (TFE) model formulation, default is 0.5. See Chen, Schmidt, and Wang (2014, Journal of Econometrics).

mu

The mean parameter (\mu) used for the Truncated-Normal (TN) component of the y_fd model with default set to 0. See Wang and Ho (2010, Journal of Econometrics).

beta1

The coefficient for the x1 variable (\beta_1).

beta2

The coefficient for the x2 variable (\beta_2).

eta

The decay-rate parameter (\eta) for the time-varying inefficiency component u_bc92, following Battese and Coelli (1992): u_{it} = u_i \exp(-\eta(T_i - t)). Default is 0.1. See Battese and Coelli (1992, Journal of Productivity Analysis).

b_k90, c_k90

Coefficients of the Kumbhakar (1990) inefficiency path B_{it} = (1 + \exp(bt + ct^2))^{-1} behind y_k1990, which targets psfm(model_name = "K1990"). The defaults give a path running from 0.485 to 0.182 over ten periods. Note that b and c are weakly identified: quite different pairs produce near-identical fitted paths.

rho_mvtn

Within-firm correlation of inefficiency across periods for the y_pl_mvtn column, which targets psfm(model_name = "PL80_MVTN"). The inefficiency vector is drawn from N(0, \sigma_u^2[(1-\rho)I + \rho \mathbf{1}\mathbf{1}']) truncated to the negative orthant, so 0 gives inefficiency independent over time and values near 1 approach the time-invariant "PL80" case. Default 0.5. Note the realized correlation of the drawn column is ATTENUATED relative to rho by the truncation (about 0.29 at rho = 0.5, T = 4) – rho parameterizes the underlying normal, not the truncated draw.

d_k90, e_k90

Coefficients of the modified Kumbhakar path B_{it} = 1 + d(t - T_i) + e(t - T_i)^2 behind y_k1990m, which targets psfm(model_name = "K1990modified"). The defaults give a path running from 0.145 to 1 over ten periods.

Details

A pdata.frame object with N \times t observations, containing the following columns:

The data is generated based on standard Stochastic Frontier Analysis (SFA) formulations, primarily for a **Production Frontier** where the one-sided error component u_{it} is subtracted:

For **Cost Frontier** models, the one-sided error component u_{it} is added (e.g., c_gtre).

The error terms are generated as:

The First-Difference estimation model (y_fd) uses a variation where r_{i,fd} \sim U(0,1) and u_{it,fd} is generated using a heteroskedastic truncated-normal structure, reflecting an alternative model type.

Value

A pdata.frame object containing N \times t observations suitable for Stochastic Frontier Analysis (SFA).

Author(s)

David H. Bernstein

References

Chen, Y., Schmidt, P., & Wang, H. (2014). Consistent estimation of the fixed effects stochastic frontier model. Journal of Econometrics, 181(2), 65-76.

Filippini, M., & Greene, W. H. (2016). Persistent and transient productive inefficiency: a maximum simulated likelihood approach. Journal of Productivity Analysis, 45, 187-196.

Wang, H., & Ho, C. M. (2010). Estimating fixed-effect panel stochastic frontier models by model transformation. Journal of Econometrics, 157(2), 286-296.

See Also

data_gen_cs for the cross-sectional generator, and psfm for the estimators these columns are built for.

Examples

library(sfa) 
# Generate a dataset 
data_trial <- data_gen_p(t=10, N=100, rand = 100, 
                       sig_u = 1,  sig_v = 0.3, 
                       sig_r = .2, sig_h = .4, 
                       cons = 0.5, tau = 0.5,
                       mu= 0.5, beta1 = 0.5,
                       beta2 = 0.5)
 # See the first few rows 
 head(data_trial)

Nonparametric Stochastic Frontier Models

Description

Fits a stochastic frontier whose frontier function is estimated by kernel regression rather than assumed linear. Two estimators are available: the two-step estimator of Fan, Li and Weersink (1996), and the local method-of-moments estimator of Simar, Van Keilegom and Zelenyuk (2017), which additionally lets both variance components vary with the covariates.

Usage

npsfm(formula, data, method = c("FLW", "SVKZ", "PSZ", "KPST", "MY", "SZ"),
      dist = c("hn", "exp", "gamma", "unif"),
      regtype = c("lc", "ll"), bw.sel = c("cv.ls", "cv.aic"),
      bw = NULL, cost = FALSE, eff = TRUE,
      maxit = 5000, tol = 1e-3, iter = 25,
      rts = c("vrs", "crs", "drs", "irs"),
      prior.fit = NULL, log.form = TRUE, verbose = FALSE)

Arguments

formula

A single-part symbolic description of the frontier, y ~ x1 + x2. Unlike the package's parametric entry points npsfm() takes no | z segment: heteroskedasticity is handled nonparametrically through the covariates themselves under method = "SVKZ", and a pipe is an error rather than something silently ignored.

data

A data frame containing the variables named in formula.

method

Which estimator to use.

"FLW"

Fan, Li and Weersink (1996). Estimates E[y|x] by kernel regression, then recovers the scale parameters from the residuals – by maximizing their concentrated likelihood in \lambda when dist = "hn", and by inverting central moments otherwise. \sigma_u and \sigma_v are constants; only the frontier is smooth.

"SVKZ"

Simar, Van Keilegom and Zelenyuk (2017). Runs three local-linear regressions – y on x, then the squared and cubed residuals on x – and inverts the local moments, so \sigma_u(x) and \sigma_v(x) both vary with the covariates. No optimizer runs. Normal-half normal only.

"PSZ" (or "KPST")

Park, Simar and Zelenyuk. Local maximum likelihood: at every evaluation point the frontier and both log variance components are given local-linear expansions and the kernel-weighted normal-half normal likelihood is maximized in those 3(k+1) parameters. One optimization per observation, so it is much slower than the two above. \sigma_u(x) and \sigma_v(x) vary with the covariates.

"MY"

Martins-Filho and Yao. Iterative local likelihood: alternates a local-linear fit of the frontier at every evaluation point, holding (\lambda,\sigma) fixed, with a global update of (\lambda,\sigma) from the resulting composed residuals, until the scale parameters stop moving. Cost is (observations \times iterations) optimizations. \sigma_u and \sigma_v are constants; only the frontier is local.

"SZ"

Simar and Zelenyuk (2011). Not an estimator in its own right: it takes an already-estimated smooth frontier and passes its fitted values through an output-oriented DEA, imposing the monotonicity and (under rts = "vrs"/"crs") convexity that a kernel fit does not guarantee. Supply the prior fit through prior.fit, or leave it NULL to fit "SVKZ" first. Requires the Benchmarking package and is implemented for production frontiers only.

Matching ignores case, so "nhn", "NHN" and "Nhn" are the same choice.

dist

Distribution of the one-sided inefficiency term, for method = "FLW" only: "hn" (half normal, the default), "exp" (exponential), "gamma", or "unif" (uniform on [0,b]). Every other method is derived for the half-normal case and errors for anything else.

regtype

Kernel regression type passed to np::npregbw: "lc" (local constant, the default) or "ll" (local linear). Applies to "FLW"; "SVKZ" is local linear throughout by construction.

bw.sel

Bandwidth selection method: "cv.ls" (least-squares cross-validation, the default) or "cv.aic". Ignored when bw is supplied.

bw

Optional numeric vector of bandwidths, one per covariate. When supplied, cross-validation is skipped and these are used directly – useful for undersmoothing, for sensitivity analysis, or simply to avoid paying for bandwidth selection repeatedly in a simulation.

cost

Logical. FALSE (the default) fits a production frontier, in which inefficiency is subtracted; TRUE fits a cost frontier.

eff

Logical. Compute observation-level inefficiency predictions (u_hat, exp_u_hat). Available for dist = "hn" and dist = "exp"; the gamma and uniform cases return moment estimates only. Defaults to TRUE.

maxit

Maximum function evaluations for each local optimization under "PSZ" (passed to minqa::bobyqa) and maximum iterations for each local fit under "MY" (passed to optim). Ignored by the other methods. Defaults to 5000. Raise it if convergence reports many non-zero codes.

tol

Convergence tolerance for "MY": the iteration stops when the squared change in (\lambda,\sigma) falls below this. Defaults to 1e-3.

iter

Maximum number of outer iterations for "MY". Defaults to 25.

rts

Returns-to-scale assumption for the DEA step under "SZ": "vrs" (the default), "crs", "drs" or "irs". Passed to Benchmarking::dea.

prior.fit

For "SZ": a numeric vector of already-estimated frontier values, one per observation, to be monotonized. If NULL (the default) an "SVKZ" fit is computed first and used.

log.form

For "SZ": whether the data are in logs, as is conventional in this literature. When TRUE (the default) the frontier and covariates are exponentiated before the DEA step and the result is returned to the log scale.

verbose

Logical. Report progress through the per-observation loops of "PSZ" and the outer iterations of "MY". Defaults to FALSE.

Details

Both estimators relax the parametric frontier of sfm while keeping the composed-error structure y = m(x) + v - u. Because a kernel regression of y on x estimates E[y|x] = m(x) - E[u] rather than m(x), both proceed by fitting that conditional mean and then shifting it back up by an estimate of E[u].

Where they differ. "FLW" treats \sigma_u and \sigma_v as constants, so its correction is a single number and the fitted gradients are those of the conditional mean. "SVKZ" estimates \sigma_u(x) from the local third moment, so the correction varies across observations and the frontier gradient picks up an extra term through the chain rule.

Least squares versus local likelihood. "FLW" and "SVKZ" both begin from a least-squares kernel regression, which estimates E[y|x] = m(x) - E[u] and therefore needs the mean shift described above. "PSZ" and "MY" instead maximize the composed-error likelihood locally, in which the local intercept is m(x) directly and no shift is applied. They pay for that with one numerical optimization per observation – for "MY", per observation per iteration – so expect them to be one to two orders of magnitude slower than "FLW". Both are seeded from an "FLW" fit.

Which to use. On a simulated nonlinear frontier with \sigma_u = 0.6, \sigma_v = 0.25 (3 replications), mean absolute frontier error at n = 300 was 0.070 for "MY", 0.079 for "FLW", and 0.116 for both "SVKZ" and "PSZ". "FLW" is the steadiest for its cost; "MY" is the most accurate if the run time is acceptable; "SVKZ" and "PSZ" earn their keep when \sigma_u genuinely varies with x, which this design does not test.

Wrong skew. The identification of \sigma_u rests on the residuals being negatively skewed. "FLW" inverts a single sample moment and either succeeds or warns. "SVKZ" inverts a local third moment, which is far noisier, and at any point where the estimated skew has the wrong sign the implied \sigma_u(x)^3 is negative; following the paper those points are floored at \sigma_u(x) = 0 and their contribution to the frontier gradient is set to zero. wrong.skew records which observations these were. A large share of them means the local third moment is too noisy to be informative and the "SVKZ" fit should not be trusted – "FLW" is much steadier at moderate sample sizes.

The np dependency. Both estimators need kernel regression and bandwidth selection from the np package, which is listed under Suggests rather than Imports because nothing else in sfa requires it. npsfm() checks for it and stops with an install instruction if it is missing. Bandwidth selection by cross-validation is the dominant cost and scales quadratically in the sample size; supply bw to skip it.

Value

An object of class "npsfareg". This is deliberately not an "sfareg" object: there is no parameter vector with standard errors, so coef(), vcov() and logLik() would have nothing meaningful to return. fitted(), residuals(), nobs(), print() and summary() are provided. Components:

frontier

The estimated frontier \hat{m}(x), i.e. the kernel fit shifted up by the estimated E[u].

frontier.grad

Matrix of estimated frontier gradients, one row per observation and one column per covariate.

conditional.mean

The uncorrected kernel fit of E[y|x], before the mean shift.

residuals

Composed residuals measured against the corrected frontier, y - \hat{m}(x). These are negative up to noise, unlike the centered residuals of the underlying kernel regression.

mean.correction

The estimated E[u]: a scalar for "FLW", a vector for "SVKZ".

sigma.u, sigma.v

Estimated scale parameters. Scalars under "FLW"; vectors of \sigma_u(x_i), \sigma_v(x_i) under "SVKZ".

lambda, sigma

Returned by "FLW" with dist = "hn" only: \lambda = \sigma_u/\sigma_v and \sigma = \sqrt{\sigma_u^2+\sigma_v^2}.

theta

Returned by "FLW" with dist = "exp" or "gamma": the rate parameter of the one-sided term.

b

Returned by "FLW" with dist = "unif": the estimated upper bound of the uniform.

sigma.u.grad, wrong.skew

Returned by "SVKZ": the gradient of \sigma_u(x), and a logical vector flagging observations whose local third moment had the wrong sign. "PSZ" also returns sigma.u.grad and sigma.v.grad, the local-linear slopes of the two log variance functions.

convergence

Returned by "PSZ": the minqa::bobyqa status code from each observation's local optimization, 0 for success. A large share of non-zero codes means maxit is too low and the local fits should not be trusted.

iterations, converged, tol.reached

Returned by "MY": how many outer iterations ran, whether the tolerance was met, and the final squared change in (\lambda,\sigma).

prior.fit, dea.efficiency, rts

Returned by "SZ": the smooth frontier that was monotonized, the DEA efficiency scores, and the returns-to-scale assumption used.

u_hat, exp_u_hat

Jondrow et al. (1982) inefficiency predictions E[u|\varepsilon] and Battese-Coelli (1988) efficiency predictions E[\exp(-u)|\varepsilon]. Returned when eff = TRUE and dist is "hn" or "exp".

bws

The bandwidth object(s) used: one for "FLW", a list of three (r1, r2, r3) for "SVKZ".

method, dist, formula, call, cost, regtype, bw.sel, nobs, total_time, data

Settings and bookkeeping.

Note

Requires the np package, which is in Suggests. Bandwidth cross-validation dominates the run time and scales quadratically in the sample size.

Author(s)

Christopher F. Parmeter and David H. Bernstein

References

Fan, Y., Li, Q. and Weersink, A. (1996) 'Semiparametric estimation of stochastic production frontier models', Journal of Business & Economic Statistics, 14(4), pp. 460-468.

Simar, L., Van Keilegom, I. and Zelenyuk, V. (2017) 'Nonparametric least squares methods for stochastic frontier models', Journal of Productivity Analysis, 47(3), pp. 189-204.

Jondrow, J., Lovell, C.A.K., Materov, I.S. and Schmidt, P. (1982) 'On the estimation of technical inefficiency in the stochastic frontier production function model', Journal of Econometrics, 19(2-3), pp. 233-238.

Battese, G.E. and Coelli, T.J. (1988) 'Prediction of firm-level technical efficiencies with a generalized frontier production function and panel data', Journal of Econometrics, 38(3), pp. 387-399.

See Also

sfm for parametric cross-sectional frontiers, psfm for panel models, and data_gen_cs for simulating data with known true parameters.

Examples


if (requireNamespace("np", quietly = TRUE)) {
  set.seed(42)
  n  <- 150
  x1 <- runif(n, 1, 4)
  x2 <- runif(n, 1, 4)
  m  <- 1 + 0.6 * log(x1) + 0.4 * sqrt(x2)   # nonlinear frontier
  d  <- data.frame(y = m + rnorm(n, 0, 0.25) - abs(rnorm(n, 0, 0.6)),
                   x1 = x1, x2 = x2)

  ## Fan, Li and Weersink, normal-half normal
  f <- npsfm(y ~ x1 + x2, data = d, method = "FLW", dist = "hn")
  f
  head(fitted(f))
  head(f$exp_u_hat)

  ## Simar, Van Keilegom and Zelenyuk: sigma_u and sigma_v vary with x
  g <- npsfm(y ~ x1 + x2, data = d, method = "SVKZ")
  summary(g$sigma.u)
}


Panel89

Description

The dataset is a cross-section of U.S. commercial banks for 1989, extracted from the panel dataset used by Kumbhakar, Parmeter and Tsionas (2013) and based on the Federal Reserve Bank of Chicago's Reports of Condition and Income. It contains detailed cost data with inputs and outputs defined under the intermediation approach, and input prices constructed as expense-quantity ratios.

Usage

data("panel89")

Format

A data frame with 4,985 observations on the following 11 variables.

y

a numeric vector containing the natural logarithm of total cost*

q1

a numeric vector containing the natural logarithm of installment loans

q2

a numeric vector containing the natural logarithm of real estate loans

q3

a numeric vector containing the natural logarithm of business loans

q4

a numeric vector containing the natural logarithm of federal funds sold and securities purchased

q5

a numeric vector containing the natural logarithm of other assets

w1

a numeric vector containing the natural logarithm of the price of labour*

w2

a numeric vector containing the natural logarithm of the price of capital*

w3

a numeric vector containing the natural logarithm of the price of purchased funds*

w4

a numeric vector containing the natural logarithm of the price of interest-bearing deposits in total transaction accounts*

z

a numeric vector containing the natural logarithm of total assets

Details

*The cost and input price variables are normalised by that of a fifth input: the price of interest-bearing deposits in total non-transaction accounts. Total cost is defined as the sum of total expenses for each input. Input prices are derived by dividing the total expense for each input by the corresponding input quantity.

Source

Kumbhakar, S.C., Parmeter, C.F. and Tsionas, E.G. (2013) 'A zero inefficiency stochastic frontier model', Journal of Econometrics, 172(1), pp. 66-76. doi:10.1016/j.jeconom.2012.08.021.

References

Kumbhakar, S.C. and Tsionas, E.G. (2005) 'Measuring technical and allocative inefficiency in the translog cost system: a Bayesian approach', Journal of Econometrics, 126(2), pp. 355-384. doi:10.1016/j.jeconom.2004.05.006.

Examples

data(panel89)
  plot(panel89)

Predictions, fitted values and residuals for stochastic frontier fits

Description

Standard modelling generics for objects of class "sfareg", as returned by sfm, psfm, zsfm and ttsfm.

Usage

## S3 method for class 'sfareg'
predict(object, newdata = NULL,
        type = c("frontier", "response", "efficiency"), ...)
## S3 method for class 'sfareg'
fitted(object, ...)
## S3 method for class 'sfareg'
residuals(object, ...)

Arguments

object

An object of class "sfareg".

newdata

Optional data frame containing the frontier regressors at which to predict. If omitted, the data the model was fitted to is used.

type

"frontier" (default) returns the deterministic frontier x'\beta. "response" shifts it by the predicted inefficiency, giving an estimate of E[y \mid x] rather than of the frontier itself, with the sign following the production/cost convention the model was fitted under. "efficiency" returns the predicted technical efficiency E[\exp(-u) \mid \varepsilon].

...

Ignored.

Details

Only the first part of a pipe formula (y ~ x | z | zp) describes the frontier; later parts parameterize variances and take no part in prediction.

Frontier coefficients are identified by matching the fitted coefficient names against the frontier design matrix, so these methods work for every model in the package without depending on where a given model happens to place its auxiliary parameters (lambda/sigma for NHN, sigv/sigu for NE, sigmaSq/gamma for PL80, and so on).

type = "efficiency" and type = "response" are available only for the estimation sample, since both require the composed residual and therefore the response; requesting them with newdata raises an error rather than returning something misleading. "efficiency" additionally requires a model that returns an efficiency prediction – see sfm for which ones do.

Value

A numeric vector, one element per observation.

Author(s)

David H. Bernstein

See Also

sfm, psfm, sfareg-methods

Examples


library(sfa)
d <- data_gen_cs(N = 200, rand = 1, sig_u = 0.8, sig_v = 0.3,
                 cons = 1, beta1 = 0.5, beta2 = 0.5, a = 5, mu = 0.5)
f <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d)

head(fitted(f))
head(residuals(f))
head(predict(f, type = "efficiency"))
head(predict(f, newdata = d[1:5, ]))


Print method for "sfareg" objects

Description

Compact display of a stochastic frontier fit, as returned by sfm, psfm, zsfm or ttsfm.

Usage

## S3 method for class 'sfareg'
print(x, ...)

Arguments

x

An object of class "sfareg", as returned by sfm, psfm, zsfm or ttsfm.

...

Additional arguments passed to other methods

Details

Prints the model name, the estimated coefficients with their standard errors and t-values (the out matrix), and the log-likelihood where the estimator has one. For a fuller display, including the frontier and variance-parameter blocks, use summary.sfareg.

Value

No return value, called for side effects

Author(s)

David H. Bernstein

See Also

summary.sfareg, sfareg-methods

Examples


library(sfa)     

cs_data_trial   <- data_gen_cs(N= 1000, rand   = 1,  sig_u  = 0.3, sig_v  = 0.3, 
cons   = 0.5,       beta1  = 0.5,   beta2  = 0.5, a      = 4, mu     = 1)

cs.nhnz     <-  sfm(formula    = y_pcs_z ~ x1 +x2| z,    model_name = "NHN_Z",
                    data       = cs_data_trial,          PSopt      = TRUE)
print(cs.nhnz)


psfm

Description

Function to implement various panel data stochastic frontier estimators

Usage

psfm(formula, model_name = c("TRE_Z", "GTRE_Z", "TRE",
                    "GTRE", "GTRE_FML", "TFE", "TFE_WMLE", "FD",
                    "GTRE_SEQ1", "GTRE_SEQ2", "SSFE",
                    "PL80", "PL80_MVTN", "BC92", "K1990", "K1990modified"), data,
                    maxit.bobyqa = 5000, maxit.nlminb = 500,
                    maxit.psoptim = 100, maxit.optim = 1000,
                    REPORT = 1, trace = 3, pgtol = 0, individual,
                    halton_num = NULL, start_val = FALSE, gamma = FALSE,
                    PSopt = FALSE, optHessian = TRUE, inefdec = TRUE, Method = "L-BFGS-B",
                    verbose = FALSE, rand.gtre = NULL, rand.psoptim = NULL,
                    OPG_calc = FALSE,
                    estimator = c("fiml", "sml", "seq1", "seq2"),
                    collinear_action = c("start_only", "error", "warn_drop"),
                    time = NULL, tfe_lambda_max = 100)

Arguments

formula

a symbolic description for the model to be estimated

model_name

Which panel estimator to fit. All are described in ‘Details’; in brief:

"TRE", "TRE_Z"

Greene's (2005) true random effects, by simulated maximum likelihood. _Z adds determinants of \sigma_u.

"GTRE", "GTRE_Z"

the four-component model separating persistent from transient inefficiency, by simulated maximum likelihood. _Z adds determinants.

"GTRE_FML"

the same four-component model by full information maximum likelihood through its closed-skew-normal representation; deterministic, but requires a balanced panel.

"GTRE_SEQ1", "GTRE_SEQ2"

two-step/moment-based estimators of the four-component model. Not maximum likelihood – no logLik().

"TFE"

Greene's (2005) true fixed effects. Changed meaning in 1.1.3 – see ‘Details’.

"TFE_WMLE"

Chen, Schmidt and Wang's (2014) within maximum-likelihood estimator.

"FD"

the first-difference estimator of Wang and Ho (2010).

"SSFE"

Schmidt and Sickles' (1984) fixed-effects estimator, inefficiency read off the firm effects. Not maximum likelihood – no logLik().

"PL80"

Pitt and Lee (1980): time-invariant inefficiency.

"BC92"

Battese and Coelli (1992): exponential time decay.

"PL80_MVTN"

Pitt and Lee's (1981) Model III: inefficiency varies over time and is CORRELATED within a firm, through a multivariate normal truncated to the negative orthant. This is the likelihood from their Appendix 2, which they derived but did not use, calling the T-dimensional integrals intractable. Balanced panels only. See PL80_MVTN.

"K1990", "K1990modified"

Kumbhakar (1990) time patterns for inefficiency.

Matching ignores case, so "nhn", "NHN" and "Nhn" are the same choice.

data

A panel data set. This may be an ordinary data.frame (or tibble/data.table) – the panel index is constructed internally from the individual and time arguments – or a plm::pdata.frame that already carries an index, which is used as-is.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine

maxit.nlminb

Maximum number of iterations for the nlminb optimization routine. Previously hard-coded (200 in the GTRE_FML branch, 500 elsewhere) and not settable from the call.

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine

maxit.optim

Maximum number of iterations for the optim optimization routine

REPORT

reporting parameter

trace

Integer. Tracing level passed through to the optimizer; larger values print more.

pgtol

Numeric. Projected-gradient tolerance passed to optim()'s "L-BFGS-B" method. 0 uses the optimizer's own default.

individual

individual unit in the regression model

halton_num

number of Halton draws to use in SML models

start_val

starting value (optional)

gamma

Logical. Report the variance parameters in the \gamma = \sigma_u^2/(\sigma_u^2 + \sigma_v^2) parameterization instead of the default. Defaults to FALSE.

PSopt

use psoptim optimization routine (T or F)

optHessian

Logical. Should a numerically differentiated Hessian matrix be returned while using the optim routine? (for optim routine)

inefdec

Production or cost function

Method

The method to be used for optim. See 'Details' within optim.

verbose

Logical. Print optimization progress messages? Default is FALSE.

rand.psoptim

Integer. Seed for replication of psoptim. Default to NULL.

rand.gtre

Integer. Seed for replication of the gtre model. Default to NULL.

OPG_calc

Logical. Should the outer product of gradients be calculated for inference. Default to FALSE.

estimator

How to estimate the four-component GTRE model, in the same spirit as sfm's estimator: one model, several routes to it. Applies to model_name = "GTRE" only, and is ignored with a warning elsewhere.

"fiml"

the default. Full information maximum likelihood through the closed-skew-normal representation. Deterministic – nothing is integrated by Monte Carlo, so repeated fits to the same data agree exactly. Requires a balanced panel.

"sml"

Simulated maximum likelihood over Halton draws. Handles unbalanced panels. This is what model_name = "GTRE" meant through version 1.1.3.

"seq1", "seq2"

The two-step moment-based estimators. Not maximum likelihood, so they carry no logLik().

The older names remain and are unchanged: "GTRE_FML", "GTRE_SEQ1" and "GTRE_SEQ2" select the same three routes directly.

On an unbalanced panel "fiml" cannot be fitted. If you simply took the default, psfm() warns and falls back to "sml" – erroring would make "GTRE" unusable by default on a whole class of data. If you asked for "fiml" explicitly it errors instead, rather than silently fitting a different estimator than the one you named.

collinear_action

One of "start_only" (default), "error", or "warn_drop". Controls what happens when the between-individual design matrix used to build starting values is rank deficient. This is checked separately from the pooled design because the two can differ: time dummies, for example, are routinely estimable in a pooled specification while collapsing onto the intercept once averaged within each unit, which is exactly the situation that makes the internal random-effects initialization fail. "start_only" keeps the model you asked for and removes the offending columns from the starting-value regression only, taking their starting values from a pooled OLS fit; "error" stops before estimation and names the offending columns; "warn_drop" removes those terms from the estimated model. All three report what was found. Only relevant to the models initialized by a random-effects regression (GTRE, TRE and their _Z/SEQ variants).

time

Character. Name of a time-period column in data, used by the error-components frontier models ("PL80", "BC92", "K1990", "K1990modified"). If NULL (the default), a sequential within-individual time index is constructed from row order, which assumes data is already sorted chronologically within each individual.

tfe_lambda_max

Numeric. Upper bound placed on \lambda = \sigma_u/\sigma_v when fitting model_name = "TFE"; ignored by every other model. Defaults to 100, i.e. \sigma_v is not allowed below about 1% of \sigma. This is a guard against a genuine degeneracy of Greene's likelihood rather than a convergence tolerance – see ‘Details’. A fit that pins at the bound warns and should not be read as an estimate of \lambda.

Details

The generalized true random effects model (GTRE, 4-component model) and true random effects models (TRE) are both estimated by simulated maximum likelihood based on the paper by the Fillipini and Greene (2016, JPA). The TRE_Z and GTRE_Z allow for modeling the u-component of the GTRE and TRE with determinants of inefficiency. The first-difference estimator (FD) of Wang and Ho (2010, JoE) is also available.

Choosing how GTRE is estimated. Since version 1.1.4 the four routes to the four-component model are selected with estimator rather than by picking among four model_name values that looked like four different models. model_name = "GTRE" now defaults to estimator = "fiml"; through 1.1.3 it fitted the simulated-ML estimator, which is estimator = "sml". Scripts that pass "GTRE" therefore get a different estimator than they did and are warned once per call. The explicit names "GTRE_FML", "GTRE_SEQ1" and "GTRE_SEQ2" are unaffected.

Two ways of estimating the same four-component model. "GTRE" and "GTRE_FML" both fit

y_{it} = \beta_0 + x_{it}'\beta + r_i - h_i + v_{it} - u_{it},

with r_i, v_{it} two-sided and h_i, u_{it} one-sided (half-normal), separating persistent inefficiency h_i from transient inefficiency u_{it}. They differ in how the likelihood is obtained.

"GTRE" uses simulated maximum likelihood over Halton draws. "GTRE_FML" uses full information maximum likelihood through the model's closed-skew-normal representation: with A = -[1_T, I_T], V = \mathrm{diag}(\sigma_h^2, \sigma_u^2 I_T) and \Sigma = \sigma_v^2 I_T + \sigma_r^2 1_T 1_T', the firm-level density is available in closed form and nothing is integrated by Monte Carlo. The likelihood is therefore deterministic, and repeated fits to the same data give identical answers.

The (T+1)-dimensional normal CDF this requires would normally be the bottleneck, but its covariance is diagonal-plus-rank-one for this model, so it collapses to a one-dimensional Gauss-Hermite integral – the same reduction used for "TFE".

"GTRE_FML" reports raw standard deviations sigr, sigv, sigh, sigu (individual effect, noise, persistent inefficiency, transient inefficiency), not the \lambda/\sigma reparameterization "GTRE" uses, and it requires a balanced panel – the closed-skew-normal pieces are built at a single T. It errors rather than silently mis-estimating if the panel is unbalanced; use "GTRE" in that case.

Starting values for "GTRE_FML". The closed-skew-normal likelihood carries a boundary optimum at \sigma_h = 0, where the model collapses to "TRE" and the intercept absorbs the missing E[h] = \sigma_h\sqrt{2/\pi}. A fit that falls into it can have a lower likelihood than the true parameter vector, so it is an optimizer failure rather than a limit of the data. Following Colombi (2010) and Colombi, Martini and Vittadini (2011), who recommend seeding this likelihood from the two-step moment estimates, psfm() builds two candidate starts – the random-effects decomposition shared with "GTRE", and the two-step estimator reported by "GTRE_SEQ2" – evaluates the likelihood at each, and begins from the better one. Which was chosen, and the log-likelihood at both, is returned in start_search. Supplying start_val overrides the search.

Note that \sigma_r and \sigma_h enter the density only through \sigma_r^2 + \sigma_h^2; their separation rests entirely on the skewness term. That pair is therefore the least precisely estimated part of the model even when the fit is at the global optimum, which is consistent with the confounding between latent heterogeneity and persistent inefficiency reported in the source papers.

The two true-fixed-effects estimators, and a change of meaning in "TFE". Both "TFE" and "TFE_WMLE" fit the same model,

y_{it} = \alpha_i + x_{it}'\beta + v_{it} - u_{it},\quad v \sim N(0,\sigma_v^2),\ u \sim N^+(0,\sigma_u^2),

with \alpha_i an unrestricted individual effect; they differ in how they estimate it.

"TFE" is Greene's (2005, JoE/JPA) true fixed effects estimator: the ordinary composed-error likelihood with the N individual intercepts estimated jointly with (\lambda, \sigma, \beta). This package concentrates the \alpha_i out at every likelihood evaluation (each is the unique root of a strictly decreasing score, solved by safeguarded Newton to machine precision), so the reported parameter vector, its standard errors and the out layout are the same 2 + K as "TFE_WMLE". It is the estimator subject to the incidental parameters problem: with T fixed, \sigma_u is biased upward and efficiency scores correspondingly pessimistic.

"TFE_WMLE" is Chen, Schmidt and Wang's (2014, JoE) within maximum-likelihood estimator, which applies maximum likelihood to the deviations from individual means. The \alpha_i drop out of the likelihood entirely, so the estimator is free of the incidental parameters problem; the firm effects are recovered afterwards from the mean residual with the usual \sqrt{2/\pi}\,\sigma_u adjustment.

The two are not nested and their log-likelihoods are not on the same scale – "TFE" evaluates the likelihood of the data, "TFE_WMLE" that of the within-transformed deviations – so logLik(), AIC() and BIC() must not be used to choose between them.

Through sfa 1.1.2, model_name = "TFE" selected the Chen-Schmidt-Wang estimator. It now selects Greene's, matching standard usage; the Chen-Schmidt-Wang estimator is "TFE_WMLE". Existing scripts that pass "TFE" therefore get a different estimator than they did, and are warned once per call.

Degeneracy of the Greene likelihood. Because \alpha_i is unrestricted, Greene's likelihood always has a supremum on the \sigma_v \to 0 boundary: setting \alpha_i = \max_t (y_{it} - x_{it}'\beta) makes every composed error weakly negative, \Phi(-\lambda e/\sigma) \to 1, and what remains is the deterministic-frontier likelihood, which is finite and attained. On simulated panels this supremum is reached along a path that first dips below the interior local maximum and then rises past it, so an unconstrained search that escapes the interior basin returns \sigma_v \approx 0 together with meaningless standard errors; when \lambda is large and T small, no interior maximum need exist at all. tfe_lambda_max bounds the search accordingly, and a fit that pins at the bound warns. This is a property of the estimator, not of the implementation, and is one of the motivations for "TFE_WMLE".

Both problems are small-T phenomena. Simulations in which T grows with N fixed at 100 (truth \lambda = 3.333) recover \hat\lambda = 88.5, 4.8, 3.6, 3.6, 3.5 at T = 10, 20, 30, 40, 50: the fit is pinned at the bound at T = 10 and has converged by T = 50. The degeneracy and the incidental-parameters bias dissolve together as T grows, so tfe_lambda_max matters on short panels and is irrelevant on long ones. "TFE_WMLE", having no incidental parameters, is unaffected by T either way.

"SSFE" is the classical Schmidt and Sickles (1984, JBES) fixed-effects (LSDV) estimator: a standard within/fixed-effects panel regression, with technical inefficiency read directly off the estimated firm effects (each firm's distance from the best-performing firm in the sample) rather than estimated by maximum likelihood. It requires no numerical optimization and returns no log-likelihood – logLik()/AIC()/BIC() are not defined for it (see sfareg-methods). Note this is a different model from "TFE"/"TFE_WMLE" above, despite the similar names: both of those are maximum-likelihood estimators that allow a separate stochastic noise term on top of the fixed effects, while "SSFE" attributes all within-firm deviation to inefficiency directly.

"K1990" and "K1990modified" (Kumbhakar, 1990) extend the same error-components frontier with richer time paths for inefficiency: "K1990" uses B_{it} = (1 + \exp(b t + c t^2))^{-1} and "K1990modified" uses B_{it} = 1 + d(t - T_i) + e(t - T_i)^2, against "BC92"'s \exp(-\eta(t - T_{ref})) and "PL80"'s time-invariant B_{it} = 1. All four share one likelihood, differing only in B_{it}. Note that "K1990"'s b and c are weakly identified – quite different pairs produce near-identical fitted time paths – so the fitted B_{it} path is more interpretable than either coefficient on its own.

"PL80" (time-invariant panel inefficiency, Pitt and Lee, 1980, JoE) and "BC92" (time-varying decay, Battese and Coelli, 1992, JPA) are both natively estimated maximum-likelihood implementations of the same "error components frontier": y_it = x_it'beta + v_it - B_it*u_i (production; sign flips for cost), with v_it iid normal, u_i iid half-normal and constant across time within a firm, and B_it = 1 for "PL80" or B_it = exp(-eta*(t - Tref)) for "BC92" (Tref being the last time period in the whole panel). The closed-form log-likelihood is obtained by integrating u_i out of the joint density of (epsilon_i, u_i); it nests the ordinary cross-sectional normal-half-normal likelihood when every firm has a single observation. Verified against frontier::sfa() (matching coefficients, log-likelihood, and frontier::efficiencies()-style predicted efficiencies to numerical precision across balanced/unbalanced panels and production/cost specifications) before that dependency was removed. The time argument above supplies each observation's period for "BC92"'s decay term; if left NULL, a sequential within-individual time index is constructed from row order, which assumes data is already sorted chronologically within each individual.

Note on default iteration counts: psfm()'s defaults were raised in version 1.1.3, from 100/10/10 for maxit.bobyqa/maxit.psoptim/maxit.optim to 5000/100/1000, and maxit.nlminb (previously hard-coded) is now an argument. The old values were binding rather than merely economical: the K1990/K1990modified branch already carried a note that 100 bobyqa evaluations left its seven-parameter fits several log-likelihood units short of the optimum purely on the iteration cap, and GTRE_FML at N = 500, T = 10 roughly halves its root-mean-square error against known true values when the caps are lifted, at about 1.5 times the run time. They remain below sfm()'s defaults because several psfm() models (GTRE, GTRE_Z, TRE, TRE_Z) are estimated by simulated maximum likelihood over Halton draws, so each objective evaluation is far more expensive than the closed-form likelihoods used elsewhere. Lower them explicitly if a fast approximate call is wanted.

Value

An object of class "sfareg" containing components that vary by model. All models return:

out

A matrix with parameter estimates, standard errors, and t-values.

opt

A list containing the optimization results from the final optimization procedure (not returned for GTRE_SEQ1 and GTRE_SEQ2).

total_time

The total computation time for model estimation.

start_v

The starting values used in the optimization (not returned for GTRE_SEQ1 and GTRE_SEQ2).

model_name

The name of the panel stochastic frontier model estimated.

formula

The formula used in the model specification.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE).

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

call

The matched call.

data

The data used in estimation.

Additional model-specific components:

For GTRE_FML model:

start_search

A list recording the starting-value search: n_tried, the log-likelihood at each candidate (loglik), and which was chosen ("random-effects" or "two-step"). NULL when start_val was supplied.

U

Predicted transient (time-varying) technical efficiency for each observation.

H

Predicted persistent (time-invariant) technical efficiency for each individual, named by individual.

For GTRE and GTRE_Z models:

H

Predicted time-invariant technical efficiency for each individual.

For GTRE_Z model:

out_opg

OPG (outer product of gradients) standard errors and t-values along with parameters.

out_sandwich

sandwich standard errors and t-values along with parameters.

For GTRE, GTRE_Z, TRE and TRE_Z models:

U

Predicted time-varying technical efficiency for each observation.

For TFE and TFE_WMLE models:

r_hat_m

Estimated individual-specific fixed effects, named by individual. For "TFE" these are the maximum-likelihood estimates of \alpha_i directly; for "TFE_WMLE" they are recovered after estimation from the mean residual plus \sqrt{2/\pi}\,\sigma_u, since the within likelihood does not identify them.

exp_u_hat

Predicted technical efficiency, E[\exp(-u)|\epsilon] (Battese and Coelli, 1988).

For TFE model only:

u_hat

Predicted inefficiency, E[u|\epsilon] (Jondrow et al., 1982).

For FD model:

u_hat

Predicted technical efficiency in levels.

h_hat

Estimated z heterogeneity function values.

exp_u_hat

Predicted technical efficiency.

For GTRE_SEQ1 and GTRE_SEQ2 models:

other_parms

A matrix of additional parameters (lambda, sigma, beta_0 for SEQ1; sigma_u, sigma_v, sigma_h, sigma_r, lambda, sigma for SEQ2).

For SSFE model:

alpha_hat

The estimated firm-specific fixed effects (named by individual).

u_hat

Estimated technical inefficiency for each firm, as its distance from the best-performing firm's fixed effect.

exp_u_hat

Predicted technical efficiency, exp(-u_hat).

For PL80 and BC92 models:

exp_u_hat

Predicted technical efficiency for each observation, E[exp(-B_it*u_i)|epsilon_i] (generalizes Battese and Coelli's (1988, JoE) point estimator to the panel case, pooling all of a firm's observations to predict u_i and scaling by B_it per period).

Note

Standard errors require optHessian set to TRUE

Note

The GTRE_SEQ1 and GTRE_SEQ2 models use sequential estimation methods and do not return optimization objects or starting values. The SSFE model is likewise not maximum-likelihood based and returns no optimization object, starting values, or log-likelihood. All panel models require the individual argument to identify panel units.

Author(s)

David H. Bernstein

References

Fillipini and Greene (2016, JPA); Colombi, Kumbhakar, Martini and Vittadini (2014, JPA) on the closed-skew-normal four-component likelihood, with Colombi (2010) and Colombi, Martini and Vittadini (2011) on the two-step estimator used to seed it; Greene (2005, JoE and JPA); Wang and Ho (2010, JoE); Chen, Schmidt and Wang (2014, JoE); Kumbhakar (1990, JoE); Pitt and Lee (1980, JoE); Battese and Coelli (1992, JPA); Battese and Coelli (1988, JoE); Jondrow, Lovell, Materov and Schmidt (1982, JoE); Schmidt and Sickles (1984, JBES)

See Also

sfm for cross-sectional models, psfm_bootstrap for bootstrap standard errors, data_gen_p for simulating panel data with known true parameters, and sfareg-methods for extracting results.

Examples


library(sfa)     

data_trial <- data_gen_p(t=10,N=100, rand = 100, 
                         sig_u = 1,  sig_v = 0.3, 
                         sig_r = .2, sig_h = .4, 
                         cons = 0.5, beta1 = 0.5,
                         beta2 = 0.5)

max_tre_z   <-  psfm(formula    = y_tre_z ~ x1 +x2| z_gtre,
                     model_name = "TRE_Z",
                     data       = data_trial,
                     individual = "name",
                     PSopt      = TRUE)


Parametric Bootstrap for psfm() Panel Stochastic Frontier Models

Description

Performs a parametric bootstrap for panel stochastic frontier models fit with psfm. Supports model_name %in% c("GTRE_Z", "TRE_Z", "GTRE", "TRE", "TFE", "FD") – every psfm() model except the moment-based/LSDV estimators ("GTRE_SEQ1", "GTRE_SEQ2", "SSFE", which are not maximum-likelihood and have no $opt to bootstrap around in the same sense) and the sfa-wrapped models ("PL80", "BC92", whose $out/efficiency-score fields don't follow this package's own conventions). Each supported model's frontier/variance-component parameter blocks are read directly from psfm_object$out and psfm_object$formula; for each of BOOT replications, a new response is simulated from the fitted parameters using a data-generating process appropriate to that specific model (see Details), the model is re-estimated in parallel, and bootstrap standard errors / t-values are returned for every parameter in psfm_object$out.

Usage

psfm_bootstrap(psfm_object,
               numCores,
               BOOT,
               individual,
               h_type        = c("auto", "none", "scalar", "parametric"),
               maxit.psoptim = 1000,
               seed_offset   = 0,
               write_back    = TRUE,
               pkgs          = c("sfa", "Formula", "pbapply", "truncnorm"),
               inefdec,
               rand.gtre     = NULL,
               rand.psoptim  = NULL,
               maxit.bobyqa  = 1,
               maxit.optim   = 1)

Arguments

psfm_object

A fitted model object of class "sfareg" returned by psfm, with model_name %in% c("GTRE_Z", "TRE_Z", "GTRE", "TRE", "TFE", "FD"). Must contain $out, $data, $formula, and $model_name; $H is additionally required for "GTRE"/"GTRE_Z" (the only two models that return it) and $r_hat_m is additionally required for "TFE".

numCores

Integer. Number of cores for the parallel cluster created internally via makeCluster.

BOOT

Integer. Number of bootstrap replications (e.g. 199, 399, 999).

individual

Character. Name of the individual/panel-ID column in psfm_object$data (e.g. "firm.id").

h_type

One of "auto", "none", "scalar", or "parametric". Only meaningful for model_name %in% c("GTRE_Z", "TRE_Z", "GTRE", "TRE") (ignored for "TFE"/"FD", which have no random persistent-inefficiency h term); controls how the individual-specific, time-invariant inefficiency term h is simulated:

"none"

The model has no h term at all (TRE, TRE_Z).

"scalar"

h_i ~ abs(N(0, sigma_h)), a single scalar standard deviation – used automatically for bare "GTRE" (which has no h-covariates at all, only a single fitted sigh).

"parametric"

h_i ~ abs(N(0, exp(data_h %*% h_vec))), using the third right-hand-side part of the model formula as the h-covariates – used automatically for "GTRE_Z".

"auto"

(Default) Resolves per model_name: "parametric" for "GTRE_Z", "scalar" for bare "GTRE", "none" for "TRE_Z"/"TRE".

maxit.psoptim

Integer. Passed through to psfm() as maxit.psoptim on each bootstrap re-estimation. Default 1000.

seed_offset

Integer added to the bootstrap index b when calling set.seed(), allowing reproducible-but-distinct seeds across multiple bootstrap runs. Default 0.

write_back

Logical. If TRUE (default), the returned list includes model, a copy of psfm_object with the bootstrap standard errors and t-values written into $out[,2] and $out[,3]. If FALSE, only the raw bootstrap results are returned.

pkgs

Character vector of package names to load on each cluster worker via clusterCall. Default c("sfa", "Formula", "pbapply", "truncnorm")truncnorm is only actually used when bootstrapping an "FD" fit, but is included in the default so a single default works for every supported model.

inefdec

Logical, with no default; must be supplied. Passed through to psfm() on each bootstrap re-estimation. Should match the inefdec setting used to originally fit psfm_object.

rand.gtre

Passed through to psfm() as rand.gtre on each bootstrap re-estimation. Default NULL.

rand.psoptim

Passed through to psfm() as rand.psoptim on each bootstrap re-estimation. Default NULL.

maxit.bobyqa

Integer. Passed through to psfm() as maxit.bobyqa on each bootstrap re-estimation. Default 1.

maxit.optim

Integer. Passed through to psfm() as maxit.optim on each bootstrap re-estimation. Default 1.

Details

Supported models fall into three families, each with its own data-generating process and $out row layout (see R/psfm_bootstrap.R's header comment for the full derivation, including how each was cross-checked against that model's own likelihood in psfm.R):

"randeff" family (GTRE_Z, TRE_Z, GTRE, TRE)

y* = X %*% beta_hat + v + u + r + h (sign per inefdec), with v ~ N(0, sigma_v^2), u half-normal (covariate-driven sigma for GTRE_Z/TRE_Z, a single homoskedastic scalar for bare GTRE/TRE, which have no z pipe at all), r ~ N(0, sigma_r^2) (one draw per individual, repeated over time), and h per h_type (GTRE/GTRE_Z only). $out row layout: sigv, sigr, [x-block], [z-block], [h-block] for GTRE_Z/TRE_Z; lambda, sigma, sigr, [sigh], [x-block] for bare GTRE/TRE (note the different reparameterization – see psfm's own model-family sigma-link inconsistency note).

"tfe" family (TFE)

Fixed-effects model: the individual effects are held FIXED at psfm_object$r_hat_m's original point estimates across every replication (not redrawn – redrawing them would contradict treating them as fixed rather than random), and only v ~ N(0, sigma_v^2) and u ~ |N(0, sigma_u^2)| (both derived from the fitted lambda/sig, or gamma/sig if the original fit used psfm(..., gamma = TRUE)) are redrawn. $out row layout: lambda (or gamma), sig, [x-block] (no intercept row – the within-transformation removes it).

"fd" family (FD)

Wang and Ho (2010) first-difference estimator: a single time-invariant u_i ~ TruncatedNormal(mean = mu, sd = sqrt(sig_u2), lower = 0) draw per individual (via rtruncnorm), scaled by a deterministic time-varying factor h_it = exp(z_it %*% delta), plus v_it ~ N(0, sig_v2). $out row layout: sig_u2, sig_v2, mu, [x-block], [z-block] (no intercept row in either the x- or z-block).

psfm_bootstrap validates that nrow(psfm_object$out) matches the expected layout for the resolved model/h_type before running any bootstrap replications, and stops with an informative error if it does not. For GTRE/GTRE_Z it similarly validates up front that length(psfm_object$H) equals the number of unique individuals implied by the individual column ($H is one value per individual, not one per observation) – TRE/TRE_Z/TFE/FD do not return $H at all, so boot_eff_h is simply NULL for those four models (not an error).

The model is re-estimated on each simulated dataset via sfa::psfm(), executed in parallel across replications using parLapply (or pblapply for a live progress bar, when the pbapply package is installed). Whether the psoptim (particle-swarm) optimizer stage is used on each refit is model-dependent: it is enabled for the "randeff" family (matching this function's original behavior, tested clean), but disabled for "TFE" – testing found psoptim's wide exploration made TFE repeatedly collapse onto a degenerate variance-boundary solution (reproduced even refitting the SAME real data across different psoptim seeds), a problem that disappeared entirely once that stage was skipped (matching psfm()'s own default of PSopt = FALSE). As a defense-in-depth safety net across every family, any replication whose refit still lands on a degenerate variance-boundary mode (a scale parameter pinned near its numerical floor – the same well-understood boundary-mode phenomenon documented for ttsfm()'s TTNE/TTHN models) is treated as a per-replication FAILURE rather than silently corrupting the bootstrap standard error with a meaningless draw.

Re-estimation failures (optimizer errors, or the degenerate-mode check just described) are caught per replication, recorded with their message, and set to NA in the output matrices rather than stopping the bootstrap; a single warning at the end lists every failed replication index together with its message. Bootstrap standard errors are computed with na.rm = TRUE, so they remain well-defined even when some replications failed.

Requires the parallel and Formula packages (and truncnorm when bootstrapping an "FD" fit), and requires sfa (and any other packages named in pkgs) to be installed and loadable on each cluster worker.

Value

A list with components:

boot_par

A BOOT x (nrow(psfm_object$out) + 2) matrix of bootstrapped parameter draws, with the final two columns "loglik" and "hours". Rows corresponding to failed replications are NA.

boot_eff

A BOOT x nrow(psfm_object$data) matrix of bootstrapped per-observation efficiency-score draws ($U for the "randeff" family, $exp_u_hat for "TFE"/"FD" – the field is named differently across these models in psfm.R, but always collected into this same boot_eff matrix here). Rows corresponding to failed replications are NA.

boot_eff_h

Only non-NULL for model_name %in% c("GTRE", "GTRE_Z") (the only two models that return $H at all); NULL otherwise. A BOOT x length(psfm_object$H) matrix of bootstrapped time-invariant, individual-specific inefficiency-score ($H) draws – one column per unique individual (not one column per observation; $H does not vary over time within an individual). Columns are named by the individual ID values. Rows corresponding to failed replications are NA.

se

Named numeric vector of bootstrap standard errors for every parameter in psfm_object$out.

tval

Named numeric vector of bootstrap t-values (estimate / boot SE) for every parameter in psfm_object$out.

failures

Integer vector of replication indices b for which re-estimation failed, or NULL if none failed.

model

Only present when write_back = TRUE. A copy of psfm_object with $out[,2] and $out[,3] replaced by the bootstrap standard errors and t-values.

Author(s)

David H. Bernstein and Christopher F. Parmeter

See Also

psfm, parLapply, pblapply, Formula

Examples

## Not run: 
## Not run: a meaningful bootstrap needs BOOT in the hundreds, each
## replication refitting the model on a PSOCK cluster, so this takes
## minutes rather than seconds.

data_trial <- data_gen_p(t = 5, N = 50, rand = 100,
                         sig_u = 1,   sig_v = 0.3,
                         sig_r = 0.2, sig_h = 0.4,
                         cons  = 0.5, beta1 = 0.5, beta2 = 0.5)

GTRE_ZZ <- psfm(y_gtre_zz ~ x1 + x2 | z_gtre | zp_gtre,
                model_name = "GTRE_Z",
                data       = data_trial,
                individual = "name")

boot_results <- psfm_bootstrap(
  psfm_object = GTRE_ZZ,
  numCores    = 2,
  BOOT        = 199,
  individual  = "name",
  h_type      = "auto",
  inefdec     = FALSE
)

## Bootstrap standard errors / t-values for every model parameter
boot_results$se
boot_results$tval

## Bootstrap draws of time-varying (U) and time-invariant (H)
## efficiency/inefficiency scores, one row per replication
boot_results$boot_eff
boot_results$boot_eff_h

## Fitted object with bootstrap SEs/t-values written into $out
boot_results$model$out

## End(Not run)

Optimizer Diagnostics for Stochastic Frontier Fits

Description

Reports how the optimizer actually finished and how well the likelihood identifies the parameters: the convergence code and message, the eigenvalue spectrum and conditioning of the Hessian, the implied parameter correlations, and – where the objective was retained – the gradient at the reported optimum and a likelihood slice through each parameter.

Usage

sfa_diagnostics(object, ...)

## S3 method for class 'sfareg'
plot(x, which = 1:4, n_grid = 41, span = 0.25, ...)

## S3 method for class 'sfadiag'
print(x, ...)

Arguments

object, x

An object of class "sfareg" (for sfa_diagnostics and plot), or of class "sfadiag" (for print).

which

Which panels to draw: 1 the Hessian eigenvalue spectrum, 2 the parameter correlation matrix, 3 a likelihood slice per parameter, 4 the gradient. Panels that cannot be drawn are dropped silently; asking for only unavailable panels is an error rather than an empty plot.

n_grid

Number of points in each likelihood slice. Defaults to 41.

span

Half-width of each slice, as a fraction of max(|estimate|, 1). Defaults to 0.25.

...

Currently unused.

Details

Every stochastic frontier fit in this package runs a staged minimizer and stores optim()'s convergence code, message, evaluation counts and numerically differentiated Hessian. None of that was reported: a fit that stopped on the iteration cap printed exactly like a converged one. sfa_diagnostics() is the reporting side of that.

What the Hessian tells you. optim() returns the Hessian of the negative log-likelihood, so at a genuine interior optimum it is positive definite. A non-positive eigenvalue means the reported point is not a minimum in that direction. A very small one means the likelihood is nearly flat along it, which is what weak identification looks like numerically – $hessian$flat_direction names the parameters that load on that flattest axis, and they are usually the pair the data cannot separate.

Slices, not profiles. Panel 3 varies one parameter and holds the others at their estimates. A true profile would re-optimize the remaining parameters at every grid point, at the cost of one optimization per point. The slice is the right object for reading curvature at the optimum and for the simplest check there is: the estimate should sit at the peak of its own slice. If it does not, the optimizer stopped early.

Retaining the objective. The gradient and the slices need the likelihood itself, which an ordinary fit does not keep. Pass keep_objective = TRUE to sfm to store it. This is off by default because a closure carries its enclosing environment, so a fit saved with one serializes the estimation data alongside the results – on a 200-observation example the fitted object grows from roughly 38 KB to 1.7 MB.

The convergence code alone is not diagnostic, and the flags do not treat it as though it were. Across NHN, NE and NTN at n = 150, 500 and 1500, code 52 (“ABNORMAL_TERMINATION_IN_LNSRCH”) turns up routinely alongside a maximum relative gradient of about 10^{-6} and a positive definite Hessian: the staged minimizer had already found the optimum and the final L-BFGS-B stage simply could not take a step away from it. The same code on NTN at n = 150 came with a relative gradient of 5 \times 10^{7} and an indefinite Hessian, which is a genuine failure. What separates the two is the gradient and the Hessian, so the verdict is built from those and the code is reported as context:

benign

a line-search code (51/52) together with a small relative gradient and a positive definite Hessian. Reported as noise.

unverified

a line-search code with no retained objective, so there is no gradient to settle it. Reported as an open question rather than resolved in the fit's favour – refit with keep_objective = TRUE.

failure

anything else, including code 1 (the iteration limit), which is never excused: it means the search was still moving when it ran out of budget, whatever the curvature at the stopping point.

Flags are raised for a non-zero convergence code, a Hessian that is not positive definite or is worse than 10^{8} conditioned, any parameter pair correlated beyond |0.95|, and a large relative gradient. They are phrased as statements about what the fit means, not as generic warnings.

Value

sfa_diagnostics() returns an object of class "sfadiag": a list with model_name, call, pnames, estimates, convergence (code, meaning, message, counts, logLik), hessian (eigenvalues, condition, pos_def, flat_direction), correlation (matrix, worst_pair, worst_value); convergence additionally carries benign_nonzero and unverified_nonzero, gradient (gradient, max_abs, relative, max_rel; NULL without a retained objective), flags, and has_objective.

plot.sfareg() is called for its side effect and returns the "sfadiag" object invisibly. print.sfadiag() returns its argument invisibly.

Author(s)

David H. Bernstein

See Also

sfm, sfareg-methods, summary.sfareg

Examples


library(sfa)
d <- data_gen_cs(N = 300, rand = 3, sig_u = 1, sig_v = 0.3,
                 cons = 0.5, beta1 = 0.5, beta2 = 0.5, a = 5, mu = 0.5)

## Retaining the objective enables the gradient and the likelihood slices
f <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d, keep_objective = TRUE)
sfa_diagnostics(f)

## Four-panel diagnostic display
plot(f)

## Works without the objective too, on the Hessian alone
g <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d)
sfa_diagnostics(g)$hessian$condition


Extractor methods for "sfareg" objects

Description

Standard extractor methods for objects of class "sfareg", as returned by psfm, sfm, zsfm, and ttsfm. These let "sfareg" fits be used with the usual R modeling idioms (e.g. coef(fit), logLik(fit), AIC(fit), BIC(fit)) instead of reaching into the returned list by hand.

AIC() and BIC() are not defined explicitly for "sfareg" objects; they fall through to their default methods in the stats package, which only require a working logLik method (and, for BIC(), a working nobs method) to work correctly.

Usage

## S3 method for class 'sfareg'
coef(object, ...)
## S3 method for class 'sfareg'
vcov(object, ...)
## S3 method for class 'sfareg'
logLik(object, ...)
## S3 method for class 'sfareg'
nobs(object, ...)

Arguments

object

An object of class "sfareg".

...

Currently unused; included for S3 method consistency.

Details

coef.sfareg returns the named vector of estimated parameters (object$coefficients).

vcov.sfareg returns the estimated variance-covariance matrix. If the fit was estimated with optHessian = TRUE and the Hessian is invertible, this is solve(object$opt$hessian). Otherwise it falls back to a diagonal matrix built from the reported standard errors (object$std.errors), with a warning; if neither is available it returns a matrix of NAs with a warning.

logLik.sfareg returns the maximized log-likelihood as an object of class "logLik", with "df" and "nobs" attributes set so that AIC()/BIC() work automatically. Every estimator in this package minimizes the negative summed log-likelihood internally, so logLik = -object$opt$value. Exception: psfm()'s "GTRE_SEQ1" and "GTRE_SEQ2" models are estimated by a moment-based procedure and "SSFE" by fixed-effects (LSDV) regression, not by maximum likelihood; none of the three carries an $opt component, so calling logLik() (or, transitively, AIC()/BIC()) on those fits returns NA with a warning rather than erroring.

nobs.sfareg returns the number of observations used in estimation, recovered from object$data when the fitting function stored it, or by re-evaluating the data argument of the original call otherwise. Returns NA if neither is available.

Value

coef.sfareg returns a named numeric vector.

vcov.sfareg returns a numeric matrix.

logLik.sfareg returns an object of class "logLik".

nobs.sfareg returns a single integer (or NA).

See Also

psfm, sfm, zsfm, ttsfm, print.sfareg, summary.sfareg


sfm

Description

Implementation of the cross-sectional stochastic frontier model across an array of distributional assumptions for both v and u (user specified). For panel models, see the psfm() call.

Usage

sfm(formula, model_name = c("NHN", "NHN_Z", "NE", "NE_Z", "NR", "THT",
                       "NTN", "NG", "NNAK", "NU", "NGE", "NLN", "NW", "tHN"),
    data, maxit.bobyqa = 10000, maxit.nlminb = 500, maxit.psoptim = 1000,
    maxit.optim = 1000, REPORT = 1, trace = 2, pgtol = 0,
    start_val = FALSE, PSopt = FALSE, use.nlminb = "auto",
    use.bobyqa = "auto", optHessian = TRUE, inefdec = TRUE, upper = NA,
    Method = "L-BFGS-B", robust = c("mle", "mlqe", "psi", "mdpd"),
    c_mlqe = 0.20, eta = 0.01, alpha = 0.2, verbose = FALSE,
    Nsim = "auto", rand.psoptim = NULL, keep_objective = FALSE,
    estimator = c("mle", "cols"), cols_boot = 0, rand.cols = NULL)

Arguments

formula

a symbolic description for the model to be estimated

model_name

Which distributional pair to fit. The noise term v is normal except where stated, and the one-sided inefficiency term u follows:

"NHN"

half normal.

"NHN_Z"

half normal with heteroskedastic \sigma_u, parameterized by the determinants z given in the second part of the formula (y ~ x | z) as \sigma_u = \exp(z'\delta).

"NE"

exponential.

"NE_Z"

exponential with heteroskedastic \sigma_u, parameterized as for "NHN_Z".

"NTN"

truncated normal.

"NR"

Rayleigh. Note this is a distinct family from "NHN", not a reparameterization of it: the Rayleigh density carries an extra factor of u, and its standardized skewness is fixed at -0.631 against the half normal's -0.996.

"NU"

uniform.

"NG"

gamma. See ng_starts under ‘Value’ – this likelihood determines E[u] = \mu\sigma_u far better than either factor separately.

"NNAK"

Nakagami, with shape m. Nests "NR" at m = 1. The shape is weakly identified in practice, so \sigma_u and m are strongly correlated across samples.

"NGE"

generalized exponential.

"NLN"

lognormal. Estimated by simulated maximum likelihood over Halton draws; see Nsim.

"NW"

Weibull. Also simulated maximum likelihood.

"tHN"

half normal, with Student-t noise.

"THT"

half t, with Student-t noise.

Matching ignores case, so "nhn", "NHN" and "Nhn" are the same choice.

data

A data frame containing the variables named in formula.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine

maxit.optim

Maximum number of iterations for the optim optimization routine

REPORT

reporting parameter

trace

Integer. Tracing level passed through to the optimizer; larger values print more.

pgtol

Numeric. Projected-gradient tolerance passed to optim()'s "L-BFGS-B" method. 0 uses the optimizer's own default.

start_val

starting value (optional)

PSopt

use psoptim optimization routine (T or F)

maxit.nlminb

Maximum number of iterations for the nlminb optimization routine.

use.nlminb

"auto" (default), or a logical. Whether to run the stats::nlminb stage, which precedes bobyqa in the optimizer stack and accepts an analytic gradient where one is available. Under "auto" it runs only for the models where it has been verified to reach an equal-or-better optimum than the previous bobyqa-based path (NHN, NE, NTN, NU), where it is roughly 4-8 times faster at identical accuracy.

use.bobyqa

"auto" (default), or a logical. Whether to run the derivative-free bobyqa stage. Under "auto" it runs for every model except those handled by nlminb above, reproducing the previous behaviour exactly. It is kept on by default for the remaining models deliberately: for NR and NGE the nlminb path converges to a materially worse optimum, so accuracy, not speed, decides which stage runs.

optHessian

Logical. Should a numerically differentiated Hessian matrix be returned while using the optim routine? (for optim routine)

inefdec

Production or cost function

upper

Vector of upper values for the optim package.

Method

The method to be used for optim. See 'Details' within optim.

robust

One of "mle" (default), "mlqe", "psi", or "mdpd". Selects a robust divergence-based objective in place of ordinary maximum likelihood – see Details. Currently only implemented for model_name = "NHN"; sfm() stops with a clear error if a non-"mle" value is requested for any other model.

c_mlqe

Numeric, default 0.20. The MLqE power parameter (c in f^c; the corresponding q = 1-c), used only when robust = "mlqe".

eta

Numeric, default 0.01. The Psi-likelihood power parameter, used only when robust = "psi".

alpha

Numeric, default 0.2. The MDPD (minimum density power divergence) power parameter, used only when robust = "mdpd".

verbose

Logical. Print optimization progress messages? Default is FALSE.

Nsim

Number of simulation draws for the simulated-maximum-likelihood models ("NLN", "NW"); ignored by every closed-form model. "auto" (the default) uses max(400, ceiling(8*sqrt(n))). Simulated ML is consistent only if the draw count grows with the sample size – at a fixed Nsim the simulation bias does not vanish and every parameter except the frontier slopes converges to the wrong value. The previous fixed default of 100 did exactly that: at n = 3000 it returned (0.362, 0.821, 1.284, 0.328) against a truth of (0.3, 1.0, 1.5, 0.5) for "NW", while Nsim = 400 returned (0.298, 1.028, 1.558, 0.509). A numeric value below the automatic floor warns. "NLN" needs considerably more than the default: its lognormal tail makes the simulated integral converge slowly, and at n = 3000 its estimates were still moving toward the truth at Nsim = 6400. Raise it until they stop moving.

rand.psoptim

Integer. seed for replication of psoptim. Default to NULL.

keep_objective

Logical. Store the likelihood function on the fitted object, so that sfa_diagnostics can compute the gradient at the optimum and draw likelihood slices. Defaults to FALSE: a closure carries its enclosing environment, so a fit saved with one serializes the estimation data too – on a 200-observation example the object grows from about 38 KB to 1.7 MB. Everything else sfa_diagnostics() reports works without it.

estimator

"mle" (default) or "cols". "cols" selects corrected ordinary least squares instead of maximum likelihood: OLS slopes are kept as they are, the second and third central moments of the OLS residuals are inverted for the scale parameters, and the intercept is shifted up by the implied E[u]. Closed form, no optimizer, deterministic. Implemented for "NHN", "NE" and "NG"; other models error, because the moment inversion is distribution-specific. See ‘Details’.

cols_boot

Integer. Number of nonparametric bootstrap resamples used to obtain standard errors under estimator = "cols". Zero (the default) skips the bootstrap, in which case only the OLS slope standard errors are reported and the scale parameters and corrected intercept carry NA. Ignored when estimator = "mle".

rand.cols

Integer. Seed for the COLS bootstrap, for reproducibility. The caller's random-number stream is restored afterwards. Default NULL.

Details

The options include the Normal-Half Normal (NHN), Normal-exponential (NE), Student's t-Half t (THT), the Normal-Truncated Normal (NTN), the Normal-Uniform (NU), the Normal-Generalized Exponential (NGE), the Normal-Lognormal (NLN), the Normal-Weibull (NW) and the Student's t-half normal (tHN). NHN_Z and NE_Z are extensions for the NHN and NE models that allow for modeling the u-component of those models with determinants of inefficiency.

Corrected ordinary least squares (estimator = "cols"). The moment estimator of Olson, Schmidt and Waldman (1980, Journal of Econometrics). OLS is consistent for the slopes of a composed-error frontier whatever the one-sided distribution; only the intercept is biased, by E[u], because the composed error does not have zero mean. COLS therefore keeps the OLS slopes, inverts the central moments of the OLS residuals for the scale parameters, and shifts the intercept up by the implied E[u]. Writing m_2, m_3 for the second and third central moments of e = v - u: for "NHN", m_3 = \sigma_u^3\sqrt{2/\pi}(1-4/\pi) and m_2 = \sigma_v^2 + \sigma_u^2(1-2/\pi) with E[u] = \sigma_u\sqrt{2/\pi}; for "NE", m_3 = -2\sigma_u^3 and m_2 = \sigma_v^2+\sigma_u^2 with E[u] = \sigma_u; for "NG", from the third and fourth cumulants, \sigma_u = -k_4/(3k_3) and \mu = -k_3/(2\sigma_u^3) with E[u] = \mu\sigma_u.

No optimizer runs and the answer is deterministic, which makes COLS a natural robustness check against a maximum-likelihood fit that may have settled at a local optimum. It is less efficient than ML when the distributional assumption holds, and the "NG" inversion in particular leans on a fourth cumulant and is noticeably noisier than the other two at moderate sample sizes.

Wrong skew under COLS. A production frontier implies m_3 < 0. When a sample comes out with m_3 \ge 0 the moment equations have no admissible solution: sfm() warns, reports sigu as 0 with the whole residual variance assigned to sigv, and returns no efficiency predictions. Read that as no evidence of inefficiency in these data, not as an estimate of zero. This is the Type I failure of Olson, Schmidt and Waldman and is common in small samples; $wrong_skew and $residual_moments record it.

NR is not NHN. "NR" places a Rayleigh distribution on u, not a half-normal, and the two are separate families rather than reparameterizations of one another: the Rayleigh density carries an extra factor of u, and no rescaling removes it. The practical consequence is that the standardized skewness each contributes to the composed error is a different constant, -0.631 against the half-normal's -0.995, so a sample generated under one cannot be fitted by the other. \sigma_u is reported on the second-raw-moment convention E[u^2] = \sigma_u^2, matching "NHN" and making the two directly comparable; the Rayleigh scale is then \sigma_u/\sqrt{2}, with E[u] = \sigma_u\sqrt{\pi}/2 and Var(u) = (1-\pi/4)\sigma_u^2. Starting values come from inverting these moments, since the Rayleigh skewness is a constant and so m_3 identifies Var(u) outright. Use data_gen_cs()'s y_pcs_r column to test it, not y_pcs.

tHN is not THT. In THT (Tancredi 2002) a single scale mixture is shared by both error components, so the noise and the inefficiency are both Student's t with the same degrees of freedom, and that shared mixture is what makes the composed error a closed-form skew-t. In tHN the noise is Student's t and the inefficiency is half-normal, drawn independently, so the two components have different tail behaviour, there is no closed form, and the density is the convolution f(e) = \int_0^\infty f_v(e+u) f_u(u) du evaluated by Gauss-Legendre quadrature. tHN is the heavy-tailed-noise model, which is what makes it the natural parametric comparison for the density-power robust estimators (robust = "mlqe", "psi", "mdpd"); THT cannot play that role because its inefficiency term is heavy-tailed too. Parameters are reported as (sigv, sigu, nu), the conventional order, not THT's inverted (sigu, sigv, a).

Two properties of tHN are worth knowing before using it. First, the degrees of freedom are weakly identified: on data simulated from the model at n = 1000 with a true \nu = 5, the profile log-likelihood varies by only about 0.24 across \nu from 10 to 100 and peaks near 20, so a single reported \nu should not be taken at face value. Profile over a grid of fixed \nu and report the path. Because of that flat ridge sfm() always fits tHN from several widely separated starting values, keeps the best, and records the outcome in thn_starts; it warns when they reach different optima. Second, on real data \sigma_u can collapse onto zero, the heavy noise tail absorbing the entire one-sided component and leaving mean predicted efficiency near one. That is a property of the model rather than a numerical failure, so it is reported rather than suppressed: sfm() warns and sets thn_sigma_u_at_bound. Fits are also substantially slower than the closed-form models, since every likelihood evaluation is a quadrature whose node count grows with \sigma_u/\sigma_v.

"NU" assumes u \sim U(0,\theta) (Li 1996; Nguyen 2010), giving the closed-form composed density f(e) = \theta^{-1}[\Phi((e+\theta)/\sigma_v) - \Phi(e/\sigma_v)]. The reported theta is the upper bound of the inefficiency support, so predicted inefficiency is bounded by construction. "NGE" assumes a generalized exponential F(u) = (1-e^{-\lambda u})^2, whose composed density is a difference of two exponentially-tilted normal tail terms; it nests the exponential model and permits a non-monotonic inefficiency density. NGE reports sigu on the same scale as NE (the exponential mean, \lambda = 1/\sigma_u) so the two are directly comparable.

"NLN" (u lognormal, reported as sigu = sdlog and mu = meanlog) and "NW" (u Weibull, reported as sigu = scale and k = shape) have no closed-form composed density and are estimated by simulated maximum likelihood: f(e) = E_u[\phi((e+u)/\sigma_v)/\sigma_v] is averaged over Nsim Halton draws per observation. The draws are generated once and reused at every optimizer iteration, so the objective is deterministic and reproducible; increase Nsim to reduce simulation error.

Outputs include E[exp(-u)|e] given by exp_u_hat, following Battese and Coelli (1988, JoE), where appropriate. As of this version exp_u_hat is also returned for NE, NTN, NU, NGE, NLN and NW, together with the Jondrow et al. (1982) point predictor u_hat = E[u|e]; previously the exponential and truncated-normal models returned no efficiency prediction at all.

Note on default iteration counts: sfm()'s optimization defaults (thousands of iterations) are much higher than psfm()'s (100/10/10). This is intentional: sfm()'s likelihoods are closed-form (no simulation), so each objective-function evaluation is cheap and many default iterations cost little; several of psfm()'s models require simulated maximum likelihood over Halton draws, where each evaluation is far more expensive. See psfm's Details for more.

Robust divergence-based estimation (robust argument). For model_name = "NHN", sfm() can maximize a robust divergence objective instead of the ordinary log-likelihood, downweighting the influence of outlying observations:

All three reduce to "mle" as their power parameter approaches 0. Psi and MDPD produce identical parameter point estimates for the same power parameter (c = \eta = \alpha) – the two objectives are a positive rescaling of one another, \text{MDPD}(c) = (1+c)\cdot \text{Psi}(c), which does not change the argmax; both are exposed for compatibility with the corresponding paper's three-method terminology, not because they differ numerically. MLqE is a genuinely distinct estimator.

Because these are M-estimators rather than ordinary MLE, the naive Hessian-inverse standard error (optHessian = TRUE's usual output) is not statistically valid for them – it assumes the information-matrix equality, which these objectives do not satisfy in general. When robust != "mle", sfm() instead reports a sandwich-form standard error, A^{-1} B A^{-1}, where A is the Hessian of the summed robust objective (as already computed for the point estimate) and B = \sum_i g_i g_i' is built from the per-observation gradient of that objective (obtained via numerical differentiation, jacobian). This requires optHessian = TRUE; std.errors/t.values are NA otherwise, same as for ordinary MLE.

Formulas verified against an independently audited reference implementation (cross-checked by direct numerical integration and finite differences), including the sandwich SE's own verification (the c -> 0 limit against the ordinary MLE Hessian SE, and a Monte Carlo check against the empirical spread of repeated fits). Currently only implemented for model_name = "NHN" – other models will error clearly if robust != "mle" is requested.

Value

An object of class "sfareg" containing the following components:

out

A matrix with parameter estimates, standard errors, and t-values.

opt

A list containing the optimization results from the final optimization procedure.

total_time

The total computation time for model estimation.

start_v

The starting values used in the optimization.

model_name

The name of the stochastic frontier model estimated.

formula

The formula used in the model specification.

exp_u_hat

Predicted technical efficiency, E[exp(-u)|e]. Available for models: NHN, NHN_Z, NR, NG, NNAK, NE, NTN, NU, NGE, NLN, NW, THT and tHN.

u_hat

Predicted inefficiency, E[u|e] (Jondrow et al. 1982). Available for NE, NTN, NU, NGE, NLN, NW, THT and tHN.

thn_sigma_u_at_bound

Logical, tHN only. TRUE when the fitted \sigma_u collapsed onto zero, so the model reports essentially no inefficiency. Accompanied by a warning. Treat the efficiency scores as uninformative when this is set.

thn_starts

List, tHN only. Multi-start diagnostic: how many starting values were tried and converged, the log-likelihood reached from each, and n_distinct, the number of separate optima found. n_distinct > 1 means the likelihood surface is multimodal for these data and the single returned fit is only the best of those found.

estimator

"cols" for a corrected-OLS fit; absent for maximum-likelihood fits.

wrong_skew

Logical, estimator = "cols" only. TRUE when the OLS residuals were skewed the wrong way and the moment equations had no admissible solution.

residual_moments

The second, third and fourth central moments of the OLS residuals, estimator = "cols" only.

cols_boot_draws

The matrix of bootstrap replicates (cols_boot rows, one column per parameter) when cols_boot > 0; NULL otherwise.

ng_starts

Returned by NG only. A record of the starting-value search: how many candidates were built (n_tried), how many were polished before choosing (n_polished), the log-likelihood at each candidate (loglik_at_start), and the best value reached (best). NULL when start_val was supplied. The normal-gamma likelihood determines E[u] = \mu\sigma_u far better than it determines either factor – the weak identification Ritter and Simar (1997) describe – so the candidates hold E[u] at a moment estimate and sweep the shape along that ridge rather than searching across it. Inspect this if a normal-gamma fit looks implausible: widely differing log-likelihoods among the candidates indicate the surface is more awkward than usual for that data set.

sd_exp_u_hat

Standard deviation of exp(-u) given the residual, following Tancredi (2002, section 2.2). Returned by THT only. It is the quantity that distinguishes the skew-t frontier from the half-normal one: for a large positive residual the half-normal model concentrates exp(-u) at 1 and reports near-zero uncertainty, while the skew-t model treats the point as an outlier, widens this spread and does not push the efficiency estimate to 1.

med_u_hat

Predicted technical efficiency (median values). Available only for the NHN model.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE). Sandwich-form (not the ordinary Hessian-inverse form) when robust != "mle" – see Details.

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

call

The matched call.

robust

Only present for model_name = "NHN": the robust argument used to fit this model ("mle", "mlqe", "psi", or "mdpd").

robust_c

Only present for model_name = "NHN": the power parameter (c_mlqe/eta/alpha, whichever applies) used, or NA when robust = "mle".

Note

Standard errors require optHessian set to TRUE. When robust != "mle" (model_name = "NHN" only), a sandwich-form SE is reported instead of the ordinary Hessian-inverse one – see Details.

Author(s)

David H. Bernstein and Alexander Stead

See Also

psfm for panel models, zsfm for zero-inefficiency models, ttsfm for two-tier models, data_gen_cs for simulating data with known true parameters, and sfareg-methods for extracting results.

Examples


library(sfa)     

cs_data_trial   <- data_gen_cs(N= 1000, rand   = 1,  sig_u  = 0.3, sig_v  = 0.3, 
cons   = 0.5,       beta1  = 0.5,   beta2  = 0.5, a      = 4, mu     = 1)

cs.nhnz     <-  sfm(formula    = y_pcs_z ~ x1 +x2| z,    model_name = "NHN_Z",
                    data       = cs_data_trial,          PSopt      = TRUE)

## Robust divergence-based estimation (NHN only): downweights the influence
## of outlying observations relative to ordinary MLE. std.errors/t.values
## are NA for these methods -- see Details.
cs.nhn.mlqe <-  sfm(formula = y_pcs ~ x1 + x2, model_name = "NHN",
                    data = cs_data_trial, robust = "mlqe", c_mlqe = 0.20)
cs.nhn.psi  <-  sfm(formula = y_pcs ~ x1 + x2, model_name = "NHN",
                    data = cs_data_trial, robust = "psi", eta = 0.20)


Summary method for "sfareg" objects

Description

Summary of a stochastic frontier fit, as returned by sfm, psfm, zsfm or ttsfm.

Usage

## S3 method for class 'sfareg'
summary(object, ...)

Arguments

object

An object of class "sfareg", as returned by sfm, psfm, zsfm or ttsfm.

...

Additional arguments passed to other methods

Details

Prints the call, the model name, and the estimated parameters with standard errors and t-values, together with the log-likelihood and the number of observations where the estimator provides them. Which parameters appear depends on the model: several report the \lambda = \sigma_u/\sigma_v, \sigma = \sqrt{\sigma_u^2 + \sigma_v^2} reparameterization rather than the raw scale parameters.

Value

Invisibly returns its object argument unchanged, having printed the summary as a side effect.

Author(s)

David H. Bernstein

See Also

print.sfareg, sfareg-methods

Examples


library(sfa)     

cs_data_trial   <- data_gen_cs(N= 1000, rand   = 1,  sig_u  = 0.3, sig_v  = 0.3, 
cons   = 0.5,       beta1  = 0.5,   beta2  = 0.5, a      = 4, mu     = 1)

cs.nhnz     <-  sfm(formula    = y_pcs_z ~ x1 +x2| z,    model_name = "NHN_Z",
                    data       = cs_data_trial,          PSopt      = TRUE)
summary(cs.nhnz)                    


Two-Tier Stochastic Frontier Model

Description

Code to estimate the two-tier stochastic frontier model (2TSF), which decomposes a symmetric error term into two one-sided components (e.g. buyer/seller informational rents) in addition to the usual two-sided noise term.

Usage

ttsfm(formula, model_name = c("TTNE", "TTHN", "TTNLS"),
data, maxit.bobyqa = 80000, maxit.psoptim = 1000, maxit.optim = 1000,
REPORT = 1, trace = 0, pgtol = 0, start_val = FALSE, PSopt = FALSE,
optHessian = TRUE, inefdec = TRUE, upper = NA,
Method = "L-BFGS-B", logit = TRUE, verbose = FALSE, rand.psoptim = NULL)

Arguments

formula

a symbolic description for the model to be estimated. Use y ~ x | zu | zw to allow determinants zu of the first one-sided component and zw of the second.

model_name

model name for the estimation: "TTNE" for the Normal-Exponential-Exponential two-tier model, "TTHN" for the Normal-Half Normal-Half Normal two-tier model, or "TTNLS" for a nonlinear-least-squares two-tier estimator with no distributional assumption on u/w beyond their means. Matching ignores case, so "nhn", "NHN" and "Nhn" are the same choice.

data

A data frame containing the variables named in formula.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine

maxit.optim

Maximum number of iterations for the optim optimization routine

REPORT

reporting parameter

trace

Integer. Tracing level passed through to the optimizer; larger values print more.

pgtol

Numeric. Projected-gradient tolerance passed to optim()'s "L-BFGS-B" method. 0 uses the optimizer's own default.

start_val

starting value (optional)

PSopt

use psoptim optimization routine (T or F)

optHessian

Logical. Should a numerically differentiated Hessian matrix be returned while using the optim routine? (for optim routine)

inefdec

Production or cost function

upper

Vector of upper values for the optim package.

Method

The method to be used for optim. See 'Details' within optim.

logit

Choice of using logit function

verbose

Logical. Print optimization progress messages? Default is FALSE.

rand.psoptim

Integer. seed for replication of psoptim. Default to NULL.

Details

The two-tier stochastic frontier model (Polachek and Yoon, 1987) models the composed error as v + u - w, where v is a symmetric normal noise term and u, w are independent one-sided components. "TTNE" assumes u and w are exponential; "TTHN" assumes they are half-normal; "TTNLS" makes no distributional assumption and instead estimates \sigma_u/\sigma_w by nonlinear least squares via the scaling property. Determinants of u and w can be included via the second and third parts of a two-pipe formula, which parameterize \sigma_u and \sigma_w through a log link.

Like the other composed-error models in this package, all three two-tier likelihoods can have a genuine (non-error) boundary mode where a variance component is estimated near its numerical floor in some samples; spot-check fits before serious use. "TTHN"'s post-estimation information-deficiency metrics (the metrics component of the returned object) involve a large number of bivariate-normal-CDF evaluations and are more experimental than the parameter estimates themselves – NULL if the underlying calculation fails. For "TTNLS", the reported standard error for the \sigma_v-position parameter (present only for layout consistency with "TTNE"/"TTHN", unused in the NLS objective) is always NA by construction.

Note on default iteration counts: like sfm()/zsfm(), ttsfm()'s likelihoods are closed-form, so its optimization defaults are set much higher than psfm()'s (which uses simulated maximum likelihood for several models and defaults to far fewer iterations for speed). See psfm's Details for more.

Value

An object of class "sfareg" containing the following components:

out

A matrix with parameter estimates, standard errors, and t-values.

opt

A list containing the optimization results from the final optimization procedure.

total_time

The total computation time for model estimation.

start_v

The starting values used in the optimization.

model_name

The name of the two-tier stochastic frontier model estimated (TTNE, TTHN, or TTNLS).

formula

The formula used in the model specification.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE).

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

metrics

A list of information-deficiency measures (M1, M2, M5, M6, M7, M10; see Polachek and Yoon) evaluated at the final parameter estimates. NULL if the underlying calculation failed (see Details for the "TTHN" caveat).

call

The matched call.

Note

Standard errors require optHessian set to TRUE

Author(s)

David H. Bernstein and Christopher F. Parmeter

References

Polachek, S. and Yoon, B.J. (1987). A Two-Tiered Earnings Frontier Estimation of Employer and Employee Information in the Labor Market. The Review of Economics and Statistics.

Examples


library(sfa)

data <- data_gen_cs(N = 200, rand = 1, sig_u = 0.3, sig_v = 0.2,
                     cons = 2, beta1 = 0.5, beta2 = 0.3, a = 5, mu = 0)

eq <- y_pcs ~ x1 + x2

ttsfm(formula    = eq,
      model_name = "TTNE",
      data       = data)


Zero-Inefficiency Stochastic Frontier Models

Description

Fits the zero-inefficiency stochastic frontier model of Kumbhakar, Parmeter and Tsionas (2013), in which the sample is a mixture of fully efficient firms and firms drawn from an ordinary stochastic frontier.

Usage

zsfm(formula, model_name = c("ZISF", "ZISF_Z"), 
data, maxit.bobyqa = 10000,maxit.psoptim = 1000, maxit.optim = 1000, 
REPORT = 1, trace = 0, pgtol = 0,start_val = FALSE,PSopt = FALSE, 
optHessian = TRUE, inefdec = TRUE, upper = NA, 
Method = "L-BFGS-B",logit = TRUE,verbose=FALSE,rand.psoptim = NULL)

Arguments

formula

a symbolic description for the model to be estimated

model_name

Which mixture specification to fit. "ZISF" holds the probability of being fully efficient constant across the sample; "ZISF_Z" lets that probability depend on covariates supplied in the second part of the formula (y ~ x | z). See ‘Details’. Matching ignores case, so "nhn", "NHN" and "Nhn" are the same choice.

data

A data frame containing the variables named in formula.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine

maxit.optim

Maximum number of iterations for the optim optimization routine

REPORT

reporting parameter

trace

Integer. Tracing level passed through to the optimizer; larger values print more. 0 (the default) is silent.

pgtol

Numeric. Projected-gradient tolerance passed to optim()'s "L-BFGS-B" method. 0 uses the optimizer's own default.

start_val

starting value (optional)

PSopt

use psoptim optimization routine (T or F)

optHessian

Logical. Should a numerically differentiated Hessian matrix be returned while using the optim routine? (for optim routine)

inefdec

Production or cost function

upper

Vector of upper values for the optim package.

Method

The method to be used for optim. See 'Details' within optim.

logit

Logical, "ZISF_Z" only. Link mapping the linear predictor z'\gamma to the probability of full efficiency. TRUE (the default) uses the logistic link \Lambda(z'\gamma), which covers (0,1). FALSE uses \Phi(z'\gamma)/(1 + \Phi(z'\gamma)), which is bounded above by 0.5: under that link no observation can be assigned a probability of full efficiency exceeding one half, whatever the data. This is not the probit analogue of the default and is retained only for backward compatibility; prefer logit = TRUE unless you specifically want that restriction.

verbose

Logical. Print optimization progress messages? Default is FALSE.

rand.psoptim

Integer. seed for replication of psoptim. Default to NULL.

Details

The zero-inefficiency stochastic frontier model treats the sample as a mixture of two regimes: with probability p a firm sits exactly on the frontier (u = 0, noise only), and with probability 1 - p it is drawn from an ordinary normal-half normal frontier. It is the natural specification when some fraction of the sample is thought to be fully efficient, which an ordinary frontier cannot represent because it forces u > 0 almost surely.

Under "ZISF" the mixing probability is a single constant, parameterized as p = \exp(-|\gamma|). That parameterization is exactly symmetric in \gamma: +\gamma and -\gamma give identical fits and the optimizer may return either, so the sign of the reported gamma carries no meaning. Under "ZISF_Z" the probability varies with covariates z through the link chosen by logit.

post.prob reports the posterior probability that each observation belongs to the fully efficient regime, given its residual; jlms reports the Jondrow et al. (1982) inefficiency prediction.

Note on default iteration counts: like sfm(), zsfm()'s likelihood is closed-form, so its optimization defaults (thousands of iterations) are set much higher than psfm()'s (which uses simulated maximum likelihood for several models and defaults to far fewer iterations for speed). See psfm's Details for more.

Value

An object of class "sfareg" containing the following components:

out

A matrix with parameter estimates, standard errors, and t-values.

opt

A list containing the optimization results from the final optimization procedure.

total_time

The total computation time for model estimation.

start_v

The starting values used in the optimization.

model_name

The name of the zero-inefficiency stochastic frontier model estimated ("ZISF" or "ZISF_Z").

formula

The formula used in the model specification.

jlms

Predicted technical efficiency using the Jondrow et al. (1982) conditional mean estimator (JLMS).

post.prob

Posterior probabilities of being fully efficient.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE).

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

call

The matched call.

Note

Standard errors require optHessian set to TRUE

Author(s)

Christopher F. Parmeter and David H. Bernstein

References

Kumbhakar, S.C., Parmeter, C.F. and Tsionas, E.G. (2013) 'A zero inefficiency stochastic frontier model', Journal of Econometrics, 172(1), pp. 66-76. doi:10.1016/j.jeconom.2012.08.021.

See Also

sfm, psfm, ttsfm, sfareg-methods, panel89

Examples


library(sfa)  

eqz     <- y ~ q1 + q2 + q3 + q4 + q5 + w1 + w2 + w3 + w4 | z

data(panel89)

zsfm(formula    = eqz,
     model_name = "ZISF_Z",
     data       = panel89,
     logit      = TRUE)