Skip to content

Commit

Permalink
minor doc edits
Browse files Browse the repository at this point in the history
  • Loading branch information
jgabry committed Mar 11, 2022
1 parent fb3893f commit 79ef4cb
Show file tree
Hide file tree
Showing 8 changed files with 88 additions and 85 deletions.
37 changes: 18 additions & 19 deletions R/loo.R
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,16 @@
#' with MCMC. If MCMC draws are used and `r_eff` is not provided then
#' the reported PSIS effective sample sizes and Monte Carlo error estimates
#' will be over-optimistic. If the posterior draws are independent then
#' `r_eff=1` and can be omitted. See the [relative_eff()]
#' helper functions for computing `r_eff`.
#'
#' `r_eff=1` and can be omitted. The warning message thrown when `r_eff` is
#' not specified can be disabled by setting `r_eff` to `NA`. See the
#' [relative_eff()] helper functions for computing `r_eff`.
#' @param save_psis Should the `"psis"` object created internally by `loo()` be
#' saved in the returned object? The `loo()` function calls [psis()]
#' internally but by default discards the (potentially large) `"psis"` object
#' after using it to compute the LOO-CV summaries. Setting `save_psis=TRUE`
#' will add a `psis_object` component to the list returned by `loo`. Currently
#' this is only needed if you plan to use the [E_loo()] function to compute
#' weighted expectations after running `loo`.
#' will add a `psis_object` component to the list returned by `loo`.
#' Currently this is only needed if you plan to use the [E_loo()] function to
#' compute weighted expectations after running `loo`.
#' @template cores
#' @template is_method
#'
Expand All @@ -47,32 +47,31 @@
#' `c("psis_loo", "loo")` and components:
#' \describe{
#' \item{`estimates`}{
#' A matrix with two columns (`Estimate`, `SE`) and three rows
#' (`elpd_loo`, `p_loo`, `looic`). This
#' contains point estimates and standard errors of the expected log pointwise
#' predictive density (`elpd_loo`), the effective number of parameters
#' (`p_loo`) and the LOO information criterion `looic` (which is
#' just `-2 * elpd_loo`, i.e., converted to deviance scale).
#' A matrix with two columns (`Estimate`, `SE`) and three rows (`elpd_loo`,
#' `p_loo`, `looic`). This contains point estimates and standard errors of the
#' expected log pointwise predictive density ([`elpd_loo`][loo-glossary]), the
#' effective number of parameters ([`p_loo`][loo-glossary]) and the LOO
#' information criterion `looic` (which is just `-2 * elpd_loo`, i.e.,
#' converted to deviance scale).
#' }
#'
#' \item{`pointwise`}{
#' A matrix with five columns (and number of rows equal to the number of
#' observations) containing the pointwise contributions of the measures
#' (`elpd_loo`, `mcse_elpd_loo`, `p_loo`, `looic`, `influence_pareto_k`).
#' in addition to the three measures in \code{estimates}, we also report
#' pointwise values of the Monte Carlo standard error of \code{elpd_loo}
#' (\code{mcse_elpd_loo}), and statistics describing the influence of
#' each observation on the posterior distribution (\code{influence_pareto_k}).
#' in addition to the three measures in `estimates`, we also report
#' pointwise values of the Monte Carlo standard error of [`elpd_loo`][loo-glossary]
#' ([`mcse_elpd_loo`][loo-glossary]), and statistics describing the influence of
#' each observation on the posterior distribution (`influence_pareto_k`).
#' These are the estimates of the shape parameter \eqn{k} of the
#' generalized Pareto fit to the importance ratios for each leave-one-out
#' distribution. See the [pareto-k-diagnostic] page for details.
#' distribution (see the [pareto-k-diagnostic] page for details).
#' }
#'
#'
#' \item{`diagnostics`}{
#' A named list containing two vectors:
#' * `pareto_k`: Importance sampling reliability diagnostics. By default,
#' these are equal to the \code{influence_pareto_k} in \code{pointwise}.
#' these are equal to the `influence_pareto_k` in `pointwise`.
#' Some algorithms can improve importance sampling reliability and
#' modify these diagnostics. See the [pareto-k-diagnostic] page for details.
#' * `n_eff`: PSIS effective sample size estimates.
Expand Down
40 changes: 20 additions & 20 deletions R/loo_compare.R
Original file line number Diff line number Diff line change
Expand Up @@ -17,28 +17,28 @@
#'
#' @details
#' When comparing two fitted models, we can estimate the difference in their
#' expected predictive accuracy by the difference in `elpd_loo` or
#' `elpd_waic` (or multiplied by \eqn{-2}, if desired, to be on the
#' deviance scale).
#' expected predictive accuracy by the difference in
#' [`elpd_loo`][loo-glossary] or `elpd_waic` (or multiplied by \eqn{-2}, if
#' desired, to be on the deviance scale).
#'
#' When using `loo_compare()`, the returned matrix will have one row per
#' model and several columns of estimates. The values in the `elpd_diff`
#' and `se_diff` columns of the returned matrix are computed by making
#' pairwise comparisons between each model and the model with the largest ELPD
#' (the model in the first row). For this reason the `elpd_diff` column
#' will always have the value `0` in the first row (i.e., the difference
#' between the preferred model and itself) and negative values in subsequent
#' rows for the remaining models.
#' When using `loo_compare()`, the returned matrix will have one row per model
#' and several columns of estimates. The values in the
#' [`elpd_diff`][loo-glossary] and [`se_diff`][loo-glossary] columns of the
#' returned matrix are computed by making pairwise comparisons between each
#' model and the model with the largest ELPD (the model in the first row). For
#' this reason the `elpd_diff` column will always have the value `0` in the
#' first row (i.e., the difference between the preferred model and itself) and
#' negative values in subsequent rows for the remaining models.
#'
#' To compute the standard error of the difference in ELPD --- which should
#' not be expected to equal the difference of the standard errors --- we use a
#' paired estimate to take advantage of the fact that the same set of \eqn{N}
#' data points was used to fit both models. These calculations should be most
#' useful when \eqn{N} is large, because then non-normality of the
#' distribution is not such an issue when estimating the uncertainty in these
#' sums. These standard errors, for all their flaws, should give a better
#' sense of uncertainty than what is obtained using the current standard
#' approach of comparing differences of deviances to a Chi-squared
#' To compute the standard error of the difference in [ELPD][loo-glossary] ---
#' which should not be expected to equal the difference of the standard errors
#' --- we use a paired estimate to take advantage of the fact that the same
#' set of \eqn{N} data points was used to fit both models. These calculations
#' should be most useful when \eqn{N} is large, because then non-normality of
#' the distribution is not such an issue when estimating the uncertainty in
#' these sums. These standard errors, for all their flaws, should give a
#' better sense of uncertainty than what is obtained using the current
#' standard approach of comparing differences of deviances to a Chi-squared
#' distribution, a practice derived for Gaussian linear models or
#' asymptotically, and which only applies to nested models in any case.
#'
Expand Down
29 changes: 15 additions & 14 deletions man/loo.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 20 additions & 20 deletions man/loo_compare.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions man/loo_subsample.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 6 additions & 5 deletions man/parallel_psis_list.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions man/psis_approximate_posterior.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions man/update.psis_loo_ss.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 79ef4cb

Please sign in to comment.