Output format returned by postprob(lcModelKML, ...) was not valid

[Pubuditha]

Hi there,

I recently learned about the latrend package and started using it for clustering a big longitudinal data set.
I followed the steps to download the packages and also run the example code with the data set that comes in the package. Everything works well for that data set. However, when I perform the clustering for my own dataset, kml fitting works fine without any errors but 50 warnings as below (line below is repeated for 50 times with either Inf or -Inf at the end):

In FUN(newX[, i], ...) : no non-missing arguments to min; returning Inf

Not sure what this means. So no idea how I can correct this.

Then when I do "summary(model)" it gives me the following error:

Error in postprob(object, ...) :
lcModelKML implementation error: Output format returned by postprob(lcModelKML, ...) was not valid: nrow(pp) not equal to nIds(object)

I'm assuming the error is caused by the warnings I get but I'm very new to using this package and not sure how to correct this.
If someone can help me with this that will be great!

Thank you in advance!

[niekdt]

Hello,

Thanks for the detailed write-up. Are you using latrend version 1.3?

I'm unable to find where the warnings might be coming from in the package code, could you set options(warn = 2) and rerun the code? The warning will become an error, and then I'd like to see the output from traceback().

The error in postprob() is a consequence of the internal KmL model being different from what latrend is expecting. I've added strict checks to the lcModel functions such as postprob() to prevent (silent) faulty output.

Most likely, the trigger for the issue you are experiencing is that your dataset contains missing values. I've improved the robustness of latrend to missing data based on previous reports this year, but it would seem you may have found another special case that needs fixing.

[Pubuditha]

Thank you for the quick reply @niekdt

Yes I am using latrend version 1.3

I set options(warn = 2) and rerun the code. Then get the error below with summary(model):

Error: nrow(pred) not equal to .ndobs(model)

Output of traceback() is as below:

17: stop(assertError(attr(res, "msg")))
16: assert_that(is.numeric(pred), ncol(pred) == nClusters(model),
nrow(pred) == .ndobs(model), has_colnames(pred), is_valid_cluster_name(colnames(pred),
model = model), all(clusterNames(model) %in% colnames(pred)))
15: withCallingHandlers(expr, warning = function(w) if (inherits(w,
classes)) tryInvokeRestart("muffleWarning"))
14: suppressWarnings(assert_that(is.numeric(pred), ncol(pred) ==
nClusters(model), nrow(pred) == .ndobs(model), has_colnames(pred),
is_valid_cluster_name(colnames(pred), model = model), all(clusterNames(model) %in%
colnames(pred))))
13: transformFitted(newpred, model, clusters)
12: transformFitted(newpred, model, clusters)
11: transformFitted(., model = object, clusters = clusters)
10: transformFitted(., model = object, clusters = clusters)
9: predict(object, newdata = newdata) %>% transformFitted(model = object,
clusters = clusters)
8: fitted.lcApproxModel(object, clusters = clusters, ...)
7: fitted(object, clusters = clusters, ...)
6: residuals.lcModel(object)
5: residuals(object)
4: withCallingHandlers(expr, warning = function(w) if (inherits(w,
classes)) tryInvokeRestart("muffleWarning"))
3: suppressWarnings({
res = residuals(object)
if (length(res) == 0) {
res = as.numeric(NA)
}
props = tryCatch(clusterProportions(object), error = function(...) rep(NaN,
nClusters(object)))
ss = new("lcSummary", method = getLcMethod(object), name = getName(object),
nClusters = nClusters(object), nObs = ifelse(is.null(nobs(object)),
0L, nobs(object)), id = idVariable(object), coefficients = coef(object),
residuals = res, clusterNames = clusterNames(object),
trajectoryAssignments = trajectoryAssignments(object),
clusterSizes = clusterSizes(object), clusterProportions = props)
})
2: summary.lcModel(model_test2)
1: summary(model_test2)

I also thought the problem is that my data set contains missing values. So I checked the data set multiple times and removed the rows with missing values before calling it in the code and I'm 100% sure there are no missing values in the data set.

One question though, let's say I have longitudinal data for 10 students. Do I have to have the same number of timepoints for every student? If that's a requirement for this method, then that's the issue. For an example, I have students who has data for 5 years (5 timepoints) and some who might have data for 10 years or 3 years, so they don't have the same number of repeated measures.

[niekdt]

In your original post you mentioned 50 warnings during fitting. Could you rerun that with options(warn = 2) as well? Thanks

KmL indeed requires trajectories to be time-aligned and of equal length. The KmL package provides some imputation options for absent trajectory observations, but it could be that the package is removing some nearly-empty trajectories during fitting, resulting in a mismatch with what the latrend package is expecting. I will look into how to detect this, because ideally latrend should be able to handle whatever the kml package spits out.

Regarding your analysis:
If there's censoring or time-to-event is of interest, I would recommend looking into joint modeling using, for example, the lcmm package (see the Jointlcmm function). This is outside of the scope of the latrend package, for now.

Otherwise, I recommend group-based trajectory modeling (GBTM). It's probably anyway the preferred starting point for exploratory purposes, in this context. It can be regarded as a parametric version of KmL, with similar results under the condition that the cluster trajectory formulas can describe the trends properly. This method is supported by the latrend package, see lcMethodLcmmGBTM, which uses the lcmm package.

If you just want to know how students progress during their study, regardless of total study years, you could consider normalizing the time variable, such that all students start at 0 and end at 1. Whether that makes any sense at all completely depends on your data assumptions and aims. You can then use GBTM to cluster the data.

[niekdt]

I was able to reproduce the errors you were getting when using lcModelKML. These are now fixed (see #111) in the latest commit on master.

[Pubuditha]

When I rerun that with the options(warn=2), I get the following:

---

• Longitudinal clustering using: longitudinal k-means (KML)

---

Method arguments:
time: getOption("latrend.time")
id: getOption("latrend.id")
nClusters: 4
nbRedrawing: 20
maxIt: 200
imputationMethod:"copyMean"
distanceName: "euclidean"
power: 2
distance: function() {}
centerMethod: meanNA
startingCond: "nearlyAll"
nbCriterion: 1000
scale: TRUE
response: "STSDMT"

Checking and transforming the training data format.
Preparing the training data for fitting...
Error in matrix(dt[[response]], byrow = TRUE, nrow = length(ids), ncol = length(times)) :
(converted from warning) data length [112000] is not a sub-multiple or multiple of the number of rows [489]

For this, I have used a subgroup of data where I have 3 timepoints for each student. But the time points do not align for the whole data set. For some students the gap between the two time points are 2 years and for some it's 5 years (for an example). After reading your reply, I think this could be the issue for the errors I am getting. I will try the methods that you have suggested. In the mean time, if you find a fix for the problem please do let me know.
I can't normalise the time.

Thank you!

[Pubuditha]

Sorry @niekdt there was some error and my reply had not been posted for some reason. Thank you for the fix. I'll see whether it's working without a problem now.

Thank you for suggesting few other methods. I've being testing GTBM with my data.

[Pubuditha]

Hi @niekdt, I just updated the latrend installation and rerun the code. But still getting the same error msgs as before. See below:

---

      - Longitudinal clustering using: longitudinal k-means (KML)
      -----------------------------------------------------------------
      Method arguments:
       time:           getOption("latrend.time")
       id:             getOption("latrend.id")
       nClusters:      4
       nbRedrawing:    20
       maxIt:          200
       imputationMethod:"copyMean"
       distanceName:   "euclidean"
       power:          2
       distance:       function() {}
       centerMethod:   meanNA
       startingCond:   "nearlyAll"
       nbCriterion:    1000
       scale:          TRUE
       response:       "STSDMT"
      -----------------------------------------------------------------
      Checking and transforming the training data format.
      Preparing the training data for fitting...
      Fitting the method...

~ Fast KmL ~
********************S
Done fitting the method (7 secs)
-----------------------------------------------------------------
There were 50 or more warnings (use warnings() to see the first 50)

summary(model_test2)
Error in postprob(object, ...) :
lcModelKML implementation error: Output format returned by postprob(lcModelKML, ...) was not valid: nrow(pp) not equal to nIds(object)

Not sure whether I did the update correctly to reflect your fixes to the issue. What I did was this:

wnloading GitHub repo philips-software/latrend@v1.4.0
These packages have more recent versions available.
It is recommended to update all of them.
Which would you like to update?

1: All
2: CRAN packages only
3: None
4: tinytex (0.39 -> 0.40) [CRAN]

Enter one or more numbers, or an empty line to skip updates: 1

Your help is much appreciated.

[niekdt]

Thanks for the update. The other packages do not need to be updated, as the attempted fix is with the latrend package. Did you restart the R session after updating?

Unfortunately, I'm unable to reproduce the error. It does suggest that there is another data scenario for which the kml package is excluding one or more trajectories, resulting in the latrend error you are seeing. In order to better understand the source of this error, I've added an additional check today in fbb06eb. Could you install the latest commit of latrend via remotes::install_github('philips-software/latrend'), restart R, and rerun your code? You should be getting a different error this time.

I appreciate your help!

[Pubuditha]

Hi @niekdt,

I did install the latest commit and rerun the code. This time still got the 50 warnings but no errors.

~ Fast KmL ~
********************S
Done fitting the method (6.4 secs)

There were 50 or more warnings (use warnings() to see the first 50)

Then when I check the summary:

summary(model_test2)
Called from: transformPredict(pred = preddata, model = object, newdata = newdata)
Browse[1]>

And the p[lotting also works without an error.

[niekdt]

Thanks @Pubuditha, good to hear you are not getting any errors anymore.

The 50+ warnings about "In FUN(newX[, i], ...) : no non-missing arguments to min; returning Inf" are coming from the kml package. I have no clue what aspect of your dataset would be triggering that. If you ever find out, please let me know so I can include it as a test case in the latrend package.

The Browse[1] part of the console output you posted concerned me, so I checked the latrend source code and it turns out there was a browser() statement in an error condition flow. I've now described and fixed this in #120.

[Alik-V]

Hey @niekdt, thanks for this fantastic package and its ongoing development!

I decided to add a comment in this thread as I think I have a very similar problem discussed here but with the GCKM method:

Error in h(simpleError(msg, call)) :
error in evaluating the argument 'object' in selecting a method for function 'show': lcModelLMKM implementation error: Output format returned by postprob(lcModelLMKM, ...) was not valid: nrow(pp) not equal to nIds(object)

I have run KML prior to GCKM and got similar warnings as above but no errors. The GCKM was run through latrendBatch which resulted with no errors, however when trying to call the summary I get the error in the quote above.

My patient data also contains different time lengths that were censored due to deaths. Would the recommendation be the same here - use jointlcmm for this type of data and analysis?

The big problem with that is runtimes and convergence for my dataset. I have ~350k patients with ~5mil records and the lcmm methods are certainly struggling there, hence I was trying to use the methods outlined in this handy summary in your documentation.

Any advice would be greatly appreciated! :)

[niekdt]

Hi Alik,

You're welcome! I appreciate the kind words. 🙂

Since you have a dataset with informative censoring/drop-out, you'll definitely want to account for that. Indeed, jointlcmm is the right choice for such a use case. Unfortunately, jointlcmm is not supported by latrend, only lcmm::hlme. Alternatively, you can do a conditional analysis ("given that patients survived up to day N, what do their trajectories look like over days [1, N]?") Here, the methods supported by latrend are applicable.

There's little benefit to running the analysis with such a huge number of patients, which you'll definitely want to reduce because of the typically quadratic scaling with the number of clusters, as you have probably noticed. I use the guideline that there should be at least a 100 patients for every cluster, and that each resulting cluster has enough patients to represent it (depends on the results). In any case, you can easily identify a small number of clusters with just 2000 patients. Use bootstrap sampling (see latrendBoot) to (visually) assess the stability of the cluster results to check if you need a bigger sample size.

Regarding the LMKM error; this should be addressed as it may indicate the model is misreporting results. Could you check your data for trajectories without observations/valid data?

[Alik-V]

Thank you for the quick and detailed response!

I think I needed to add that the key question is that I want to find some "hidden" clusters in the real world data healthcare data. As a baseline, I am very hesitant to narrow the cohort further since it would defeat the purpose of looking for that cluster of patients since I might remove interesting ones based on bias or else.

jointlcmm is the right choice for such a use case

The biggest problem is the runtime and convergence. I don't think it is feasible to use my full dataset for jointlcmm.

("given that patients survived up to day N, what do their trajectories look like over days [1, N]?")

That could be an interesting idea but I think it will take away a part of the research idea in my case unfortunately.

In any case, you can easily identify a small number of clusters with just 2000 patients. Use bootstrap sampling (see latrendBoot) to (visually) assess the stability of the cluster results to check if you need a bigger sample size.

That would be ideal, however narrowing down the patient number this much will likely remove the subset I am really interested in my research.

Could you check your data for trajectories without observations/valid data?

I have had a better look and I have patients with only 1 datapoint total. I will re-try GCKM without those.

[Alik-V]

Additionally, if you don't mind me asking - what's the main difference between GCKM and hlme?

It seemed to me that both are parametric, but the runtimes seem to differ quite a bit. I understand that hlme is written in Fortran whereas underlying GCKM method uses RcppEigen, but not being a math guy - it has been difficult to fully wrap my head around these

[niekdt]

If you're looking for distinctive outlier clusters, then even with a relatively low sample size you may be able to find them, because they will greatly increase the fit of the model to the data. Growth mixture modeling is particularly useful for this because it puts together all patients with a similar shape, so there's more "room" to represent clusters with distinctive shapes.

Because of the huge sample size and the interest in specific small clusters, a hierarchical approach might be more efficient.
This allows you to break up your dataset into smaller chunks with similar characteristics, and continue the analysis on the subset(s) of interest. E.g., filter out all patients with a constant trend may reduce your sample size by half. In latrend, a quick way to start could be through stratification based on clinical knowledge (see latrend::lcMethodStratify)

latrend does not support hierarchical output so will be of limited help here.

For inspiration, see latent growth trees or distance-based methods (dtwclust), as hierarchical clustering is based on pair/group-wise distances.

Additionally, if you don't mind me asking - what's the main difference between GCKM and hlme?

It seemed to me that both are parametric, but the runtimes seem to differ quite a bit. I understand that hlme is written in Fortran whereas underlying GCKM method uses RcppEigen, but not being a math guy - it has been difficult to fully wrap my head around these

GCKM uses a feature-based approach, where the trajectory coefficients (random effects) are estimated per trajectory, and then clustered using k-means. For large datasets with a sufficient number of observations per trajectory, this works very well and scales much better as you have noticed. Hlme estimates a mixture of regression models (used for GBTM, GMM) in one go, where individual trajectories are not estimated, making it much more robust under low sample size and low number of observation conditions.

GCKM does not account for censoring, so whether it is applicable in for your analysis depends on how informative the censoring is.

[Alik-V]

Great explanations and suggestions - thank you for taking your time to write these up :)