Merging several matrix profile outputs

[vvrahul11]

Hello,

Thank you for a wonderful package. I enjoy using it. I have two use case questions on the matrix profile.

I would like to ask you if there is a way to merge the outputs of two matrix profile outputs or merge the output of a matrix profile and an unsupervised algorithm's anomaly score (ex: One class svm, Isolation Forest)?

Could you tell me a way to do it Intuitively? I explored calculating the "mean of two matrix profiles" or "mean of a matrix profile and anomaly score of an unsupervised algorithm" after normalizing both within [0, 1] bounds. However, it didn't look like the right approach. (Also I do not want to use the multi-dimensional approach).

I would appreciate any suggestions. Thank you.

[seanlaw]

@vvrahul11 Welcome to the STUMPY community and it's great to hear that you are liking STUMPY! Regarding your question:

I would like to ask you if there is a way to merge the outputs of two matrix profile outputs or merge the output of a matrix profile and an unsupervised algorithm's anomaly score (ex: One class svm, Isolation Forest)?

I think that there are some key questions that one should first ask:

1. What does the output from merging two matrix profiles mean?
2. What do you hope to accomplish by merging unrelated inputs? (Sure, you get "dimensionality reduction" but that is not a "goal" per se)
3. Do the input matrix profiles have the same length (so that the merged output has the same length)?
4. Do the inputs come from the same source but at different times?

As you've already pointed out below, through merging inputs, you lose intuition. In other words, you may be convoluting the signal. But, in the end, it depends on what your ultimate goal is.

Could you tell me a way to do it Intuitively? I explored calculating the "mean of two matrix profiles" or "mean of a matrix profile and anomaly score of an unsupervised algorithm" after normalizing both within [0, 1] bounds. However, it didn't look like the right approach. (Also I do not want to use the multi-dimensional approach).

Can you explain what "the right approach" looks like? Even with two matrix profiles, I don't fully know how to interpret taking the mean of two matrix profiles. Unfortunately, I may not be much help here 😞

[vvrahul11]

@seanlaw Thank you for the prompt response. Please find below the answers to your questions.

1. What does the output from merging two matrix profiles mean?

2. What do you hope to accomplish by merging unrelated inputs? (Sure, you get "dimensionality reduction" but that is not a "goal" per se)
   When an unsupervised algorithm detects anomalies inside a multivariate time series, it only labels you a point/points are anomalous "across all points in a time series at a given point" and not which time series. Instead, can I consider each time series as a univariate time series, apply matrix profile or an unsupervised algorithm and merge back the result? This gives interpretability.

3. Do the input matrix profiles have the same length (so that the merged output has the same length)?
   Yes, they will have the same length

4. Do the inputs come from the same source but at different times?
   No, they come from different sources.

[seanlaw]

To encourage others in the community to reply, it might be worth posting a simple reproducible code example so that others can see what you've tried and what you are trying to accomplish.

Unfortunately, I do not have enough experience with this to be able to provide any meaningful suggestions.

[vvrahul11]

Thank you @seanlaw. I will do it from next time.