Average Matrix Profile

[yogeshhk]

I am new to Matrix Profile (and Stumpy). I am not sure if the following idea has already been evaluated or not, but putting it here for comments, anyways.

• time-series 'ts' and segment length 'm' are arguments for generating Matrix Profile 'mp'
  • 'mp' is short at the end by 'm-1' as there is no subsequence to compute distance 'd'.
  • 'mp' shape is dependent on the choice of 'm'
  • Idea : Need to evaluate effectiveness/appropriateness of:
    • Can we divide 'd' by length of 'm' while generating 'mp'? lets call it average mp, ie 'amp'
    • Will this make 'mp' agnostic of 'm'?
    • Even the end segment of 'mp', can it be filled as at that portion?
    • There, 'm' will be 'm', 'm-1','m-2',...,3,2,1
    • E.g. take both subsequences of shorter lengths while computing 'd'
  • Experiment:
    • For a 'ts', generate 'mp's for various 'm's. They will look different and shorter at the ends.
    • For each 'mp' compute its 'amp' also
    • See if all 'amp's look the same and are full at the end.

[seanlaw]

@yogeshhk Welcome to the STUMPY community and thank you for sharing this interesting thought experiment.

So, I understand the computation of amp in your case and I have a few questions. What is the goal of computing amp? I'm missing the meaning or point of comparing different amp from different lengths. What is the goal or use case? What is the value that you get from using amp (aside from being able to compare mp computed from different lengths)?

[yogeshhk]

'm' is the hyperparameter needed to compute Matrix Profile 'mp' and it is provided based on the domain knowledge.

Was thinking if we can get rid off this choice, meaning you don't need to provide 'm' at all. That's when the idea came that, if we can divide the computed distance 'd' by length of 'm', then the distance profile becomes 'm' agnostic or indifferent.

Another effect of this could be, avoiding the truncation that happens at the end of 'mp', where it is shorter than the time-series.

At the end we can use remaining 'm' (ie m, m-1, ...2,1) for computing distance profile thus making it full ie of same length as original time-series.

Experiments suggested as to see if 'amp's generated from different length 'm's, look similar or not, thus proving if 'amp' is 'm' indifferent.

[seanlaw]

From my limited experience, the choice of m seems to matter a bit and so it would be an interesting experiment to test and learn from. Again, I haven't put much thought into this and I think one has to think about scenarios where there are multiple different motifs (with different lengths or window sizes) that exist in a single time series. This recent paper by Jessica Lin describes one such scenario (see Introduction and Fig 1).

Of course, the pragmatic thing to do would be to test it out. @yogeshhk I encourage you to explore this as I would be curious as to what you find!

[yogeshhk]

Thanks @seanlaw for the paper suggestion. I will go through it and explore the idea as well.