Computing the Full Distance Matrix

[adityabhandwalkar]

Hi

In the matrix profile what we get is the minimum dist for each sliding window right,

• • • 6.9 * * *

• • •  1.3 * 
      

like this and we get a distn matrix with minimum distances what if I would like to get

• • • 6.9 * * * <---- this entire window with all the distances in it for the distance matrix, basically an entire distance matrix from the given sequence.

[seanlaw]

@adityabhandwalkar This is not possible with STUMPY due to the high cost of storing the full distance matrix and is not advised.

[adityabhandwalkar]

[(https://github.com//discussions/608#discussioncomment-2797226)] In this post , distance matrix is calculated like this

So can we use this code and try to get second best nearest neighbor which has minimum distance in each window and to get the sum of each window on stack.

8.9  4.5  3.3  1,2   3.5  8.1 ---> 1.2  (sum = 29.5)
1.3  4.3   5.0   *      *     *  ----> 1.3 (sum = ***  )

[seanlaw]

Unfortunately, mpdist is completely different from what you are expecting. It is not returning the pairwise distance between subsequences, which is what you are looking for.

Due to potential slowness on larger datasets, the following recommendation is not supported by us and you are using at your own risk if you choose to explore this path.

You can manually compute the (z-normalized) distance profile for each row of your distance matrix via:

# Not tested or supported!!! Use at your own risk!!!

import stumpy
import numpy as np

T_A = np.random.rand(10)
T_B = np.random.rand(20)
m = 5

distance_matrix = np.array(
        [stumpy.mass(Q, T_B) for Q in stumpy.core.rolling_window(T_A, m)]
    )

Note that the subsequences in T_A and T_B are automatically z-normalized before the distances are computed (i.e., you get z-normalized Euclidean distances). For completeness, for non-normalized Euclidean distance, you'd need to add normalize=False:

distance_matrix = np.array(
        [stumpy.mass(Q, T_B, normalize=False) for Q in stumpy.core.rolling_window(T_A, m)]
    )

Again, for a sufficiently large dataset, this will be extremely slow and your computer will run out of memory. Use at your own risk!!!

[adityabhandwalkar]

ok , but T_A and T_B are two different time series what if I want to take subsequence from a T_A itself like this

can I do it with the above one?

[adityabhandwalkar]

distance_matrix = np.array(
[stumpy.mass(Q, T_A, normalize=False) for Q in stumpy.core.rolling_window(T_A, m)]
)

like this

[seanlaw]

Yes, that's all you would need to do.