Search in a multidimensional array.

[TDL77]

Please help me figure it out, I have a dataframe with a size of df1.shape = (24840, 16) and a main dataframe with a size of
df2.shape = (9335034, 16), how can I find the top 10 similar slices in the main dataframe for the first dataframe .

I get an error:

ValueError: The window size must be less than or equal to 16

# Convert dataframes to numpy arrays
subsequence = X_train_5_scaled[index_5[0]:index_5[-1]]
m =len(subsequence)
# Compute matrix profiles for both arrays
matrix_profile = stumpy.stump( X_train_5_scaled.values.astype(float),
                                 m=m,
                                 ignore_trivial=True)
subsequence_matrix_profile = stumpy.stump( subsequence.values.astype(float),
                                             m=m,
                                             ignore_trivial=True)

# Find the indices of the most similar sections in the dataframe X_train_5_scaled
most_similar_indices = np.argsort(matrix_profile[:, 0])[:5] # Example: find 5 most similar parcels

# Display the found areas
for index in most_similar_indices:
     print(X_train_5_scaled.iloc[index:index+len(subsequence)])

[NimaSarajpoor]

[suggestion]
The clearer the question, the better answer you will get.

You may want to revise your post to address the following items:

(1) what is the shape of X_train_5_scaled ?

(2) Provide the value of "index_5" (Ask yourself this question: does a reader have the information regarding the variables you used?)

(3) If you cannot provide (2), then you should at least provide its first and last elements since you used them in your code.

(4) The error shows the number 16, which is the number of columns of your df. So, if I were you, I would print the shape of subseq dataframe, m, and the shape of dataframe/array you pass to the function stumpy.stump. That may give you a hint.

[seanlaw]

Also note that stumpy.stump only works for 1-dimensional arrays. For multi-dimensional arrays, please see this tutorial

[TDL77]

thanks for your reply. let me ask again. Do I understand correctly that to search for multidimensional plots, the M= dimension must match the number of columns?
those.
subsequence = X_train_5_scaled[index_5[0]:index_5[-1]]

• Convert dataframes to numpy arrays
  X_train_5_scaled_array = X_train_5_scaled.values.astype(float)
  subsequence_array = subsequence.values.astype(float)
  m = len(subsequence_array) # 24840
• Compute matrix profiles for both arrays
  matrix_profile = stumpy.mstump(X_train_5_scaled_array, m=m)
  ....in this piece, ValueError: The window size must be less than or equal to 16
  print(X_train_5_scaled.values.astype(float).shape) # (9335034, 16)
  print(X_train_5_scaled[index_5[0]:index_5[-1]].values.astype(float).shape) # (24840, 16) <<<-- the slice we are looking for in the large dataframe

and when I did it like this, everything worked
stumpy.mstump(X_train_5_scaled_array.T, m=m)

[NimaSarajpoor]

I think when the input is pandas dataframe, the number of dimension is the number of columns. If it is numpy array, each row is one time series (so the number of rows in the numpy ndarray is the dimension in that case)