Time Series segments and stumpi

[CSharpDummy]

First of all: Great work!

I especially like the documentation and the pub section.

I have 2 questions: How do you deal with time series segments\interruptions?
Let’s say weekend or off hours in trading days? Any idea\suggestion\literature?

stumpi is very useful for real time application. It’s there a way to limit the history or look back window. And can you load historical data into stumpi let’s say from a db to speed up recovery?

Thx

[seanlaw]

@CSharpDummy Thank you for using STUMPY and for your kind words.

It is important for me to state that I am not a registered investment advisor and so I cannot and should not provide any suggestions on trading. I can only discuss, in general completely general terms, how the software may be used.

How do you deal with time series segments\interruptions?

Generally speaking, this falls into the area of "data imputation" and there is a wealth of information that one can search for on this topic. You may consider imputing the missing values by filling forward, filling backwards, averaging, interpolating, simply using np.nan. There is no single right answer. Data imputation is a whole research area on its own.

stumpi is very useful for real time application. It’s there a way to limit the history or look back window. And can you load historical data into stumpi let’s say from a db to speed up recovery?

I'm not sure what you mean by "can you load historical data in stumpy" but since we don't have any db hooks/interfaces then I'm going to guess that the answer is "no".

Now, regarding the look back, stumpi only increments one value at a time and only egress one value at a time. So, if you want to limit the comparison then you should instantiate the stumpi object with the desired look back size. So, if you instantiate with 50 data points then the total number of data points will always only be 50 (since the oldest data point is removed when the next newest data point arrives). The only time the total number of data points will increase if you set egress=False, which does not remove the oldest data point (this is probably not what most people want as the algorithm will get slower and slower over time as the length of the number of data points grows and grows).

I hope that makes sense

[CSharpDummy]

Hi,

Yes thx that all makes sense.

Using NaN for time segments that are not "typical" is a great idea and will solve my problems.