regularizing large-scale features only?

[megbedell]

Maybe in addition to doing regularization, we want to put a prior on the flux variance over large wavelength scales or something along those lines?

In orders where the telluric features are strong, the regularization validation scheme naturally prefers very weak L2 amplitude, but this can lead to weird large-scale issues:


(plots are order 63 for a G star (top) and an M star (bottom))

Similarly, I think sometimes continuum issues that really should be absorbed into the observatory rest-frame component (i.e. the tellurics) are going into the star because the star's regularization is weaker to capture its actual absorption features:

Can we decouple the regularization on every point (to get rid of noise in the case of no spectral features) from the expectation that the continuum should be flat over long scales (at least for
the star)?

[davidwhogg]

This is a great question; we might need some help / advice...

[davidwhogg]

...but for now we should either live with it or increase L1...?

[dfm]

The "regularization" that we would want to handle this is called a Gaussian Process, of course :-)

I'd say that this is probably an issue for another time, but I do have an implementation of celerite in tf so it wouldn't be too hard to incorporate in principle...

[megbedell]

ah ok, maybe a little bit down the line we want to build a GP component into the model and let it handle all continuum fitting (including the initial normalization which we're not really happy with)?