Hiya, when selecting a Fill or any colour e.g. for a Text Frame; selecting Swatches and a swatch on any of the panels then navigating away, reselecting the Fill always drops back to Colour (and its panel) - or Gradient if thats been selected, never Swatches (i.e the last selected which is what I would expect); Can't say I noticed this behaviour before 1.9 and its exhibited on Big Sur and El Capitan.
Kind Regards,
Jason