I'd just like to chime in to say that for me, on a 2018 MacBook Pro 15", the histogram is erratic if metal is switched on. Sometimes it's pretty slow to update. 20+ seconds sounds about right. Also the histogram underlays in the levels and curves popups sometimes take a long time to show up. Often long enough I stop waiting for them. I'm not sure if they would after several minutes, I never waited that long. Also, I use the Marquee option a lot for the histogram and with metal compute on it's useless: chances are it will never update to reflect the marquee selection. Or at least, I'm not going to sit and wait for 5 minutes to check.
When I switch off metal compute all of this is essentially instant, updates to the histogram take < 1sec in all cases, even on huge images. Since metal also slows down the inpainting brush, I keep it on OpenGL.
I must say I'm confused about the effects of metal for the display setting, and metal compute. If I understand the marketing, things should be faster with metal, but all I see are very serious slow downs in essential features. I thought the histogram update problem went away with metal compute off, but I really need to switch back to OpenGL fully for it to be performant for me. By the way, I work almost exclusively on ~24MP 16bit Tiff files.