When holding Option over the window's buttons (the traffic light colored ones, Close/Minimize/Full Screen), you can use the green button as Zoom instead of Full Screen (you can also select this from Window > Zoom menu). Problem is there is not much of a difference: yes Zoom doesn't create a new space for the window like Full Screen does, but it also does increase the window size to always cover the whole screen. Instead it should grow the window just enough to fit its contents (make scrollbars disappear, if possible), but no bigger, at the current image's zoom level.