Are you expected your 2D scene to remain flat in the spherical panorama, for example, as one wall/floor/ceiling of a six-sided room, or do you expect 2- or 3-point perspective to be transformed from the 2D scene into extended 3D lines?d
What field of view in the spherical panorama do you want your 2D scene to cover?
I am familiar with going the other way, from 3D to 2D. Usually the 2D images produced from a map of a 3D spherical panorama are extremely distorted when viewed as flat 2D images. The simplest case is cube mapping, where shots of the ceiling, floor and 4 walls are combined into one (strange looking) 2D image. Going the other way, from 2D to 3D, seems like magic, unless you have some particular constraints or restriction on how you want one 2D image mapped onto the inside of a sphere. 35 years ago when I was in grad school, inferring 3D shape from 2D features such as silhouettes or axes of symmetry was an extremely limited, and ambiguous, process. While computer vision has gotten a lot better in three and a half decades, it hasn't gotten magical, yet.
