So it really looks like it has something do with the adjustment layer I've got on the photo itself.
It's just a simple Brightness/Contrast adjustment. Opacity on that layer is 100%, opacity on the photo layer is 100%.
When I uncheck the bright/contrast layer and export as PDF/X-4 the photo looks fine. When I turn that layer back on and export, the photo looks like it has an opacity setting of 50%.
As a test, I tried to export as PDF/X-3 and the export failed with a basic "Could not save" error message. But X-4 works, except for this one issue...