I may be late to the party. And I may not fully understand what you're asking. The method I describe will not produce any kind of rough edges. It specifically will apply a texture to both fill and stroke of an object.
Create your vector object with stroke.
Duplicate it. Expand Stroke. Bolean "Add" stroke and fill.
Now you should have two objects. One, your first vector + Stroke. Your second object, the single vector shape with no stroke. The latter lies on top of the layer stack.
Create your pixel layer that contains your texture and constrain it to object 2. For this I just used a speckled brush using the color white.
in the layer options set object 2 to "Erase".
Attached is a screenshot of the final result and the layer stack.