Displacement Map Button Issue

When I apply a displacement map filter to a sprite which contains a button, the hit area of the button is not distorted with the filter.

Here is an example.

Is there a way to distort the hit area along with the button?
If not, does anyone know another way to solve this problem?