Rollover complex buttons

I want to have a similar effect like the one that happens in the first button to the left. How can I achieve that? And how can the button - when you roll out - return smoothly to its initial position?