Cant remove event listener (function in a function)

1

I have this function, that when given a movie clip and a destination, it moves it to there, and eases out.

I have buttons that control where it moves to. This all works great.
But I want it so that if it is moving, and a new destination is clicked, it stops and moves to the new destination.

When I remove the event listener, nothing happens. But the event listener removes properly when it stops. (just wont when it is moving)

So the tricky part is this. I have a function in a function, It is there so the moving event:enter_frame parrt can see the same variables passed to it from the click.

If I move this function out of the other function, I can remove it. But it can not access the variables and does not move.

Part that does not work
if (isMoving == true) {// are we moving?
trace(‘is moving:’+isMoving);
[COLOR=“Red”]removeEventListener(Event.ENTER_FRAME, mover);// stop moving[/COLOR]



function easeTo(toMove:MovieClip, dest:Number):void {

	trace(toMove +' is being moved to: '+ dest);

	if ( toMove.x  != dest ) {//are we already there? If so, do nothing
		if (isMoving == true) {// are we moving?
			trace('is moving:'+isMoving);
			removeEventListener(Event.ENTER_FRAME, mover);// stop moving
		} else {// not moving, set dest and go.
			trace('is moving:'+isMoving);// reads faluse
			isMoving = true;// now set to true

			addEventListener(Event.ENTER_FRAME, mover);// start moving
		}
	} else {
		trace('we are already there');// do nothing
	}

	function mover(event:Event) {// move. Function here because it needs to access internal varibles
		
		if (Math.abs(dest - toMove.x) < 1) { // are we there? yes? stop event listener
			removeEventListener(Event.ENTER_FRAME, mover);
			isMoving = false;
			wDirection = '';
		} else {
			toMove.x += (dest - toMove.x) * speed;
		}

	}
}