Always remove event listeners?

is it necessarily to add and remove event listeners like so:



addEventListener(MouseEvent.MOUSE_DOWN,mDown);

function mDown(e:MouseEvent){
    removeEventListener(MouseEvent.MOUSE_DOWN,mDown); 
    addEventListener(MouseEvent.MOUSE_UP,mUp);
    //do stuff
}

function mUp(e:MouseEvent){
    removeEventListener(MouseEvent.MOUSE_UP,mUp)
    addEventListener(MouseEvent.MOUSE_DOWN,mDown);
    //do stuff
}

assuming rapid input, is it really better to do this rather than just let the mouseDown and MouseUp Listeners run all the time? It *seems *like it would be strenuous to constantly add and remove listeners like this. But this is how i generally see it doneā€¦ (i think)