I´ve a class, in my application, that generates different layout interfaces depending on what to do. Every interface contains a number of event listeners for buttons etc. The maximum numbers of interfaces is between 4-5.
Turn on/off the different interfaces by the visible=true/false property and don´t remove any event listers.
Turn off the different interfaces using removeChild() and then remove all listeners. When turning on you creates the interfaces again.
Every event listener use memory but creating the interface everytime also have some affects (/CPU).