Flv video crashing

I have made a custom flv player using netstream. Every now and again my video freezes and stops playing. When it freezes it also doesn’t respond to any controls, like togglePause or seek. Sometimes my video doesn’t appear on screen at all. This is really annoying because sometimes my video just plays completely fine. Has anyone had any similar problems ? What might cause this?