HTTP streaming: setting low buffertime has no effect

I have an http streaming .mp4 file that is encoded to 300kbps. The file size is 2.9mb and the play time is just over one minute.

I want to set the buffertime to a low number so the video starts playing almost immediately, but whether or not I set the buffertime to 6 or do not set it at all (which should put it at the default of 0.1) it still completely buffers the entire video before starting play back.

Is there I am missing about http streaming and buffertime?

PS This happens with my own player AND the flash cs3 FLVplayback (I changed the file name to .flv for this one.)

Thank you!