Non-rectangular graphics

I am a little new to photoshop. How do I make graphic elements that are not rectangular? Let’s say I make some cool text, when I save it as a jpeg there is a white retangular background…can I get hid of it?

thanks