I've done both. If you use a live usb then you will not need to change the security sector of your computer's hard drive. If you install non-live onto the USB you might. There are ways around it, but I think the installer will add an entry to your list of bootable partitions. One method is slightly lower impact and somewhat more portable than the other. One has more flexible storage choices and has less overhead getting data out of the USB. You can also more easily read its partitions from another running linux system.
The live version might have storage limitations on the USB, because it is a compressed filesystem. Mounting it in a second linux system takes some trouble. It can be a bit of a pain to change the size of the partition depending on how you go about the install. Some compressed filesystems grow as you need them to. Some don't. Some are stored in a file on the USB and have a hard size limit such as 8 or 32 gig regardless of how whopping large your USB is. Some have a partition on the USB that isn't compressed. The installers for live systems do not offer a lot of options. Their job is to get you up and running, usually with a minimal system, almost a disposable system.
USB thumb drives can be slow. What I have done successfully to get around that while still being portable is to install on a live USB that includes a hypervisor (virtual box) and to also have a large, fast external drive that isn't a thumb drive. The hypervisor runs from the thumb drive, but inside Virtual Box install another OS running that uses that faster external drive in an uncompressed partition (in my case I used a Toshiba external hard drive with a USB 3 connector). For this you'll have to do some googling on how to format the partition and give your hypervisor access to it. A big drive has a faster read/write than a USB drive does, so the resulting hypervisor (virtual) system running inside the slower live system can be effectively faster what with the faster drive, and it can still be portable. You don't have to change any boot records on the bare metal system. For this try to use a live distribution which fits entirely into RAM memory, so that its not constantly trying to access the slow USB. Then your only speed limitation comes from the virtual system and its faster drive. It will run plenty fast, faster than the hypervisor system.
