My biggest problem is more of a mechanical issue, the darn micro USB connector is difficult to align and after a while of plugging in and out(especially due to being difficult to quickly see if it's right side up) it becomes loose.
As far as power itself, I have only run into catastrophic problems when attempting to use a hand held solar powered battery backup bank and when I tried to use a car cigarette lighter USB adapter. That being said it is my understanding that RPIS use a special power management scheme that actually throttles the processor cycles without any indicator, including the lightning bolt so maybe that laggyness could be power issues too? (Edit: There might be an iindicator of the low power throttle in the kernel log)
I actually bought some small micro usb power dongles with switches on them. So you only plug it into the pi once, then plug the dongle into the actual power source. Also means you can turn it on/off without being forced to pull the plug.
