Guidance from the IRS suggests that you need to maintain cost basis information for cryptocurrency, and pay tax on any gains if/when you do something that causes it to be converted to US dollars.
Hell, even with a cash allowance, if it exceeds a certain value during the year, you owe gift tax (though it's obviously difficult for the IRS to figure out about your allowance, especially if you never put it in a bank account).
