You pay taxes on it. The $100 you made in interest during 2013 is considered income by the IRS. So you have to pay taxes on it just like any other income from a job. So when you file your taxes, you will include that letter along with your W2 and any 1099 statements (income from side jobs) when you file.
So the $100 is just additional income that you add to your other income to pay taxes on.
I got a letter from my bank saying I have $100 in interest income. Do I get $100, or pay it?