The name comes from the fact that most sell for a few pennies- some less than a penny. Most large brokerages do not want to handle them, and most exchanges will not list a stock unless the price is above their minimum standard, often $2 or more. Companies that fall below that minimum will be de-listed from the NYSE, NASDAQ or other large exchange. They wind up on what is called the "Pink Sheet".
Generally- penny stocks are a fool's investment, and a very fast way to lose money.
Penny stocks are pure speculation. Period.