The Giant Penny is a perennial feature of the Batcave throughout its various incarnations. It has its origins in a criminal named Joe Coyne, alias the Penny Plunderer. Coyne's shtick was stealing pennies. One of his heists involved a museum with an exhibit of penny related items (rare penny banks, rare stamps that originally cost a cent, etc.). One of the items on display was a giant Lincoln head penny that Batman rolled into several of Coyne's goons.
The Penny later appeared in the trophy gallery, first in Wayne Manor and later in the Batcave. Following the earthquake that leveled most of Gotham City, the Penny was knocked into an underwater crevice and jammed there. Later, a bored Batman contacted Aquaman to help him recover the Penny.
Batman: The Animated Series
In the Batman: The Animated Series episode "Almost Got 'm'", Two-Face tells a story of how he and his Two-Ton Gang had tried to steal $2,000,000 in two dollar bills from the Gotham mint. When Batman tried to stop them, Two-Face was able to tie him to a giant penny on a catapult. As Two-Face says, "The coin lands face down, you'll be squashed flat. It lands face up, you'll just break every bone in your body."
Batman is able to escape from the ropes using Two-Face's coin, which he had taken from him. After defeating the gang, the Gotham Mint allows Batman to keep the giant penny.