"Me and a Gun" was the first single released from the Tori Amos LP, Little Earthquakes, released on October 21, 1991 in the United Kingdom, as a CD single, 7" and 12". It was also released in Germany.

The song is three minutes and forty-four seconds long. It was inspired by a rape that occurred in LA when Amos was 21. After she performed at a bar, a patron asked her if he could have a ride home. She obliged, and he raped her at knifepoint (many journalists mistakenly state she was raped at gunpoint, possibly due to the song's title). Fortunately, she escaped. Years later, in London, Amos saw the film Thelma and Louise and was stirred. On the way to a show, she wrote the song in her head. That night, she performed the song a cappella.

Tori Amos explains the experience:

"I'll never talk about it at this level again but let me ask you. Why have I survived that kind of night, when other women didn't", she says.

"How am I alive to tell you this tale when he was ready to slice me up? In the song I say it was 'Me and a Gun' but it wasn't a gun. It was a knife he had. And the idea was to take me to his friends and cut me up, and he kept telling me that, for hours. And if he hadn't needed more drugs I would have been just one more news report, where you see the parents grieving for their daughter".

"And I was singing hymns, as I say in the song, because he told me to. I sang to stay alive. Yet I survived that torture, which left me urinating all over myself and left me paralysed for years. That's what that night was all about, mutilation, more than violation through sex".

"I really do feel as though I was psychologically mutilated that night and that now I'm trying to put the pieces back together again. Through love, not hatred. And through my music. My strength has been to open again, to life, and my victory is the fact that, despite it all, I kept alive my vulnerability".

This song did not take off very well as a single, possibly given its serious subject matter; in fact, the track was not the A-side to its own single. "Silent All These Years," another song from Little Earthquakes, was the first track on this single, with "Gun" appearing third (or as the B-side on the 7" release). "Silent" was a more accessible song, and radio stations began to play that instead. Ultimately the single was re-released with nearly identical packaging but retitled as Silent All These Years.

Amos made a habit of singing this song during live appearances. In 1994, the DC Rape Crisis Center awarded Amos a Visionary award for the song and the co-creation of RAINN. Ultimately Amos stopped singing the song live in December of 2001 and did not sing it live again until September 2007 (with the exception of one performance in Istanbul in August 2005). 1