Guitarist John Lombardo, a founding member of 10,000 Maniacs, and Mary Ramsey, a classically trained violinist from the age of five, formed as a duo in Buffalo. Lombardo's rock inspiration and Ramsey's folk and classical influences made them a popular local act. With production help from Mitch Easter, John Mary recorded Victory Gardens in 1990; it was released the following year. The follow-up, The Weedkiller's Daughter, featured contributors Alex Chilton and Robert Buck (from 10,000 Maniacs). After Natalie Merchant left 10,000 Maniacs in 1994, John Mary joined the group.