The intriguing history of The Grove can be traced right back to the 15th century. The dwelling which originally occupied this site is believed to be a "Ty Hir", a 15th century long house. This building survives, standing just apart from the main mansion house. This solidly built, two-storey long house would have housed not only the owners but their livestock too.

Between the main house and the Long House stands the Coach House. This was almost certainly used as a tannery by the Poyer family who owned the Grove in the 1700s. The property passed through many generations of this family until Anne Poyer Callen died in 1808, bequeathing the Grove to her husband, William Callen of Merrixton. The house was unoccupied as a family home for many years, until an heiress of the Callens bought it for her husband, J. L. G. P. Lewis of the Henllan estate at nearby Llanddewi Velfrey. He extended the Georgian L-shaped house in the late 19th Century, adding the hall, staircase, lounge, master bedroom and the library on the first floor landing. This enlargement was designed by the notable architect, John Pollard Seddon, in the neo-gothic Victorian style, heavily influenced by the Arts and Crafts movement, which is still very much in evidence today.