The next few years saw changes to the interior of the church - a stone pulpit (removed a few years ago because it was becoming unsafe), the Stations of the Cross carved on stone panels, and a gallery for the organ and choir. However the outer walls threatened to collapse because poor quality materials had been used by dishonest builders: buttresses were installed to make the walls safe.
In 1866, the Hibbert family left Rugby and sold Bilton Grange. Captain Hibbert was in poor health and needed specialist treatment only easily available in London. He planned one final gift to the congregation of St.Maries - a tower and spire, designed in the Gothic style by Bernard Whelan. Still the highest spire in the county at nearly 200 feet, it was completed in 1872. Eight bells, cast in the Whitechapel foundry and also the gift of Captain Hibbert, are still rung, not by ropes but as a carillon with levers operating hammers to strike the bells.
Sadly, Captain Hibbert was too ill to attend the opening ceremony when the bells were played for the first time, although he paid for a splendid lunch for 200 guests. Captain Hibbert died three years later. The body of the great benefactor of the Catholic community in Rugby rests with that of his wife in the family vault beneath the Hibbert chapel.