After playing for Hibs and Falkirk, he came south to join Oldham Athletic in 1960. After one season at Boundary Park, he headed to Chester and established himself as a reliable goalkeeper during his two-year stay.
Avenue boss Jimmy Sklar brought him to Park Avenue in December 1963 as cover for reserve keeper John Dine, who was filling in for the injured Bert Gebbie. In a twist of fate, Dine broke his thumb in training the following week, and Hardy was selected for the first team at Chesterfield on 21 December. He sat out the rest of the season and made the position his own.
Over the next six seasons, he rarely missed a game and was firm favorite, appearing in 265 of 299 league games until Avenue lost their place in the Football League. Only Chick Farr (294 games) and Ernie Scattergood (268 games) played in more goals for the club. Hardy played in Avenue's first non-league match against Netherfield in August 1970, but moved to First Division side Crystal Palace later that month for a transfer fee of £5,000.
Sadly, due to the form of Palace keeper John Jackson, he was dropped from the first team and by the following year he had dropped to non-league football, having short spells at both Gainsborough Trinity and Goole Town. After retiring from the game he settled in Bradford and revived his former profession as a qualified cabinetmaker. He ended his days in his hometown of Edinburgh.