Ardross Castle was built on the site of the old Sutherland hunting lodge. From 1847, nearly 400 men, including masons, were employed in work at the new house or in the grounds.
Just after crossing the high stone bridge over the Tollie Burn, an improved entrance road led to the new mansion, which was built round two sides of a square, with a courtyard and servants’ quarters in the inner angle. The building was to be two storeys in height excluding attics, with the main entrance facing east and entered from a port cochere, each arch of which was ten feet wide and the front arch would contain the family crest of Sir Alexander Matheson. The Tower was four storeys high with tasteful Chinese railings, in the centre of which, on the east side, there would be another family crest. The North Wing would contain Drawing Room, Billiard Room, Business Room, while the South Wing would contain Drawing Room, Dining Room and Library. Along this range, a verandah would be carried above the Drawing Room window. The bedrooms were to be on the second floor and above the windows of the principal bedrooms there was to be a fine piece of tracery with the letters AM interwoven within it. As well as the main tower, there would be five bellcast turrets.
It was proposed to cut the sloping bank in front of the drawing room into a double terrace, a flower garden of some four acres would be placed to the east of the house, a kitchen garden of two acres to the west and an orchard of three acres to the north. Fourteen men were employed in thinning out the old trees and planting more valuable timber. Not less than 400 acres would be planted with trees. A Bowling Green would be laid out at the front of the house and new bridges would be built across the Tollie Burn and Alness River.
There were considerable alterations to the mansion in 1880/81, with Alexander Ross of Inverness as the architect. It was at this time that the Great Hall with its magnificent hammerbeam roof was built.
After the death of Sir Alexander in 1886, his son Kenneth inherited Ardross Castle, but by then Duncraig Castle had been greatly enhanced and he preferred life there. From 1887, Ardross Castle was tenanted by Sir J.H. Granville Smyth of Long Ashton, Clifton, and remained in his tenancy until 1898 when it ws sold to C. W. Dyson Perrins. It wasn’t long before he was making more alterations.
Charles William Dyson Perrins (1864-1958) was heir to the firm of Lea & Perrins, manufacturers of Worcester sauce. Described as a quiet, unassuming man, he nevertheless fulfilled many public duties in the area. He was a collector of pictures and other artefacts, many of which he gave to libraries and museums. Two servants in the castle were the poet Hugh Macdiarmid and his wife, and the former described Dyson Perrins in complimentary terms.