Mr Ashley has accused the MP Iain Wright, of "abusing parliamentary procedure" in order to create a "media circus" which was not in the interests of his staff.
This comes after Mr Wright the chairman of the Commons Business, Skills and Innovation Committee, warned Mr Ashley last week that he would be in contempt of Parliament unless he agreed a date to appear before them.
In a letter to Mr Wright the Sports Direct boss makes no offer of a date to give evidence - although he repeats an invitation to the committee to visit Sports Direct's premises at Shirebrook in Derbyshire.
"I was disgusted to learn that you have adopted a stance that is deliberately antagonistic," he wrote.
Sea fret - a look at the foggy phenomenon which spoils sunny days in South Shields
South Shields man jailed for terrifying physical and sexual assault on woman in her own home
South Tyneside man given jail warning after breaching court order
13 great pictures as the sunshine sees crowds flock to South Shields beaches
The 16 places where most crime was reported across South Tyneside in June
"By refusing to visit Sports Direct to see things with your own eyes, you are missing out on a genuine opportunity to gain a detailed and balanced understanding of the matters you wish to discuss.
"I believe you are abusing parliamentary procedure in an attempt to create a media circus in Westminster, which is not in the best interests of any of the people who work at Sports Direct."
The committee has been seeking to question Mr Ashley about working practices at his Shirebrook warehouse, including reports of poor working conditions and the use of controversial zero-hours contracts.
In his letter, Mr Wright warned if Mr Ashley failed to agree a date to give evidence, the committee reserved the right to take the matter further "including seeking the support of the House of Commons in respect of any complaint of contempt".
It is thought that no one has been charged with contempt of Parliament since the 1950s.
However, in the Commons on Thursday, the shadow leader of the House, Chris Bryant, said the House could "force him to attend".
"He may be the 22nd richest man in Britain but he is running a modern-day sweatshop and this House will get to the truth," he said.
A spokesman for the committee said members would consider Mr Ashley's letter and their next steps when they meet on Tuesday.