Mr Tharoor questioned the need to push it while the Minister called his remark "ignorant".
Sushma Swaraj, in her reply to a question, said: "It is often asked why Hindi is not an official language in the UN. Today, I will want to tell the House, the biggest problem is the procedure."
The Minister explained that as per the procedure, two-thirds of the 193 members of the organisation -- which comes to 129 -- will have to vote in favour of making Hindi an official language and also share the financial expenditure that would be incurred in the process.
"The problem comes when apart from voting, the burden of the amount also falls on them. Economically weaker countries that support us shy away from this. We are working on it, we are making attempts to get support of countries like Fiji, Mauritius, Surinam... where people of Indian origin are there.
"When we get that kind of support and they are also ready to bear the financial burden, it will become an official language," she said.
When a member pointed out that making Hindi an official language will require an expenditure of Rs. 40 crore every year, Ms Swaraj said: "Not just Rs. 40 crore, the government is ready to spend Rs. 400 crore on it."
She, however, added that spending money would not serve the purpose.
Sushma Swaraj also highlighted that Prime Minister Narendra Modi and she had spoken at the UN in Hindi.
"Even when we have (foreign) guests, if they speak in English, we speak in English. If they speak in their own language, we speak in Hindi. As far as glory of the language is concerned, the External Affairs Ministry never had so much work done in Hindi as now," she said.
Mr Tharoor, who worked in the UN and announced his retirement after finishing second in the 2006 election for UN Secretary-General, questioned the need to push for Hindi, which he pointed out was not even the national language of India.
"Hindi is not the national language, it is an official language. Seeking to promote Hindi raises an important question. Why do we need an official language in the UN? Arabic does not have more speakers than Hindi, but Arabic is spoken by 22 countries, whereas Hindi is only used as an official language by one country -- us," he said.
"The question is what purpose is being served by this. If indeed we have a Prime Minister or Foreign Minister who prefers to speak Hindi, they can do so and we can pay to get that speech to be translated. Why should we put our future Foreign Ministers and Prime Ministers who may be from Tamil Nadu in a position," he said.
"The government has to defend its position. I understand the pride of Hindi-speaking people, but people of this country who do not speak in Hindi also take pride in being Indian," he said.
The statement did not go well with several members of the treasury benches who raised the pitch in protest.
Ms Swaraj said Hindi was spoken in several other countries as well as by the Indian diaspora abroad. "Saying Hindi is spoken only in India is your ignorance."
In a written reply, she said India was in touch with 129 countries to make this happen