Justice Khanwilkar, who was the part of a bench headed by Chief Justice Dipak Misra, did not give any reason for opting out of hearing the matter.
The bench, which also comprised Justice D Y Chandrachud, said a new bench would be constituted for hearing the matter on March 28.
The apex court was to hear the petition filed by BJP leader Ajay Agrawal challenging the May 31, 2005 decision of the Delhi High Court quashing all charges against all the accused persons in the case.
The CBI informed the court that it has also filed an appeal against the high court order on February 2.
Today, the apex court was to hear the petition of Agrawal on his locus of filing the appeal as a third party in the matter.
Agrawal, who had contested the 2014 Lok Sabha election from Rae Bareli against then Congress president Sonia Gandhi, has been pursuing the case for over a decade in the apex court.
He had filed the appeal after the CBI had failed to challenge the high court verdict within the mandatory 90-day period.
Justice R S Sodhi (since retired), had delivered the May 31, 2005 judgement in which the charges were quashed against Europe-based industrialists Hinduja brothers.
Before the 2005 verdict of Justice Sodhi, another judge of the Delhi High Court, retired Justice J D Kapoor, had on February 4, 2004, exonerated the late prime minister in the case and directed the framing of charge of forgery under section 465 of the IPC against Bofors company.
The Rs 1,437-crore deal between India and Swedish arms manufacturer AB Bofors for the supply of 400 155mm Howitzer guns for the Indian Army was entered into on March 24, 1986.
Swedish Radio on April 16, 1987, had claimed that the company had paid bribes to top Indian politicians and defence personnel.
The CBI on January 22, 1990 had registered the FIR for alleged offences of criminal conspiracy, cheating and forgery under the Indian Penal Code and other sections of the Prevention of Corruption Act against Martin Ardbo, the then president of AB Bofors, alleged middleman Win Chadda and the Hinduja brothers.
It had alleged that certain public servants and private persons in India and abroad had entered into a criminal conspiracy between 1982 and 1987 in pursuance of which the offences of bribery, corruption, cheating and forgery were committed.
The first charge sheet in the case was filed on October 22, 1999, against Chadda, Ottavio Quattrocchi, the then defence secretary S K Bhatnagar, Ardbo and the Bofors company.
A supplementary charge sheet was filed against the Hinduja brothers — S P Hinduja, G P Hinduja and P P Hinduja– on October 9, 2000.
A special CBI court in Delhi on March 4, 2011, had discharged Quattrocchi from the case saying the country could not afford to spend hard-earned money on his extradition which had already cost Rs 250 crore.
Quattrocchi, who had fled from here on July 29-30, 1993, never appeared before any court in India to face prosecution.
He passed away on July 13, 2013. The other accused persons who died are Bhatnagar, Ardbo and Chadda.