As libertarians we should all understand that Mitt Romney and Barack Obama have rights just like any other human being. Their decision to debate only each other is fully within their rights of freedom of speech. No one should be legally forced to have a conversation with people they don't want to talk to on national television. To think otherwise is extremely un-libertarian.
The only way to change this situation is for enough people to refuse to vote for anyone who isn't willing to debate all candidates.
I would never support a law that forced Obama and Romney to debate Gary Johnson and Jill Stein against their will. The only true libertarian solution is for the people of this country to demand politicians participate in third-party inclusive debates if they want to be elected.