I can see how someone might reach that conclusion, but also consider that there are a lot of "understandings" that we liberty minded people are trying to debunk. It's commonly understood that Iran having a nuclear weapon is a serious threat; it's commonly understood that the terrorists hate us because of our freedoms; and, it's commonly understood that the government should be at all involved with the notion of "positive liberties" such as education and healthcare. Would you not blame a member of Congress for voting for bills based on common understandings such as these?
In both electing officials for Congress and electing delegates, we are supposed to elect those that will best do their job. Just as we shouldn't elect people to Congress based on how well they will vote how the majority of their constituency wants, we should also not promote the idea of matching delegate votes to the popular vote - which if that were the case, delegates would not have any real function or purpose at all. The job of an elected member of Congress is to protect individual liberty with a primary focus on protecting it against its greatest threat - government intrusion, all within his/her Constitutional requirements and limitations - AS HE/SHE SEES FIT - NOT as the usually absent-minded majority of the public desires. The job of a delegate is to vote to choose the candidate for president that they see most fit for the job.