When the gold and silver sales people call you on the phone to get you to order more, it's usually because they expect it to go DOWN soon. I realize it's a long term investment, but I know the guy I buy from through Midas calls me to ask if I want to buy, then I notice it tends to take a dive after that.
This doesn't mean don't buy gold and silver, but they are "short selling" and you happen to be the one taking the initial loss. I still buy from them. Just realize this fact.