In the broadest terms, being fiscally conservative means that you desire a balanced budget. In that way, Romney's plan is "similar" to Paul's in that it increases the sovereign debt the least among the other 3. Santorum's and Gingrich's proposals are bat-shit insane and will continue to increase our national debt. Maybe not as fast as Obama has done but far faster than Romney.
I'm not saying we have much in common with Romney, but we have the least conflicts with him. With Santorum and Gingrich, I would never even consider voting for them. With Romney, I'd have to give it some thought and consider Romney's VP and the statements and debates leading up to November. But ultimately, assuming I'm still a New York voter then, I'd vote Libertarian (if I can verify that it's not another "Bob Barr").
The reason there is it's almost impossible for New York to go Republican so my vote would be "thrown away" either way. At least supporting the Libertarian Party, I can advance their policy positions and help them qualify for federal funds matching and equal airtime. But after New York votes, I'll start in earnest identifying a GOP primary candidates for my Congressional district and Senate.
Kirsten Gillibrand was OK as a Congresswoman but I really don't like her votes on liberty issues. If your state has already held its primary or caucus, the best thing you can do now is to research your elected Representatives and primary candidates. Don't limit yourself to the GOP either--the Democrats might have fiscally conservative or at least liberty-minded candidates. They may do better than moderate Republicans.