If your not hurting anyone, I'd say go for it.
My mother (sorry for the reference, I'm 35) and some other people I know have dated really younger men and they got a lot of slack for it but the people that are going to talk crap are going to talk crap no matter what situation your in. I think, as long as your happy and not hurting anyone, have fun! Just make sure you have some really good zingers for the nay sayers.
Lorena