It's been relatively straight forward me i.e: if you see a beautiful woman, 'the mind' automatically starts wondering about her midriff. In the last year or so, I have noticed a trend in myself of appreciating visuals of just female stomachs... without the face. It's been a huge leap for me. Usually, I would disregard any bellies without a face attached to them.
Technically, there is no difference. You may be able to tell subtle differences here and there but generally speaking, there is no "scientific evidence" which suggest male belly buttons are different to female ones. I believe however, that there are various forms of having a belly fetish; all though they may share a commonality, the internal "mechanics of arousal" will vary from person to person.
I think Anggun put it best "It's all in your mind"
https://www.youtube.com/watch?v=Q8D_63hhZKYGod she's hawt. 8)