First of all, although you seem to be identifying nationality, that doesn't necessarily correspond to native language. Italy particularly has many languages. When I would go skiing in the Dolomites, I'd hear Italian natives speaking Italian, Ladin, German (Tyrolian) and even some Slovenian.
http://www.ethnologue.com/country/IT/languages
So, presuming that you speak Slovak and your husband speaks standard Italian, I think it is a wonderful opportunity for each of you to speak those languages with your son. If you do it from the time he is a baby, he will have native ability in both languages. I'm not quite so judgemental to call it "a shame," but it will definitely be a lost opportunity if you don't.
He will hear you speaking French, so he will probably pick that up as well. French is very useful, but the fact that neither of you have a native accent might be a disadvantage. Some French people are very derogatory about foreigners accents. If you have never had a problem with this Chauvinism, your accent is probably acceptable.
If you remain in Hungary, your son will learn that as well. Consider daycare and playgroups as an advantage.
The only disadvantage of gaining ability in all these languages is that he will have trouble answering the question: "What's your native language." It is possible to be "native" in all these as well as others. I had a Romanian friend who was native in five.
Your son is very fortunate to have you two as parents. I commend your concern.