They were punished to such a great degree that our Sages declared (Bava Batra 3b) that whoever says, “I descend from the House of the Hasmoneans” must be a slave since all the true Hasmoneans were obliterated on account of violating Jacob’s command that the scepter shall not depart from Judah. The descendants of the Hasmonean king Simon may have been punished because of their affiliation with the heretical Sadducees (see Berachot 29a and Kiddushin 66a), but all the immediate descendants of the righteous Matisyahu perished only because they violated Jacob’s command.
And since they ruled improperly, they were punished middah k’neged middah. G-d gave their slaves dominion over them, and those slaves killed all the members of the Hasmonean family.
The Ramban entertains the suggestion that the Hasmoneans’ sin was serving as kings despite being kohanim. Numbers 18:7 states, “Tishmiru et kehunat’chem l’chol d’var hamizbeyach, ul’mibeit laparochat va’avaditem, avodat matanah etayn et kehunatchem – You shall safeguard your priesthood regarding all matters of the Altar and within the Curtain, and you shall serve; I have presented your priesthood as a service that is a gift.” The Hasmoneans, therefore, should not have become kings; they should have remained performing the service of G-d in the Temple.
In discussing this matter, the Ramban cites a baraita in the Talmud Yerushalmi (Horayot 3:2) which states that kohanim may not be anointed as kings. R. Yehudah Antoria said the prohibition is based on the verse (Genesis 49:8), “The scepter shall not depart from Judah.”
R. Chiya bar Abba, however, bases the prohibition on Deuteronomy 17:20 and 18:1. The first verse states, “…lema’an ya’arich yamim al mamlachto hu u’banav b’kerev Yisrael – …so that [your king] will prolong his years over his kingdom, he and his sons amid Israel.” The verse immediately following states, “Lo yi’hiyeh lakohanim haleviyim… – There shall not be for the kohanim, the Levites, the entire tribe of Levi, a portion and an inheritance with Israel.” R. Chiya bar Abba sees the juxtaposition of these two verses intimating that kohanim should not be anointed kings.
Therefore, although the Jewish people may – according the opinion of R. Yehudah Antoria – accept a king upon themselves from one of the other tribes for some temporary necessity, they may not anoint them. This way, these rulers won’t have the grandeur of official kingship; they will be like judges and officers instead. The reason kohanim are singled out is because kohanim are potentially fit to be anointed as kohanim gedolim. We therefore need to be informed that they can’t be anointed as kings. As Horayot 11b states clearly: One must not anoint a king unless he is from the House of David.
The Ramban, however, concurs with R. Chiya bar Abba that a kohen is biblically prohibited from ever serving as a king – even if he isn’t anointed.
According to this Ramban, it is entirely possible that Rabbi Yehudah HaNassi purposely omitted any mention of the Hasmonean kings, as well as the mitzvah of kindling the Chanukah lights (the core mitzvah of Chanukah), from the Mishnah to indicate that the Hasmoneans, righteous as they were, erred when they improperly crowned themselves kings over Israel.