Children comes after a Man marries and has a wife. Without wife there are no children. So the wife is entitled as Ahl al-bayt before the children. As for Ali then he is blood relative of Prophet(saws), so was ibn Abbas, Aqeel, Jafar, etc. If you say he had his status due to marrying Fatima(ra), then you agree that the possibility of divorce is there in this relation.
Yazid b. Hayyan reported, I went along with Husain b. Sabra and 'Umar b. Muslim to Zaid b. Arqam and, as we sat by his side, Husain said to him:
Zaid. you have been able to acquire a great virtue that you saw Allah's Messenger (ﷺ) listened to his talk, fought by his side in (different) battles, offered prayer behind me. Zaid, you have in fact earned a great virtue. Zaid, narrate to us what you heard from Allah's Messenger (ﷺ). He said: I have grown old and have almost spent my age and I have forgotten some of the things which I remembered in connection with Allah's Messenger (ﷺ), so accept whatever I narrate to you, and which I do not narrate do not compel me to do that. He then said: One day Allah's Messenger (ﷺ) stood up to deliver sermon at a watering place known as Khumm situated between Mecca and Medina. He praised Allah, extolled Him and delivered the sermon and. exhorted (us) and said: Now to our purpose. O people, I am a human being. I am about to receive a messenger (the angel of death) from my Lord and I, in response to Allah's call, (would bid good-bye to you), but I am leaving among you two weighty things: the one being the Book of Allah in which there is right guidance and light, so hold fast to the Book of Allah and adhere to it. He exhorted (us) (to hold fast) to the Book of Allah and then said: The second are the members of my household I remind you (of your duties) to the members of my family. He (Husain) said to Zaid:
Who are the members of his household? Aren't his wives the members of his family? Thereupon he said: His wives are the members of his family (but here) the members of his family are those for whom acceptance of Zakat is forbidden. And he said: Who are they? Thereupon he said: 'Ali and the offspring of 'Ali, 'Aqil and the offspring of 'Aqil and the offspring of Ja'far and the offspring of 'Abbas. Husain said: These are those for whom the acceptance of Zakat is forbidden. Zaid said: Yes.Sahih Muslim
Book 44, Hadith 55
http://sunnah.com/muslim/44/55