I will once again repeat what I said earlier.
"We (i.e. Shi'as) believe our Aimmah (a.s) are guiding people by the Permission of ALLAH (SWT)."
To understand better what I mean by Permission of ALLAH (SWT), let me ask you a question.
ALLAH (SWT) says in HIS Book (i.e. Al-Quran) that HE has Power to guide everyone and yet we see that all people are not guided. Why is this the case?
SAHIH INTERNATIONAL [al-An’am 6:149]
Say, "With Allah is the far-reaching argument. If He had willed, He would have guided you all."
SAHIH INTERNATIONAL [Yoonus 10:99]
And had your Lord willed, those on earth would have believed - all of them entirely. Then, [O Muhammad], would you compel the people in order that they become believers?
SAHIH INTERNATIONAL [al-Qasas 28:56]
Indeed, [O Muhammad], you do not guide whom you like, but Allah guides whom He wills. And He is most knowing of the [rightly] guided.
Its not about guidance but about performing the duties they were supposed to do.
Muhammad Sarwar: (Muhammad), if they turn away from your message, know that We have not sent you as their keeper. Your duty is only to deliver the message. When We grant mercy to the human being, he becomes joyous, but when he is afflicted by evil as a result of his own deeds, he proves to be ungrateful. (42:48)
Did they (your divine imams) deliver the message to public of their time?
Did they declare to the public that they are divinely appointed whom they are supposed to follow?
Did they say to the public that we are the hujjah of Allah on earth after messengers?
Did they announce in public that recognise us otherwise you will die death of jahilliyah?
If they did not then blame will not be on public. And none will go to hell bcoz of this reason aka imamah.