The house of Prophet was adjacent to masjid e nabwi.So why would he conduct the congregation in his outhouse and not masjid e
عن عائشة – رضي الله عنها – : أن رسول الله – صلى الله عليه وسلم – صلى في المسجد ذات ليلة فصلى بصلاته ناس ، ثم صلى من القابلة فكثر الناس ، ثم اجتمعوا من الليلة الثالثة أو الرابعة ، فلم يخرج إليهم رسول الله – صلى الله عليه وسلم – فلما أصبح قال : ” قد رأيت الذي صنعتم ، فلم يمنعني من الخروج إليكم إلا أني خشيت أن تفرض عليكم . قال : وذلك في رمضان
[`A’ishah narrated: The Messenger (saw)
prayed at the mosque one night and some people prayed behind him. Then he (saw) did similarly the next day and the number of people increased. On the third or fourth, people gathered but the Prophet (saw) never came out till morning, then he (saw) stood and addressed them saying: “I see what you’re doing and the only thing that prevented me from coming out to you was that I feared it may become a prescribed obligatory duty.” And that was in Ramadan. (Sunan al-Nasa’i al-Kubra)]
Prophet did Prohibit Trawih in congregation
This prohibition was out of fear that it might not become obligatory on people. It wasn't regarding unlawfulness of prayering taraweeh in congregation, since Prophet(saws) himself lead the prayer to people and he used to pray 8 rakah in Ramadhan. If it was prohibitory then he would have done so after the first 2 rakah itself, or at least the next day itself. Prophet(saws) didn't object the congregation at all for the days people prayed behind him, approx 33 units for three days. This proves the legality of praying Taraweeh in Jama'a.
This is also supported by the hadeeth of Ubay bin ka'ab(ra) you posted which shows that he didn't object in leading the congregation.
As for the issue of which one is better then, Most of the knowledgeable scholars agree that Nawafil prayer at home is better. But better doesn't means the other is prohibited/unlawful to be prayed. Moreover, there are many Muslims who memorized just two or three Surah of Quran, or they didn't memorize any Surah, such people prefer to offer prayer behind the Imam who at the least knows a good portion of Quran.
Also, what encourages them is this line of the Prophet(saws).
عَنْ أَبِي ذَرٍّ ، قَالَ : صُمْنَا مَعَ رَسُولِ اللَّهِ صَلَّى اللَّهُ عَلَيْهِ وَسَلَّمَ رَمَضَانَ ، فَلَمْ يَقُمْ بِنَا شَيْئًا مِنَ الشَّهْرِ حَتَّى بَقِيَ سَبْعٌ ، فَقَامَ بِنَا حَتَّى ذَهَبَ ثُلُثُ اللَّيْلِ ، فَلَمَّا كَانَتِ السَّادِسَةُ لَمْ يَقُمْ بِنَا ، فَلَمَّا كَانَتِ الْخَامِسَةُ قَامَ بِنَا حَتَّى ذَهَبَ شَطْرُ اللَّيْلِ ، فَقُلْتُ : يَا رَسُولَ اللَّهِ ، لَوْ نَفَّلْتَنَا قِيَامَ هَذِهِ اللَّيْلَةِ ، قَالَ : فَقَالَ : إِنَّ الرَّجُلَ إِذَا صَلَّى مَعَ الْإِمَامِ حَتَّى يَنْصَرِفَ حُسِبَ لَهُ قِيَامُ لَيْلَةٍ ، قَالَ : فَلَمَّا كَانَتِ الرَّابِعَةُ لَمْ يَقُمْ ، فَلَمَّا كَانَتِ الثَّالِثَةُ جَمَعَ أَهْلَهُ وَنِسَاءَهُ وَالنَّاسَ ، فَقَامَ بِنَا حَتَّى خَشِينَا أَنْ يَفُوتَنَا الْفَلَاحُ ، قَالَ : قُلْتُ : وَمَا الْفَلَاحُ ؟ قَالَ : السُّحُورُ ، ثُمَّ لَمْ يَقُمْ بِقِيَّةَ الشَّهْرِ
[Abu Dharr reported: We fasted Ramadan with the Messenger of Allah (saw) and he did not lead us in the night prayers (i.e Qiyam) until there were seven days left of the month, then he led us in night prayers until one-third of the night had passed. Then, when there were six days left, he did not lead us. When there were five days left, he led us in praying the night until half that night had passed. At that point, I requested: “O Messenger of Allah (saw), why don’t you lead us for the rest of this night?” He (saw) answered me: “Worry not,
If a man prays with the leader until he leaves, it will be counted as if he spent the whole night in prayer.” Then, when there were four days left, he did not lead us in the night prayers. When there were three days left he sent for his daughters and women, and gathered the people, and he led us in praying all throughout the night until we feared that we would miss “al-Falah”. He did not lead for the remainder of that month.” Dawud (one of the narrators) asked: “What is al-Falah?” I said: “The time of Suhur.” (Sunan al-Tirmidhi)]