Sahih Muslim 1333a
'A'isha (Allah be pleased with her) reported: Allah's Messenger may peace be upon him) said to me: Had your people not been unbelievers in the recent past (had they not quite recently accepted Islam), I would have demolished the Ka'ba and would have rebuilt it on the foundation (laid) by Ibrahim;
for when the Quraish had built the Ka'ba, they reduced its (area), and I would also have built (a door) in the rear.
حَدَّثَنَا يَحْيَى بْنُ يَحْيَى، أَخْبَرَنَا أَبُو مُعَاوِيَةَ، عَنْ هِشَامِ بْنِ عُرْوَةَ، عَنْ أَبِيهِ، عَنْ عَائِشَةَ، قَالَتْ قَالَ لِي رَسُولُ اللَّهِ ﷺ " لَوْلاَ حَدَاثَةُ عَهْدِ قَوْمِكِ بِالْكُفْرِ لَنَقَضْتُ الْكَعْبَةَ وَلَجَعَلْتُهَا عَلَى أَسَاسِ إِبْرَاهِيمَ فَإِنَّ قُرَيْشًا حِينَ بَنَتِ الْبَيْتَ اسْتَقْصَرَتْ وَلَجَعَلْتُ لَهَا خَلْفًا " .