It was narrated that Anas bin Malik said:
“They said: ‘O Messenger of Allah, is Hajj (required) every year?’ He said: ‘If I were to say yes, it would have become obligatory, and if it were to become obligatory, you would not (be able to) do it, and if you did not do it you would be punished.’”

حَدَّثَنَا مُحَمَّدُ بْنُ عَبْدِ اللَّهِ بْنِ نُمَيْرٍ، حَدَّثَنَا مُحَمَّدُ بْنُ أَبِي عُبَيْدَةَ، عَنْ أَبِيهِ، عَنِ الأَعْمَشِ، عَنْ أَبِي سُفْيَانَ، عَنْ أَنَسِ بْنِ مَالِكٍ، قَالَ قَالُوا يَا رَسُولَ اللَّهِ الْحَجُّ فِي كُلِّ عَامٍ قَالَ " لَوْ قُلْتُ نَعَمْ لَوَجَبَتْ وَلَوْ وَجَبَتْ لَمْ تَقُومُوا بِهَا وَلَوْ لَمْ تَقُومُوا بِهَا عُذِّبْتُمْ " .