What you are implying is that since we are to live by the Spirit, there is no need for a literal temple, for the one replaces the other. But yet, even before Jesus came to Earth, since the beginning of sin, faith alone is the only thing that saved anyone. Living by the Spirit is not a new thing, it has been the way since Adam and Eve.
God doesn't change, He is constant. If He wanted people to worship physically as well as spiritually in the Old Testament, then there is no reason to think that He doesn't want that anymore. The only change is in the sacrificial and cleansing laws because Jesus cleansed us once and for all.
Am I to understand that you believe in universalism? If that is the case, then we are certainly on different planes here.