[PATCH 6.1 132/132] arm64: Restrict pagetable teardown to avoid false warning