You’re going back in time but you’re not going to be in the same place.

Think of a person walking on a road, you can drive past them and arrive somewhere before them.

You can’t do that while staying stationary. So you couldn’t say go back in time and see prehistoric Earth. It’s relative to when and where you are.