Taking the first file from ls -ut (with no need to throw away unneeded long listing information and using access time instead of modification time for sorting) instead of the last file modified from ls -lrt would seem to come closer to what you said you want to do and be both faster and easier.

But, of course, this gives you the most recently accessed file even if it hasn't been accessed for a week.

The find -newer path primary compares modification times instead of access times. The standard find primaries do not provide a way to search for files based on access times.