Odd Date Comparison Error

At work I wrote a shell script to sync certain files from a company repository. The repository is read-only.

The script has been functioning as expected for a couple of years. Until I started testing Debian 10 in the infrastructure.

The script uses bash and uses the -nt and -ot file comparison operators. For some reason, on Debian 10 systems the comparisons seemed to be failing. On Debian 10 systems the script insisted the local file was newer than the repository file.

I knew otherwise. The files were exact copies.

Or so I thought.

Adding some temporary code showed the date stamps were the same. There is some nuance in the Debian 10 systems that cause the file comparison operator anomaly.

I found no answers after poking around the web. I found references to POSIX requirements but nothing explicit why the problem might appear only in Debian.

I compared file date stamps using ls -la --time-style=full-iso. The files were indeed different based on the nanosecond portion of the time stamp.

I added that comparison to the shell script. I again experienced messages about time differences.

Eventually I discovered the scp -p command, which preserves time stamps, does not support nanoseconds. The target file nanoseconds always get zeroed. For example:

    Source files:
    file_a: 2020-01-29 16:16:15.404379767
    file_b: 2020-01-28 16:57:03.838916716

    Target files (after sending through scp -p):
    file_a: 2020-01-29 16:16:15.000000000
    file_b: 2020-01-28 16:57:03.000000000

scp -p wipes the nanoseconds even if the destination time stamp already includes nanoseconds. If the destination file has nanoseconds in the time stamp, using scp -p will overwrite the file and lose the nanoseconds.

If the file does not exist on the destination/target system, rsync -t|a will correctly transfer the time stamp including nanoseconds.

For comparing differences, seems rsync -t|a only uses the base modification time and size. After copying a file with scp -p, subsequent copies with rsync -t|a will not update the time stamp because the base modification time and size remain the same.

Using rsync -I will not preserve the time stamp, but an immediate subsequent rsync -t|a will restore the time stamp including nanoseconds.

Using the rsync --checksum option succeeds in correcting the full time. rsync -ac restores the nanoseconds even if the file and base time stamp are unchanged.

My final solution was to use --time-style="+%Y-%m-%d %H:%M:%S" for the comparison.

Interesting I never ran into this quirk after almost 20 years of using Linux based systems.

Yet Another Shoulder Shrug moment.

Posted: March 04, 2020 Category: Usability Tagged: Debian

Next: IndexedDB

Previous: Computers and Complexity