[Discuss] Issuing the 'sync' command more than once (and a tangent on how not to run a high-tech company)

[mbr at arlsoft.com (MBR)]

On 6/12/2012 11:22 PM, Jack Coats wrote:
> In old SunOS days, we could issue the 'sync' command, twice, to ensure
> all system
> buffers had been written to disk.  You could experiment to see if
> issuing it occasionally
> in your script helps.  Or issue it outside the script, even in a chron
> might help.

Actually, calling 'sync' multiple times from a script really won't 
help.  To the best of my knowledge, no Unix kernel has ever contained 
code that counts the number of times sync() (the system call that the 
'sync' command issues) has been called.  But I do know something about 
the history of the myth that typing 'sync' multiple times guarantees 
that all system buffers have been written to disk.  I'm afraid I 
couldn't keep myself from digressing a bit into the history of an 
obscure and thankfully long-defunct Unix desktop company.  Feel free to 
skip the next two paragraphs.

I first encountered Unix in 1980, when I went to work for Fortune 
Systems in Foster City, CA. A little company history here. Sorry for 
diverging so far from the 'sync' command, but Fortune is a cautionary 
tale in how not to run a high-tech company.  Fortunewas one of the first 
companies to build a Unix desktop computer. The company had some of the 
best Unix technical people I've ever met, and three founders who had 
very little interest in building a viable company or product.  Their 
primary interest was in having a company to take public when market 
hysteria for high tech was at its peak.  (See 
http://www.old-computers.com/museum/computer.asp?c=767 and 
http://articles.latimes.com/1985-01-08/business/fi-7514_1_computer-systems 
for more.  Oh, and there's also 
http://www.fundinguniverse.com/company-histories/Itel-Corporation-Company-History.html 
which recounts how before starting Fortune, its president had managed to 
lead his previous company, Itel (not to be confused with I*n*tel), into 
what at the time was the largest bankruptcy ever in U.S. history.)  The 
head of Fortune's Unix kernel group knew Bill Joy lobbied hard to hire 
him.  But he couldn't convince Fortune's foundersto do so!  A year or 
two later, Joy founded Sun Microsystems!

While the hardware guys were still designing the Motorola 68000-based 
architecture that would eventually become Fortune's product, we software 
guys did all our development on a VAX running 4.1bsd.  (YAT (Yet Another 
Tangent): 4.1bsd was before Unix had the networking functionality 
currently provided by socket calls.  If I remember correctly, the socket 
API didn't appear until Berkeley's 4.2bsd release around 1982 or 1983.)  
But the kernel Fortune was porting to their own hardware was not 4.1 
bsd.  It was Version 7 straight out of Bell Labs, which was the code 
that the Berkeley guys had enhanced to create 4.1bsd. Shortly after the 
language tools guys managed to modify the C compiler on the VAX into a 
cross-compiler that could emit 68000 code, the hardware guys produced 
some prototype hardware for us to try running our code on.

_And now, back to 'sync' (finally ...)_

I vividly remember sitting in front of a breadboard on a workbench with 
one of the OS guys, a recent Berkeley computer science grad, who'd just 
gotten the cross-compiled kernel working on the new hardware, while I 
debugged the word processor code application I was writing.  At the end 
of that session he told me that it was important to type the 'sync' 
command _*three times*_ before shutting the system down.  That seemed 
odd to me.  "Why *_three_* times?" I asked him.  He explained to me that 
'sync' at the command line issues the sync() system call, and if I read 
the sync(2) manpage carefully I'd find that sync() does /not/ cause the 
kernel's buffers to be written out to disk.  Instead, it tells the 
kernel to /schedule its buffers to be written out to disk as soon as the 
kernel can get around to it./  There /was/ no system call to tell the 
kernel to write its buffers out to disk right away and not return to the 
calling application until they were written out.  What happens if you 
run 'sync' more than once?  The first 'sync' tells the kernel to 
schedule the buffers to be written out.  Any subsequent 'sync' before 
the kernel has gotten around to writing the buffers just sets the "flush 
buffers to disk when the kernel has time" flag, that's already set to 
true, to true again. Any subsequent 'sync' after the kernel has flushed 
the buffers schedules any as-yet-unflushed buffers, of which there are 
now none, to be written to disk.  All syncs after the first one are 
effectively no-ops.

So, why type 'sync' three times?  Turns out, somebody at Berkeley had 
timed how long it took the average typist to type 'sync' and compared 
that to how long it typically took the kernel to finish the scheduled 
write operations.  They'd figured out that typing 'sync' three times 
gave you enough margin for error that you could have a high level of 
confidence that all kernel buffers would be written out to disk before 
the user finished typing.

The following is from the "Bugs" section of http://linux.die.net/man/2/sync:

    According to the standard specification (e.g., POSIX.1-2001), sync()
    schedules the writes, but may return before the actual writing is
    done. However, since version 1.3.20 Linux does actually wait. (This
    still does not guarantee data integrity: modern disks have large
    caches.)

                Mark Rosenthal