Christian Zimmermann writes
Here is what I suggest once the backup is done:
Tim, give ssh access back, so that we can check what happened.
Ivan and/or Thomas, have a look at logs
Thomas, slice up any rsync so that it does not synchronize large file structures. I strongly suspect that was the problem.
I don't think that rsync was the problem. Do you have evidence that rsync causes machines to go down? Surely last time it did, but the problem was not rsync. the problem was JMBC shoehorning 100000 files into a single directory. If you had transfered these with ftp rather than with rsync, you would have gotten the same trouble. I hope that your theory is right that having large directories is the problem. In that case, as you wrote, we need to get a larger disk. Bob send you a disk for nebka. You diverted it to IDEAS. Before we proceed, put that disk into nebka. Or if you can't do that, go to the shop, buy a 500 Gig PATA disk for $150 bucks. Then ask Bob, he seems to know about a device that can case these disks and has a USB interface, connect via USB. Buy that device, put the disk into it and hook it up to the machine. We then move /home off to the new disk. To move /var it is best to work in single user mode (second entry on the grub screen). We can do that later in a scheduled downtime. Cheers, Thomas Krichel http://openlib.org/home/krichel RePEc:per:1965-06-05:thomas_krichel phone: +7 383 330 6813 skype: thomaskrichel