Here is what I suggest once the backup is done: Tim, give ssh access back, so that we can check what happened. Ivan and/or Thomas, have a look at logs Thomas, slice up any rsync so that it does not synchronize large file structures. I strongly suspect that was the problem. Saturday, I will be on my way back to the US. I should be able to get back online at 8:00pm EST. I will be back on campus Monday, and can go in the machine room if necessary and thus relieve Tim from this. Christian Zimmermann FIGUGEGL! Department of Economics University of Connecticut 341 Mansfield Road, Unit 1063 Storrs, CT 06269-1063 http://ideas.repec.org/zimm/ christian.zimmermann@uconn.edu http://ideas.repec.org/e/pzi1.html
Christian Zimmermann writes
Here is what I suggest once the backup is done:
Tim, give ssh access back, so that we can check what happened.
Ivan and/or Thomas, have a look at logs
Thomas, slice up any rsync so that it does not synchronize large file structures. I strongly suspect that was the problem.
I don't think that rsync was the problem. Do you have evidence that rsync causes machines to go down? Surely last time it did, but the problem was not rsync. the problem was JMBC shoehorning 100000 files into a single directory. If you had transfered these with ftp rather than with rsync, you would have gotten the same trouble. I hope that your theory is right that having large directories is the problem. In that case, as you wrote, we need to get a larger disk. Bob send you a disk for nebka. You diverted it to IDEAS. Before we proceed, put that disk into nebka. Or if you can't do that, go to the shop, buy a 500 Gig PATA disk for $150 bucks. Then ask Bob, he seems to know about a device that can case these disks and has a USB interface, connect via USB. Buy that device, put the disk into it and hook it up to the machine. We then move /home off to the new disk. To move /var it is best to work in single user mode (second entry on the grub screen). We can do that later in a scheduled downtime. Cheers, Thomas Krichel http://openlib.org/home/krichel RePEc:per:1965-06-05:thomas_krichel phone: +7 383 330 6813 skype: thomaskrichel
If IDEAS can not spare that drive soon, I can take down my machine and ship another 146 gig (which is used as temp on a large job and I am not currently running that) but I would need it back, say in March. Let me know, I can see about shipping it Monday. The USB to IDE or SATA http://www.geeks.com/details.asp?invtid=USB2IDE-25-35&cpc=SCH http://www.geeks.com/details.asp?invtid=2020-OTB&cpc=SCH http://www.geeks.com/details.asp?invtid=2020&cpc=SCH I don't think there is any difference in the latter two, and I have used them on SATA drives (or similar, maybe not that exact same one). You can also find cases at nearly the same price http://www.geeks.com/details.asp?invtid=VP-62281&cat=CAS Those are examples - I have purchased from geeks.com in the past. Their inventory changes daily - two weeks ago, they had very few and I ordered from an ebay site which had low prices and high shipping. Bob Thomas Krichel wrote:
Christian Zimmermann writes
Here is what I suggest once the backup is done:
Tim, give ssh access back, so that we can check what happened.
Ivan and/or Thomas, have a look at logs
Thomas, slice up any rsync so that it does not synchronize large file structures. I strongly suspect that was the problem.
I don't think that rsync was the problem. Do you have evidence that rsync causes machines to go down? Surely last time it did, but the problem was not rsync. the problem was JMBC shoehorning 100000 files into a single directory. If you had transfered these with ftp rather than with rsync, you would have gotten the same trouble.
I hope that your theory is right that having large directories is the problem. In that case, as you wrote, we need to get a larger disk.
Bob send you a disk for nebka. You diverted it to IDEAS. Before we proceed, put that disk into nebka. Or if you can't do that, go to the shop, buy a 500 Gig PATA disk for $150 bucks. Then ask Bob, he seems to know about a device that can case these disks and has a USB interface, connect via USB. Buy that device, put the disk into it and hook it up to the machine. We then move /home off to the new disk. To move /var it is best to work in single user mode (second entry on the grub screen). We can do that later in a scheduled downtime.
Cheers,
Thomas Krichel http://openlib.org/home/krichel RePEc:per:1965-06-05:thomas_krichel phone: +7 383 330 6813 skype: thomaskrichel
_______________________________________________ RAS-run mailing list RAS-run@lists.openlib.org http://lists.openlib.org/cgi-bin/mailman/listinfo/ras-run
-- Bob Parks bparks@wustl.edu Department of Economics, Campus Box 1208 Washington University One Brookings Drive St. Louis, Missouri 63130-4899 Voice: (314) 935-5665 Fax: (314) 935-4156
I completely agree. The new machine should have been up by now, with a full backup of nebka, But being on campus only 4 days in the last six weeks did not help. I will do whatever is possible to get this done asap. Christian Zimmermann FIGUGEGL! Department of Economics University of Connecticut 341 Mansfield Road, Unit 1063 Storrs, CT 06269-1063 http://ideas.repec.org/zimm/ christian.zimmermann@uconn.edu http://ideas.repec.org/e/pzi1.html On Sat, 26 Jan 2008, Thomas Krichel wrote:
Christian Zimmermann writes
Here is what I suggest once the backup is done:
Tim, give ssh access back, so that we can check what happened.
Ivan and/or Thomas, have a look at logs
Thomas, slice up any rsync so that it does not synchronize large file structures. I strongly suspect that was the problem.
I don't think that rsync was the problem. Do you have evidence that rsync causes machines to go down? Surely last time it did, but the problem was not rsync. the problem was JMBC shoehorning 100000 files into a single directory. If you had transfered these with ftp rather than with rsync, you would have gotten the same trouble.
I hope that your theory is right that having large directories is the problem. In that case, as you wrote, we need to get a larger disk.
Bob send you a disk for nebka. You diverted it to IDEAS. Before we proceed, put that disk into nebka. Or if you can't do that, go to the shop, buy a 500 Gig PATA disk for $150 bucks. Then ask Bob, he seems to know about a device that can case these disks and has a USB interface, connect via USB. Buy that device, put the disk into it and hook it up to the machine. We then move /home off to the new disk. To move /var it is best to work in single user mode (second entry on the grub screen). We can do that later in a scheduled downtime.
Cheers,
Thomas Krichel http://openlib.org/home/krichel RePEc:per:1965-06-05:thomas_krichel phone: +7 383 330 6813 skype: thomaskrichel
participants (3)
-
Bob Parks -
Christian Zimmermann -
Thomas Krichel