January 2011 - RAS-run

starting clean
by Thomas Krichel 18 Jan '11

18 Jan '11

RAS is down. aras@nebka:~$ cat /tmp/verified.log db4.6_verify /home/aras/acis/RI/data/citec/conflicts result: '0' db4.6_verify /home/aras/acis/RI/data/ACIS/conflicts result: '0' db4.6_verify /home/aras/acis/RI/data/RePEc/conflicts result: '0' db4.6_verify /home/aras/acis/RI/data/ACIS/records result: '0' db4.6_verify /home/aras/acis/RI/data/RePEc/records result: '0' db4.6_verify /home/aras/acis/RI/data/citec/history result: '0' db4.6_verify /home/aras/acis/RI/data/ACIS/history result: '0' db4.6_verify /home/aras/acis/RI/data/RePEc/history result: '0' db4.6_verify /home/aras/acis/RI/data/citec/files result: '0' db4.6_verify /home/aras/acis/RI/data/ACIS/files result: '0' db4.6_verify /home/aras/acis/RI/data/RePEc/files result: '0' aras@nebka:~$ crontab -l no crontab for aras no crontab aras@nebka:~$ ps axf | grep contro no rid aras@nebka:~$ rid start going to start RI daemon (/home/aras/acis/bin/control_daemon.pl) done aras@nebka:~$ rm acis/RI/update_ch* aras@nebka:~$ rm acis/RI/ri.log aras@nebka:~$ rid start going to start RI daemon (/home/aras/acis/bin/control_daemon.pl) done aras@nebka:~$ updareq RePEc / 99999999999999 I will be watching this as good as I can. I set of at 17:00 from FRA but I should be having wireless access there. I should check myself. Maybe I am also corrupted, by paranoia. Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorclaim.org/profile/pkr1 skype: thomaskrichel

2 3

new material not getting added
by Christian Zimmermann 15 Jan '11

15 Jan '11

I am getting complaints new material is not getting to RAS. EDIRC additions from Jan 8 are still not there. While they are in adrepec, they are not integrated in searchable data. Is there something that needs to be started for that? Christian Zimmermann FIGUGEGL! Department of Economics University of Connecticut 341 Mansfield Road, Unit 1063 Storrs, CT 06269-1063 http://ideas.repec.org/zimm/ christian.zimmermann(a)uconn.edu http://ideas.repec.org/e/pzi1.html

3 10

more on data loss
by Christian Zimmermann 12 Jan '11

12 Jan '11

I indicated earlier that a registration test with Jagdish Bhagwati yielded only 16 works instead on the expected 200+. I tried again, with the same result. However, with a manual search by name, I get the full list. How is this possible? So there is no data loss. Some functionality is not working right. Christian Zimmermann FIGUGEGL! Department of Economics University of Connecticut 341 Mansfield Road, Unit 1063 Storrs, CT 06269-1063 http://ideas.repec.org/zimm/ christian.zimmermann(a)uconn.edu http://ideas.repec.org/e/pzi1.html

2 1

disk is fine
by Christian Zimmermann 10 Jan '11

10 Jan '11

I rebooted the machine, and everything looks dandy. Disk use is down to 52%, so disk health or use is not the problem. I did a test by registering Jagdish Bhagwati (I withdrew before sending the confirmation email :)), a profilic author. The system found only 16 works. On IDEAS, I have 213... Christian Zimmermann FIGUGEGL! Department of Economics University of Connecticut 341 Mansfield Road, Unit 1063 Storrs, CT 06269-1063 http://ideas.repec.org/zimm/ christian.zimmermann(a)uconn.edu http://ideas.repec.org/e/pzi1.html

2 3

new facility
by Christian Zimmermann 10 Jan '11

10 Jan '11

I have started working on a request for new facilities for RAS. As I am not familiar with cloud computing, and Thomas suggested we use that, I have read some about it. I do not quite see why we would want to use that. My understanding is that its main benefit is to avoid buying and maintaining hardware. This is useful when 1) you do not have the means for big expenses, 2) you do not have the ability to maintain systems and 3) if your computing power needs are very variable over time. You can then avoid spending much for peak loads. I have little worry for 1) and 2), as the St. Louis Fed can buy and maintain. As for 3), our needs are much less cyclical than we think, and by doing it intelligently, can be completely smoothed by making automatic searches load dependent. Cloud computing may make more sense for the centrality project, but given that it uses the same data as RAS, it would actually make better sense to put it on the same machine as well. Hence, I am tending towards requesting a really powerful machine that could handle RAS and centrality, trying to figure out the specs needed for 2 million templates, 50000 authors and 1.2 million author claims, essentially double what we have now. I akm thinking of adding some essential new functionalities to RAS, but none would require any substantial coumputer resources. Comments? Christian Zimmermann FIGUGEGL! Department of Economics University of Connecticut 341 Mansfield Road, Unit 1063 Storrs, CT 06269-1063 http://ideas.repec.org/zimm/ christian.zimmermann(a)uconn.edu http://ideas.repec.org/e/pzi1.html

2 9

updareq does not work
by Christian Zimmermann 09 Jan '11

09 Jan '11

aras@nebka:~/acis/RI$ updareq RePEc aee/wpaper aras@nebka:~/acis/RI$ !ta tail update_ch0.log Fri Jan 7 17:44:34 2011 processed 1 in RePEc/aee/wpaper (aborted) Fri Jan 7 17:44:34 2011 time: 0 wallclock secs ( 0.00 usr + 0.00 sys = 0.00 CPU) Fri Jan 7 17:46:28 2011 request: source: /home/aras/acis/bin/updareq [20430] collection: RePEc update: aee/wpaper () Fri Jan 7 17:46:28 2011 Error: Can't get my hands on the DB in /home/aras/acis/RI/data: DB_RUNRECOVERY: Fatal error, run database recovery (File exists) at /home/aras/acis/lib/RePEc/Index/Storage/BDBwithTxn.pm line 53. Fri Jan 7 17:46:28 2011 processed aee/wpaper in RePEc (aborted) Fri Jan 7 17:46:28 2011 time: 0 wallclock secs ( 0.22 usr + 0.03 sys = 0.25 CPU) Christian Zimmermann FIGUGEGL! Department of Economics University of Connecticut 341 Mansfield Road, Unit 1063 Storrs, CT 06269-1063 http://ideas.repec.org/zimm/ christian.zimmermann(a)uconn.edu http://ideas.repec.org/e/pzi1.html

3 5

more corruptions
by Thomas Krichel 09 Jan '11

09 Jan '11

We have more corruptions. Seemingly randomly, processing stops with U DATAFILE_FINISH: r/u/ruben.durante(a)sciences-po.org.xml save record problem: / PANIC: fatal region error detected; run recovery Txn->commit returned positive! $! = '', BerkeleyDB::Error = 'PANIC: fatal region error detected; run recovery' at /home/aras/acis/lib/RePEc/Index/Storage/BDBwithTxn.pm line 91. U DATAFILE_START: r/u/russell.gorddard(a)csiro.au.xml I am at my wit's end with this. I have deleted more unrecoverable files, but starting with new one's I get more corruptions. The lock size increase seemingly did not fix things. I have to give up on this. Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorclaim.org/profile/pkr1 skype: thomaskrichel

1 2

OK AFAIK
by Christian Zimmermann 09 Jan '11

09 Jan '11

Everything looks OK so far. All cases that were causing trouble seem fixed, as far as I can see. Christian Zimmermann FIGUGEGL! Department of Economics University of Connecticut 341 Mansfield Road, Unit 1063 Storrs, CT 06269-1063 http://ideas.repec.org/zimm/ christian.zimmermann(a)uconn.edu http://ideas.repec.org/e/pzi1.html

2 1

Re: [RAS] Cron <aras@nebka> /usr/bin/db4.6_recover -h ~/acis/RI/data ; /usr/bin/db4.6_checkpoint -1 -h ~/acis/RI/data && /usr/bin/db4.6_archive -d -h ~/acis/RI/data
by Thomas Krichel 08 Jan '11

08 Jan '11

We again look to have a problem. I did only one software change, took citatation change out of APU to try rid us from the overload.... Cron Daemon writes > db4.6_recover: file unknown has LSN 493362/2698220, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2698220, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2696775, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2696775, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2132591, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2132591, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2698220, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2698220, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2132591, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2132591, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2698220, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2698220, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2132591, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_recover: file unknown has LSN 493362/2132591, past end of log at 493362/1823218 > db4.6_recover: Commonly caused by moving a database from one database environment > db4.6_recover: to another without clearing the database LSNs, or by removing all of > db4.6_recover: the log files from a database environment > db4.6_checkpoint: /home/aras/acis/RI/data/ACIS/records: unable to flush: No such file or directory > db4.6_checkpoint: txn_checkpoint: failed to flush the buffer cache: No such file or directory > db4.6_checkpoint: txn_checkpoint: No such file or directory -- Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorclaim.org/profile/pkr1 skype: thomaskrichel

2 2

bdb files deleted
by Thomas Krichel 07 Jan '11

07 Jan '11

We have three corrupt bdbs, I deleted two files. I think the site should now be operationall, but please test drive then set back the DNS. Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorclaim.org/profile/pkr1 skype: thomaskrichel

2 2