August 2011 - RAS-run

Re: [RAS] [ACIS-tech] Migration Help
by Ivan Kurmanov 04 Sep '11

04 Sep '11

Dan, The new machine needs to share the RePEc data files with the Ideas server. Have you done anything for that? Sshfs maybe? Also, did you set up the mailing server there? I'm thinking, maybe we could even do the last checks tomorrow and migrate on Thursday... -ivan On Aug 30, 2011 8:27 PM, "Dan Hayes" <dhayes501(a)gmail.com> wrote: > I'm fine waiting until Sept 10-11. It will likely take more downtime > though, it took quite a while to dump the database (less than an hour though > I think), and then it took several hours to transfer it here. Then it took > 8+ hours to load it on the machine here. > > On Tue, Aug 30, 2011 at 11:37 AM, Ivan Kurmanov <duraley(a)gmail.com> wrote: > >> On Tue, Aug 30, 2011 at 7:22 PM, Dan Hayes <dhayes501(a)gmail.com> wrote: >> > It looks like it will take editing /etc/hosts to access the web >> interface. >> > Set authors.repec.org to 65.89.18.160. >> > >> > Also, the Affiliations section appears to be working fine now. >> >> I witness. >> >> Which means the Storable strings that were written (with nfreeze) on >> the old nebka are readable on the new one, and we do not need any >> other serialization changes at the moment. >> >> >> >>> Dan, did you measure or record the time it took to restore the >> database? >> >> It took a solid 8 hours. I didn't time it exactly though. I watched >> the >> >> size of /var/lib/mysql and it grew to over 35G, acis is 32G and rdb is >> 4.2G >> >> Then it looks like we should plan for at least 10-12 hours of downtime >> of the service for the final migration. >> >> From the point of view of the least user experience damage (and brand >> damage), it would be great to do the migration over the weekend. But >> the nearest weekend (3-4 Sep) i'm occupied except at night, when I'd >> like to have some sleep. The weekend after (10-11 Sep) i'm available. >> >> Or we can ignore the weekends and plan the transition on a business >> day. I guess Dan will do the large part of the work again, so his >> availability is key here. I'm pretty flexible at the moment. >> >> >> And of course, there's more tests and checks to be done. >> >> -i >> >> > >> > On Tue, Aug 30, 2011 at 10:19 AM, Dan Hayes <dhayes501(a)gmail.com> wrote: >> >> >> >> It took a solid 8 hours. I didn't time it exactly though. I watched >> the >> >> size of /var/lib/mysql and it grew to over 35G, acis is 32G and rdb is >> 4.2G >> >> >> >> On Tue, Aug 30, 2011 at 10:05 AM, Ivan Kurmanov <duraley(a)gmail.com> >> wrote: >> >>> >> >>> Dan, did you measure or record the time it took to restore the >> database? >> >>> >> >>> -ivan >> >>> >> >>> On Tue, Aug 30, 2011 at 5:41 PM, Dan Hayes <dhayes501(a)gmail.com> >> wrote: >> >>> > The SQL database has been loaded, and it appears to be running >> >>> > properly. >> >>> > You can test it at 65.89.18.160 or at http://nebka.stlouisfed.org. >> >>> > >> >>> > Did I not need to run the updareq commands again? It appears to be >> >>> > working >> >>> > without me needing to do anything else. >> >>> > >> >>> > On Mon, Aug 29, 2011 at 10:16 AM, Dan Hayes <dhayes501(a)gmail.com> >> >>> > wrote: >> >>> >> >> >>> >> FYI >> >>> >> >> >>> >> I rsynced the files Saturday afternoon, and dumped the sql database >> >>> >> then >> >>> >> as well. I transferred the db backup on Sunday, and I'm now >> restoring >> >>> >> the >> >>> >> sql databases on the STL server. It looks like its going to take a >> >>> >> while. >> >>> >> >> >>> >> On Sun, Aug 28, 2011 at 9:22 AM, Thomas Krichel < >> krichel(a)openlib.org> >> >>> >> wrote: >> >>> >>> >> >>> >>> Dan Hayes writes >> >>> >>> >> >>> >>> > So the reason is that its faster than MySQL for what its being >> used >> >>> >>> > for? >> >>> >>> >> >>> >>> Yes. At least we think so. >> >>> >>> >> >>> >>> Kindly send me an ssh public key to give you access to the test >> >>> >>> server. >> >>> >>> >> >>> >>> >> >>> >>> Cheers, >> >>> >>> >> >>> >>> Thomas Krichel http://openlib.org/home/krichel >> >>> >>> http://authorprofile.org/pkr1 >> >>> >>> skype: thomaskrichel >> >>> >>> >> >>> >>> _______________________________________________ >> >>> >>> ACIS-tech mailing list >> >>> >>> ACIS-tech(a)lists.openlib.org >> >>> >>> http://lists.openlib.org/mailman/listinfo/acis-tech >> >>> >> >> >>> > >> >>> > >> >>> > _______________________________________________ >> >>> > ACIS-tech mailing list >> >>> > ACIS-tech(a)lists.openlib.org >> >>> > http://lists.openlib.org/mailman/listinfo/acis-tech >> >>> > >> >>> >> >>> _______________________________________________ >> >>> ACIS-tech mailing list >> >>> ACIS-tech(a)lists.openlib.org >> >>> http://lists.openlib.org/mailman/listinfo/acis-tech >> > >> > >> > _______________________________________________ >> > ACIS-tech mailing list >> > ACIS-tech(a)lists.openlib.org >> > http://lists.openlib.org/mailman/listinfo/acis-tech >> > >> >> _______________________________________________ >> ACIS-tech mailing list >> ACIS-tech(a)lists.openlib.org >> http://lists.openlib.org/mailman/listinfo/acis-tech >>

4 26

something wrong
by 'Christian Zimmermann' 31 Aug '11

31 Aug '11

There are no template updates being written to the ftp site on nebka-uconn for about a day or so. -- Christian Zimmermann FIGUGEGL! Economic Research Federal Reserve Bank of St. Louis P.O. Box 442 St. Louis MO 63166-0442 USA http://ideas.repec.org/zimm/

1 0

Fwd: [ACIS-tech] Migration Help
by Ivan Kurmanov 30 Aug '11

30 Aug '11

---------- Forwarded message ---------- From: Ivan Kurmanov <duraley(a)gmail.com> Date: Tue, Aug 30, 2011 at 7:37 PM Subject: Re: [ACIS-tech] Migration Help To: ACIS technical list <acis-tech(a)lists.openlib.org> On Tue, Aug 30, 2011 at 7:22 PM, Dan Hayes <dhayes501(a)gmail.com> wrote: > It looks like it will take editing /etc/hosts to access the web interface. > Set authors.repec.org to 65.89.18.160. > > Also, the Affiliations section appears to be working fine now. I witness. Which means the Storable strings that were written (with nfreeze) on the old nebka are readable on the new one, and we do not need any other serialization changes at the moment. >>> Dan, did you measure or record the time it took to restore the database? >> It took a solid 8 hours. I didn't time it exactly though. I watched the >> size of /var/lib/mysql and it grew to over 35G, acis is 32G and rdb is 4.2G Then it looks like we should plan for at least 10-12 hours of downtime of the service for the final migration. >From the point of view of the least user experience damage (and brand damage), it would be great to do the migration over the weekend. But the nearest weekend (3-4 Sep) i'm occupied except at night, when I'd like to have some sleep. The weekend after (10-11 Sep) i'm available. Or we can ignore the weekends and plan the transition on a business day. I guess Dan will do the large part of the work again, so his availability is key here. I'm pretty flexible at the moment. And of course, there's more tests and checks to be done. -i > > On Tue, Aug 30, 2011 at 10:19 AM, Dan Hayes <dhayes501(a)gmail.com> wrote: >> >> It took a solid 8 hours. I didn't time it exactly though. I watched the >> size of /var/lib/mysql and it grew to over 35G, acis is 32G and rdb is 4.2G >> >> On Tue, Aug 30, 2011 at 10:05 AM, Ivan Kurmanov <duraley(a)gmail.com> wrote: >>> >>> Dan, did you measure or record the time it took to restore the database? >>> >>> -ivan >>> >>> On Tue, Aug 30, 2011 at 5:41 PM, Dan Hayes <dhayes501(a)gmail.com> wrote: >>> > The SQL database has been loaded, and it appears to be running >>> > properly. >>> > You can test it at 65.89.18.160 or at http://nebka.stlouisfed.org. >>> > >>> > Did I not need to run the updareq commands again? It appears to be >>> > working >>> > without me needing to do anything else. >>> > >>> > On Mon, Aug 29, 2011 at 10:16 AM, Dan Hayes <dhayes501(a)gmail.com> >>> > wrote: >>> >> >>> >> FYI >>> >> >>> >> I rsynced the files Saturday afternoon, and dumped the sql database >>> >> then >>> >> as well. I transferred the db backup on Sunday, and I'm now restoring >>> >> the >>> >> sql databases on the STL server. It looks like its going to take a >>> >> while. >>> >> >>> >> On Sun, Aug 28, 2011 at 9:22 AM, Thomas Krichel <krichel(a)openlib.org> >>> >> wrote: >>> >>> >>> >>> Dan Hayes writes >>> >>> >>> >>> > So the reason is that its faster than MySQL for what its being used >>> >>> > for? >>> >>> >>> >>> Yes. At least we think so. >>> >>> >>> >>> Kindly send me an ssh public key to give you access to the test >>> >>> server. >>> >>> >>> >>> >>> >>> Cheers, >>> >>> >>> >>> Thomas Krichel http://openlib.org/home/krichel >>> >>> http://authorprofile.org/pkr1 >>> >>> skype: thomaskrichel >>> >>> >>> >>> _______________________________________________ >>> >>> ACIS-tech mailing list >>> >>> ACIS-tech(a)lists.openlib.org >>> >>> http://lists.openlib.org/mailman/listinfo/acis-tech >>> >> >>> > >>> > >>> > _______________________________________________ >>> > ACIS-tech mailing list >>> > ACIS-tech(a)lists.openlib.org >>> > http://lists.openlib.org/mailman/listinfo/acis-tech >>> > >>> >>> _______________________________________________ >>> ACIS-tech mailing list >>> ACIS-tech(a)lists.openlib.org >>> http://lists.openlib.org/mailman/listinfo/acis-tech > > > _______________________________________________ > ACIS-tech mailing list > ACIS-tech(a)lists.openlib.org > http://lists.openlib.org/mailman/listinfo/acis-tech >

3 2

Fwd: Cron <aras@nebka> cd /home/aras/acis && /home/aras/acis/bin/make-repec-per.sh
by Ivan Kurmanov 29 Aug '11

29 Aug '11

Christian, do you find these messages useful in any way? -i ---------- Forwarded message ---------- From: Cron Daemon <root(a)authors.repec.org> Date: 2011/8/28 Subject: Cron <aras@nebka> cd /home/aras/acis && /home/aras/acis/bin/make-repec-per.sh To: aras(a)authors.repec.org -/home/aras/RePEc-per-pers/y/pya293.rdf -/home/aras/RePEc-per-pers/p/ppi256.rdf -/home/aras/RePEc-per-pers/c/pch502.rdf

2 1

Re: [RAS] RAS error log 2011/08/20
by Thomas Krichel 29 Aug '11

29 Aug '11

RAS/ACIS writes > [Sun Jan 09 09:11:01 2011] [error] [client 79.85.158.184] Paging sanity problem:, referer: http://authors.repec.org/research/search!95790eab?page=16 > [Sun Jan 09 09:11:01 2011] [error] [client 79.85.158.184] results: 8, referer: http://authors.repec.org/research/search!95790eab?page=16 140000 lines more of this. Christian, do you get these messages? They come every day to my desk. Sholud they be sent to Dan, you, Ivan? Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorprofile.org/pkr1 skype: thomaskrichel

3 3

possible nebka outage
by 'Christian Zimmermann' 27 Aug '11

27 Aug '11

Hurricane Irene is heading towards Connecticut, and campus authorities cannot guarantee there will be power on campus Sunday and Monday depending on how things turn out. I suggest we leave the machine on, after all there is a good chance it will not shut off. But I suggest that no big job run on Sunday and Monday, so that we do not need to receover from a big crash. If there are issues, they should be visible at http://itstatus.uconn.edu/ and http://alert.uconn.edu/ -- Christian Zimmermann FIGUGEGL! Economic Research Federal Reserve Bank of St. Louis P.O. Box 442 St. Louis MO 63166-0442 USA http://ideas.repec.org/zimm/

2 1

Re: [RAS] Migration Help
by Thomas Krichel 26 Aug '11

26 Aug '11

Ivan Kurmanov writes > please copy the /home/aras/ directory over, and then dump the mysql > databases "rdb" and "acis". I suggest not to do this. Erase the databases you already have and start anew. > Then tomorrow (after 1am EDT this night) copy over the > /home/aras/backup/2011/08/26 directory. > > That has update_db.tgz archive, which contains the RI directory. If > unpacked while in ~/acis/, it would write the needed files into > ~/acis/RI/data/... directory, and that is a safe way to move the data. I woulld be surprised if he can read the data in BDB, but I have eaten my words before. > > Ideally, the mysql dump and RI/data dump would be created at about the > same time with all data-modifying services taken down. But since this > is not a final move, it is okay to do as written above. When we migrate, we should do it the ACIS way. rsync the ReDIF data files and the AMF CitEC files annd the ~/acis/userdata, as they are, and run the update daemon. Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorprofile.org/pkr1 skype: thomaskrichel

3 2

Re: [RAS] Migration Help
by Thomas Krichel 26 Aug '11

26 Aug '11

Ivan Kurmanov writes > I've been working on nebka last week or more with RAS database. > > The first thing i did: I modified the ACIS code to use Storable's > nfreeze() function when storing the data into the db. Also at that > time I've found that part of the ACIS code on nebka was already using > nfreeze(). > > Than I've written a test script which runs through the specified > database tables (mysql) and checks their values for being written with > Storable correctly. > > Then the last 5 days or so i was working on the upgrade daemon > database (Berkeley DB) of RAS on nebka. It also contains > Storable-encoded strings, and is important. First, i've written a > script (actually, a version of the same script mentioned above), to > check the values in it (by full scanning, and checking each). That > found some issues. > > In fact, that checking has found some serious data corruption, where > some keys where mixed with values and some values being hopelessly > broken. I did suspect things where not ok because when I do fewer update (timout more then the default 1 week) things are not updated properly. I got serious complaints from CZ about it. I wanted to reduce the load on the box by setting a higher TOO_OLD. > Second, I've written a script to correct values which need correction > (via nfreeze) and to remove the ones that cannot be corrected. > > That work is now done. BUT at least a part of the update daemon > database is still corrupted there. The good news is that this part is > not RePEc-part and it would not (should not) cause any trouble to > rebuild it. The bad news is that it is corrupted on the internal > Berkeley DB level and I do not know how to fix it. The corruption is > reported by the db4.6_verify tool. Berkeley DB documentation is not > clear on how to fix it, or I'm not looking hard enough. I do a db_dump foo | db_load foo_clean when I am desparate. On PubMed 20 million records, that takes several days to do. I have been desparete many times. > > Which leads me to the whole other topic: in the longer run we should > avoid Berkeley DB and use something else instead. Luckily, this > shouldn't be too hard to do, since all the BerkleyDB-related code is > concentrated in a couple of modules or so. And there are alternatives. > Kyoto Cabinet http://fallabs.com/kyotocabinet/ being one of them. or mongo or couch.... I have had *tons* of trouble with BDB. I hate it. But still I don't think we should work on this now. CZ will kick our buts. You need to look at the ACIS code to find where the problem is. Blaming them on BDB is not the way forward I think. > Now, I've moved out the corrupted BDB file ~/acis/RI/data/ACIS/records > (renamed it). > > I think the data is now ready to be migrated to the new server > (again). The changes in ~/acis/ need to be copied over too, as well as > the (partially) fixed ~/acis/RI/data contents. Dan I suggest you to do > that. > I think he should not do this at all. Instead he should start with the text data, and the most recent ACIS release, and build a clear dataset. I have done this today and I will forward my notes on it. > It may be a good idea to wait for the nearest run of the nightly > script to create the RI database snapshots in ~/backup/2011/08/26 and > to use those. > > > With these data and code changes, it should all run on the new server, > but we'll do our tests and checks as soon as it is copied. > > Any comments? Questions? I bet that this will not work. Dan will still not be able to read your storables, and he will not read his own storables when he updates perl. Get rid of Storable. Keep BDB, for now. Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorprofile.org/pkr1 skype: thomaskrichel

2 1

setting up a new RAS
by Thomas Krichel 26 Aug '11

26 Aug '11

With the latest ACIS version. I have started this today. I found some problems that I have fixed. So some of the notes are not completely applicable, but it sohuld be enough to reproduce the process. It's not complete, but it should Dan get started. Dan I suggest you try to use the latest BDB version you can get. This will create a new set and not use the corrupted ones we have on nebka. I will not get offline, get drunk and chase women, but if not successful (rather likely) you can find me online later to talk. Installation notes from test system aras(a)holda.openlib.org 1. Make DNS entry ras.openlib.org to go to holda binder@snefru:~$ nslookup ras.openlib.org Server: 127.0.0.1 Address: 127.0.0.1#53 Name: ras.openlib.org Address: 85.10.194.76 binder@snefru:~$ nslookup holda.openlib.org Server: 127.0.0.1 Address: 127.0.0.1#53 Name: holda.openlib.org Address: 85.10.194.76 2. Install libsvm-tools aptitude install libsvm-tools 3. Mirror acis from nebka (without RI directory, so save space). Make a copy of acis, as mirrored from nebka aras@holda:~$ mv acis/ acis.org aras@holda:~$ mkdir acis 4. Install. Note I made already a new release, so the number will be different here. aras@holda:~$ cp ~aacis/public_html/code/ACIS-1.6.50-20110826a.with-libs.tar.gz . aras@holda:~$ cp ~aacis/public_html/code/ACIS-1.6.50-20110826a.with-libs.tar.gz . aras@holda:~$ tar xzf ACIS-1.6.50-20110826a.with-libs.tar.gz aras@holda:~$ cd ACIS-1.6.50-20110826a.with-libs aras@holda:~/ACIS-1.6.50-20110826a.with-libs$ sh install.sh ~/acis this is an upgrade Checking required modules... - CGI::Untaint (version 1.25+) is not found - String::Similarity (version 1+) is not found - AppConfig (version 1.52+) is not found - Date::Manip is not found - Proc::Daemon (version 0.03+) is not found - CGI::Minimal (version 1.24+) is not found - String::Approx is not found installing to /home/aras/acis directory installing scripts into /home/aras/acis/bin: cit_test.pl conf.pl rid setup setup.cgi_frontend setup.logs-browsing setup.ri_collections setup.ri_local_setup.pm setup.sid_local.pm this is an upgrade installing to /home/aras/acis directory /home/aras/acis/bin/setup: line 11: cd: /home/awho/acis: No such file or directory You need to have AppConfig perl module installed unpacked AMF-perl into dir AMF-perl-0.40 AMF-perl-0.40.tar.gz Now you might want to create main.conf file in /home/aras/acis Then run bin/setup there. There is a mistake on line 11 of setup. I am not sure what should replace the line homedir=/home/awho/acis to homedir=~/acis aacis@holda:~/dev/acis$ darcs whatsnew hunk ./home/bin/setup 3 -homedir=/home/awho/acis +homedir=~/acis Install what I can from Debian root@holda ~ # aptitude install libcgi-untaint-perl libappconfig-perl libdate-manip-perl libproc-daemon-perl libstring-approx-perl The following NEW packages will be installed: libappconfig-perl libcgi-untaint-perl libdate-manip-perl libproc-daemon-perl libstring-approx-perl libuniversal-require-perl{a} libyaml-syck-perl{a} 0 packages upgraded, 7 newly installed, 0 to remove and 0 not upgraded. Need to get 2,949 kB of archives. After unpacking 16.4 MB will be used. Two packages need to be installed manually. I use ~/usr as prefix. The versions can be downloaded from http://acis.openlib.org/code/extra_perl Since I am running on holda, which does acis.openlib.org and ras.openlib.org, I can just copy cd /tmp/ tar -xf /home/aacis/public_html/extra_perl/CGI-Minimal-1.29.tar.gz cd CGI-Minimal-1.29/ perl Makefile.PL PREFIX=~/usr make make test make install cd .. tar -xf /home/aacis/public_html/extra_perl/String-Similarity-1.04.tar.gz cd String-Similarity-1.04/ perl Makefile.PL PREFIX=~/usr make make test make install cd now get main.conf from the orginal aras@holda:~/acis$ cp ~/acis.org/main.conf main.conf then aras@holda:~/acis$ setup fails because the CGI::Minimal not found. Crap. Some modules are in share, others in lib. aras@holda:~/usr$ echo $PERL5LIB /home/aras/usr/lib/perl:/home/aras/acis/lib:/home/aras/acis/lib/ARDB aras@holda:~/usr$ find lib lib lib/perl lib/perl/5.12.4 lib/perl/5.12.4/String lib/perl/5.12.4/String/Similarity.pm lib/perl/5.12.4/perllocal.pod lib/perl/5.12.4/auto lib/perl/5.12.4/auto/String lib/perl/5.12.4/auto/String/Similarity lib/perl/5.12.4/auto/String/Similarity/.packlist lib/perl/5.12.4/auto/String/Similarity/Similarity.so lib/perl/5.12.4/auto/String/Similarity/Similarity.bs lib/perl/5.12.4/auto/CGI lib/perl/5.12.4/auto/CGI/Minimal lib/perl/5.12.4/auto/CGI/Minimal/.packlist aras@holda:~/usr$ find share share share/perl share/perl/5.12.4 share/perl/5.12.4/CGI share/perl/5.12.4/CGI/Minimal.pm share/perl/5.12.4/CGI/Minimal share/perl/5.12.4/CGI/Minimal/Multipart.pm share/perl/5.12.4/CGI/Minimal/Misc.pm share/perl/5.12.4/CGI/Minimal.pod The way I deal with this is to make "share" an alias for "lib". aras@holda:~/usr$ rsync -va share/ lib sending incremental file list perl/5.12.4/ perl/5.12.4/CGI/ perl/5.12.4/CGI/Minimal.pm perl/5.12.4/CGI/Minimal.pod perl/5.12.4/CGI/Minimal/ perl/5.12.4/CGI/Minimal/Misc.pm perl/5.12.4/CGI/Minimal/Multipart.pm sent 42452 bytes received 101 bytes 85106.00 bytes/sec total size is 42092 speedup is 0.99 aras@holda:~/usr$ rm -rf share aras@holda:~/usr$ ln -s lib share We still need to install additonal modules. cd cd ACIS-1.6.50-20110826a.with-libs ls -lt cd AMF-perl-0.40 perl Makefile.PL PREFIX=~/usr make test make install cd .. ls cd RePEc-Index-0.38.3 perl Makefile.PL PREFIX=~/usr make make test Some tests fail because of the use of plan and because Common::Data is not found. These failures are harmless. The Common::Data problem is fixed in the update. Install the ReDIF reader aras@holda:~/ACIS-1.6.50-20110826a.with-libs$ tar xzf ReDIF-perl-2.65.tar.gz aras@holda:~/ACIS-1.6.50-20110826a.with-libs$ cd ReDIF-perl-2.65 aras@holda:~/ACIS-1.6.50-20110826a.with-libs/ReDIF-perl-2.65$ perl Makefile.PL PREFIX=~/usr Preparing to install ReDIF perl package. Your system has Digest::MD5 module installed. Do you want to enable MD5 checksum calculation for templates? [yes] Do you want ReDIF-perl to check attribute values for a reasonable length? (early implementation) [no] ReDIF home directory is where you store your ReDIF data, your configuration files for ReDIF-related programs and other related stuff. Some programs require it for operation, but FOR ReDIF-perl INSTALLATION IT IS NOT REQUIRED. You can set it now and ReDIF-perl will use it always BY DEFAULT. Otherwise you will have to pass the path to ReDIF home by other means to some programs (e.g. through environment variable REDIFDIR). Do you wish to set default ReDIF home directory now? [yes] What directory to use as the default ReDIF home? [enter for not setting it now] Saved lib/ReDIF/setup_conf.pm. Current configuration: ReDIF home directory: not set Checking if your kit is complete... Looks good Writing Makefile for ReDIF-perl aras@holda:~/ACIS-1.6.50-20110826a.with-libs/ReDIF-perl-2.65$ make ... aras@holda:~/ACIS-1.6.50-20110826a.with-libs/ReDIF-perl-2.65$ make tast tests fail because of plan, considered harmless aras@holda:~/ACIS-1.6.50-20110826a.with-libs/ReDIF-perl-2.65$ make install aras@holda:~/acis$ setup fails because the Data::Common is not found. I find it's a problem in the MANIFEST file, that is fixed in a newer version. aacis@holda:~/dev/acis$ darcs whatsnew hunk ./MANIFEST 287 +lib/ACIS/Resources/Learn.pm +lib/ACIS/Resources/Learn/KnownItems.pm +lib/ACIS/Resources/Learn/Suggested.pm hunk ./MANIFEST 352 +lib/Common/Data.pm make a new version at holda perl build/dist.pl -f 1.6.50 ls -lt rm ACIS-1.6.50-20110826a.with-libs.tar.gz rm ~/public_html/code/ACIS-1.6.50-20110826a.with-libs.tar.gz mv ACIS-1.6.50-20110826b.with-libs.tar.gz ~/public_html/code/. on the test account, start da capo aras@holda:~$ rm -rf ACIS-1.6.50-20110826a.with-libs* I can untar this because repository and test are running on same box aras@holda:~$ tar xzf ~aacis/public_html/code/ACIS-1.6.50-20110826b.with-libs.tar.gz When the module RAS.pl is requested, comment this out in acis/main.conf # /meta/update interface #require-modules=RAS.pl #meta-update-clients="wop(a)128.252.177.191" #meta-update-object-fetch-func=RePEc::RAS::object_fetch You should now, on setup, see something like aras@holda:~/acis$ setup installing additional scripts into /home/aras/acis/bin: apu arpu backup_tables clean-up create_tables dump_email_handle_table events_archiving export_cit_events export_fturls_choices export_name_variations fuzzy_search_table mvrec offline_maintain recover_citation_events recreate_profile_pages reparse_config restore_table_backup rmrec sanitar2 sanitar3 sanitar sanitize_cit_suggestions sanitize_names_table sanitize_redif_files setup_ardb setup_profile_menu sid_base updareq upgrade upgrade_to_1.01 upgrade_to_1.02 upgrade_to_1.04 upgrade_to_1.0 upgrade_to_1.4.111 upgrade_to_1.4.61 upgrade_to_1.4.90 upgrade_to_1.4.95 upgrade_to_1.5.13 upgrade_to_1.5.27 upgrade_to_1.5.34 upgrade_to_1.5.3 upgrade_to_1.5.4 upgrade_to_1.5.53 upgrade_to_1.5.58 upgrade_to_1.5.64 upgrade_to_1.5.70 upgrade_to_1.5.7 upgrade_to_1.6.1 archive_critical writing /home/aras/WWW/cgi/a.cgi, obsolete /home/aras/acis/bin/setup: line 133: /home/aras/WWW/cgi/a.cgi: No such file or directory chmod: cannot access `/home/aras/WWW/cgi/a.cgi': No such file or directory cp: cannot stat `/home/aras/acis/presentation/ras/style': No such file or directory cp: cannot stat `/home/aras/acis/presentation/ras/script': No such file or directory installing log browsing script to /home/aras/WWW/cgi/gol.cgi /home/aras/acis/bin/setup: line 171: /home/aras/WWW/cgi/gol.cgi: No such file or directory chmod: cannot access `/home/aras/WWW/cgi/gol.cgi': No such file or directory DBI connect('rdb','corner',...) failed: Access denied for user 'corner'@'localhost' (using password: YES) at /home/aras/acis/lib/sql_helper.pm line 260 Can't call method "do" without a package or object reference at /home/aras/acis/lib/ACIS/ShortIDs.pm line 357. configuring ARDB for home dir DBI connect('rdb','corner',...) failed: Access denied for user 'corner'@'localhost' (using password: YES) at /home/aras/acis/lib/sql_helper.pm line 260 Can't establish database connection at /home/aras/acis/lib/ARDB/Setup.pm line 75. Can't locate ARDB/Local.pm in @INC (@INC contains: /home/aras/acis/lib /home/aras/usr/lib/perl/5.12.4 /home/aras/usr/lib/perl /home/aras/acis/lib/ARDB /etc/perl /usr/local/lib/perl/5.12.4 /usr/local/share/perl/5.12.4 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.12 /usr/share/perl/5.12 /usr/local/lib/site_perl /usr/local/lib/perl/5.12.3 /usr/local/share/perl/5.12.3 .) at bin/create_tables line 14. BEGIN failed--compilation aborted at bin/create_tables line 14. configuring ACIS for home dir parsing and checking ACIS::Web configuration 5. Setup the database. The parameters must correspond to what is in acis/main.conf. But from the old one, a change is in order Change metadata-db-name = rdb sid-db-name = rdb to metadata-db-name = acis sid-db-name = acis so only have one database, the distinction between rdb and acis is non-sensical. root@holda ~ # mysql Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1467 Server version: 5.1.58-1 (Debian) Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved. This software comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to modify and redistribute it under the GPL v2 license Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> create database acis; Query OK, 1 row affected (0.19 sec) mysql> create user 'corner'@'localhost' identified by 'a_password'; Query OK, 0 rows affected (0.28 sec) mysql> grant all on acis.* to 'corner'@'localhost'; Query OK, 0 rows affected (0.00 sec) mysql> exit; Bye aras@holda:~$ setup installing additional scripts into /home/aras/acis/bin: apu arpu backup_tables clean-up create_tables dump_email_handle_table events_archiving export_cit_events export_fturls_choices export_name_variations fuzzy_search_table mvrec offline_maintain recover_citation_events recreate_profile_pages reparse_config restore_table_backup rmrec sanitar2 sanitar3 sanitar sanitize_cit_suggestions sanitize_names_table sanitize_redif_files setup_ardb setup_profile_menu sid_base updareq upgrade upgrade_to_1.01 upgrade_to_1.02 upgrade_to_1.04 upgrade_to_1.0 upgrade_to_1.4.111 upgrade_to_1.4.61 upgrade_to_1.4.90 upgrade_to_1.4.95 upgrade_to_1.5.13 upgrade_to_1.5.27 upgrade_to_1.5.34 upgrade_to_1.5.3 upgrade_to_1.5.4 upgrade_to_1.5.53 upgrade_to_1.5.58 upgrade_to_1.5.64 upgrade_to_1.5.70 upgrade_to_1.5.7 upgrade_to_1.6.1 archive_critical writing /home/aras/WWW/cgi/a.cgi, obsolete /home/aras/acis/bin/setup: line 133: /home/aras/WWW/cgi/a.cgi: No such file or directory chmod: cannot access `/home/aras/WWW/cgi/a.cgi': No such file or directory cp: cannot stat `/home/aras/acis/presentation/ras/style': No such file or directory cp: cannot stat `/home/aras/acis/presentation/ras/script': No such file or directory installing log browsing script to /home/aras/WWW/cgi/gol.cgi /home/aras/acis/bin/setup: line 171: /home/aras/WWW/cgi/gol.cgi: No such file or directory chmod: cannot access `/home/aras/WWW/cgi/gol.cgi': No such file or directory created Short-ID tables configuring ARDB for home dir Created tables: resources acis:threads acis:cit_sug acis:ft_urls res_creators_bulk acis:citation_events acis:rp_suggestions objects acis:session_history acis:cit_old_sug acis:ft_urls_choices acis:events acis:citations_deleted acis:citations institutions res_creators_separate acis:users acis:apu_queue acis:sysprof acis:cit_doc_similarity acis:records acis:names configuring ACIS for home dir parsing and checking ACIS::Web configuration This means the tables are create. We don't need a.cgi and gol.cgi, because we use the fastCGI approach. The reference to them should be removed from a future version of ACIS. For good measure, set up .my.cnf at the same time. 6. Start rid aras@holda:~$ rid start going to start RI daemon (/home/aras/acis/bin/control_daemon.pl) done First aras@holda:~$ updareq RePEc / 9999999999999 Look at it at work. This will only work if you have the data directories set up properly in acis.conf. aras@holda:~/acis/RI$ tail -f update_ch0.log RECORD PROCESS: repec:prs:recind:rei_0154-3229_1998_num_84_1_1719|ReDIF-Article 1.0 RECORD NEW: repec:prs:recind:rei_0154-3229_1998_num_84_1_1719 U DATAFILE_FINISH: prs/recind/issue_rei_0154-3229_1998_num_84_1.rdf U DATAFILE_START: prs/recind/issue_rei_0154-3229_1994_num_70_1.rdf RECORD PROCESS: repec:prs:recind:rei_0154-3229_1994_num_70_1_1543|ReDIF-Article 1.0 RECORD NEW: repec:prs:recind:rei_0154-3229_1994_num_70_1_1543 RECORD PROCESS: repec:prs:recind:rei_0154-3229_1994_num_70_1_1544|ReDIF-Article 1.0 RECORD NEW: repec:prs:recind:rei_0154-3229_1994_num_70_1_1544 RECORD PROCESS: repec:prs:recind:rei_0154-3229_1994_num_70_1_1545|ReDIF-Article 1.0 RECORD NEW: repec:prs:recind:rei_0154-3229_1994_num_70_1_1545 7. Setup of web server holda is a sensible machine, it runs lighttpd. Ivan knows how to set this up, but I will try to do it myself. 8. Setup crontabs, and housekeeping TBD Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorprofile.org/pkr1 skype: thomaskrichel

1 0

[aras@authors.repec.org: Output from your job 44]
by Thomas Krichel 23 Aug '11

23 Aug '11

ooops... memory leak? ----- Forwarded message from RAS/ACIS <aras(a)authors.repec.org> ----- From: RAS/ACIS <aras(a)authors.repec.org> To: aras(a)authors.repec.org Subject: Output from your job 44 /home/aras/acis/bin/storable-check-ri.sh: line 4: 10647 Terminated perl bin/storable-data.pl RePEc records >> storable-data-ri.log sh: line 28: 10646 Terminated /home/aras/acis/bin/storable-check-ri.sh ----- End forwarded message ----- -- Cheers, Thomas Krichel http://openlib.org/home/krichel http://authorprofile.org/pkr1 skype: thomaskrichel

2 1