- CollEc-run - RePEc Lists

darni crashed
by Thomas Krichel 30 Jul '20

30 Jul '20

At 23:45 last night. I woke up shortly after that. A hardware reset I performed about 25 minutes later brought it back. I did not find a trace in the log as to what may have caused the crash. Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

2 4

Re: [CollEc] CollEc 2.0 Access
by Thomas Krichel 23 Jul '20

23 Jul '20

Düben, Christian writes > Vimeo apparently does not display ads. But that > might change at one point and I am not going to monitor that > platform's ad policy. Therefore, I hosted the video locally for now, > despite the mentioned disadvantages. It's up to you really. > I added the daily update routine to the crontab file. I was wondering if you had stopped because the load of the server seems to have drastically declined. I don't notice it any more. > The loading time at app startup is annoying It seems much faster now. I think it reaches ok state. I mean a lot of sites are slow these day, maybe we need to keep up with the times. > and I looked into its > causes and ways to circumvent it. Its source is the way ShinyProxy > makes the app available. Whenever a user accesses the app, > ShinyProxy spins up a new Docker container with the app in it. Once > the user becomes inactive, ShinyProxy removes the container. And the > initial startup time is essentially the time it takes to spin up > that container. If users were first directed to an index page, on > which they select the app, I could pre-initialize the container once > they enter that index page. Unfortunately, ShinyProxy does not > support pre-initializing containers when users access the app > directly, like in our case. In short, the loading time at startup > remains for now. Is it normal that it is all grey? Would it not be possible to set up static pages, and have the app only be active if there is a need to work on user input? I think that the start page explains what the service does. Then there should be a video, and then background and whosdone it. CollEc is a RePEc service that evaluates the economic literature's co-authorship network. It allows you to assess bilateral distances and centrality measures like closeness and betweenness for more than 47,000 authors. As in GraphEc, you interact with the results through graphical representations. Feel free to watch the following tutorial for a brief introduction to CollEc's functionalities. Note that the video player depends on your browser. In some browsers you may need to right-click on the video to select the full screen option. like closeness and betweenness --> called closeness and betweenness Then I suggest the video and then It is maintained by Christian Düben. Our data comes from the RePEc Author Service, a RePEc service maintained by Christian Zimmermann . Further details are available in the documentation . CollEc was founded by Thomas Krichel in 2011. I am not sure we need you and me in the end here because we have it in the footer. > Could you tell me how you linked test.collec.repec.org to the app? > As I mentioned in an earlier e-mail, it does not work. I would need > some specifics to find the bug. It was a five-minute job. root@darni ~ # cat /etc/apache2/sites-available/collec.repec.org <VirtualHost *:80> ServerName test.collec.repec.org ServerAlias test.collec.openlib.org ErrorLog /var/log/apache2/collec.proxy-error.log CustomLog /var/log/apache2/collec-access.log combined UseCanonicalName Off ProxyPass / http://localhost:8080/ ProxyPassReverse / http://localhost:8080/ </VirtualHost> > I can barely talk without coughing these days. But I am going to produce the video as soon as possible. > > Apart from the video and the URL connection issue, I think that the app is ready to become publicly available. > > I have the impression that you upset or unsatisfied with this project. Did I do something that upset you? Did the app development take too long? Do you not like the layout or the content? I could have developed the app faster, but I wanted to try various optimizations before releasing it. And familiarizing myself with Docker, MariaDB, ShinyProxy, Shiny etc. took a while. I am sorry, if I upset you. Please feel free to provide any feedback or suggestions. > I'm very sorry for appearing to be grumpy! I don't know how to use the selection screen. The select by is confusing. I don't seem to be able to get anything from it. I type "Krichel" or pkr1 in there and just nothing happens. I think we need the ranked lists there and finally we will need exports. -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

3 24

Re: [CollEc] CollEc 2.0 Access
by Thomas Krichel 17 Jul '20

17 Jul '20

Düben, Christian writes > Did the app development take too long? No way. You work lightening speed compared to me. RePEc has never been about doing things fast. But I'm reaching new lows. I have been struggling with ArchEc over the past weeks in an intesely frustrating experience. The other day I managed to ruin 170000 files there, hundreds needing manual fixing. -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

1 0

Re: [CollEc] CollEc 2.0 Access
by Thomas Krichel 17 Jul '20

17 Jul '20

Düben, Christian writes > I have the impression that you upset or unsatisfied with this project. Not at all. I'm upset with my own lack of work on this. -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

1 0

Re: [CollEc] CollEc 2.0 Access
by Thomas Krichel 15 Jul '20

15 Jul '20

Düben, Christian writes > I am still working on the database performance. Insertion times > dropped but did not reach a satisfactory level yet. However, as the > application is done except for the database issue, I decided to seek > feedback from your side already. You can access it at > "http://95.216.35.87:8080" For that I already have test.collec.repec.org > with the user name "Test_User" Why do I need to sign in? It's too slow after sign in. In general, darni is dreadfully slow today. I think the i/o is the bottleneck. I have been deleting files as we reached %94 full. Few Otto Normaleconomists will understand the weighted networks. Data entry should not ask to fill a name or fill a handle, but detect whether the entry is a name or a handle. Ideally, one should enter family names as well as they are more famliar to users who search. I think you need to start with a simple note, where the screen is cached. > The video embedded in the "Introduction" tab is a placeholder and > will be replaced with the actual tutorial upon the new CollEc's > public release. The output generated in the "Distances", "Closeness" > and "Betweenness" tabs is temporarily not available as I am currently > working on the underlying MariaDB tables. I changed the documentation > a few times and am curious to hear your comments on it. I think it's better to give people a list of top nodes at the start, or just start with a screen very similar to the to one of the existing CollEc. It may not be a the best, but some folks got used to it. A video link should go to the docmentation page. -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

3 7

Re: [CollEc] RePEc Visual
by Thomas Krichel 09 Jul '20

09 Jul '20

Düben, Christian writes > I went through some of the files and checked what I would need for > an extension of CollEc. I have a few ideas in mind on what to add > and how to present it in an interactive application. It's very hard to do a worse job than I did vizualizing that data! > When consulting our IT department here at Hamburg University, they > suggested to host RePEc Visual on one of their managed Linux > servers. At this point I am still waiting for the administration to > process my application requesting such a server. And just like every > administrative procedure at our institution, this takes a while. Once > I have access to the respective infrastructure I am going to test > implementations of RePEc Visual and potential CollEc extensions on > it. Those applications would of course run under an external domain, > not a Hamburg University domain. We could run this on the existing CollEc server. This would be especially valuable if you manage to find a way to run the calculations faster. At this time, it's dreadfully slow. You could just take over the whole thing, well almost. We need to keep the mention of the sponsor, and I'd like to be aknowledged as the orginal creator. > I do not have Telegram and apparently do not have the correct login > credentials for the Skype setup on my office Laptop. Do you use > Zoom? If you do, I can send you a meeting link. If you do not, I > will try to find out what login credentials our IT set for Skype. Zoom should be fine. I'm in UTC+7. I can do late evenings no problem. My schedule is completely open. Maybe someone else would want to attend? I copy CollEc-run. -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

4 51

Re: [CollEc] mentioning co-authorship registration failure in CollEc
by Thomas Krichel 09 Nov '19

09 Nov '19

Danny Cassimon writes > Apologies for contacting you to mention a concrete broken link or > failure of account of the CollEc part of the REPEC/IDEAS repository. We don't get enough of such feedback! > As an experienced user of the system, I now note that recently, the > system seems to fail to, or is in any case much slower in, accounting > to register new co-authors in personal co-author lists. To give a > personal example, a colleague that registered about two months ago, > and with whom I have a mutually-accepted co-authored paper (see: > https://ideas.repec.org/p/pra/mprapa/96072.html) Are you talking about Solomon here? Here is the input directory icanis@helos:~/icanis/input$ ls -lt total 388168 -rw-rw-r-- 1 icanis icanis 70223531 Nov 8 04:10 ras_mans_texts_1573182608.xml -rw-rw-r-- 1 icanis icanis 8624782 Nov 8 04:10 ras_mans_edges_1573182608.xml -rw-rw-r-- 1 icanis icanis 70223531 Nov 7 04:07 ras_mans_texts_1573096074.xml -rw-rw-r-- 1 icanis icanis 8624784 Nov 7 04:07 ras_mans_edges_1573096074.xml -rw-rw-r-- 1 icanis icanis 70223531 Nov 6 04:07 ras_mans_texts_1573009656.xml -rw-rw-r-- 1 icanis icanis 3218932 Nov 6 04:07 ras_mans_nodes_1573009656.xml -rw-rw-r-- 1 icanis icanis 8624786 Nov 6 04:07 ras_mans_edges_1573009656.xml -rw-rw-r-- 1 icanis icanis 70223531 Nov 5 04:05 ras_mans_texts_1572923121.xml -rw-rw-r-- 1 icanis icanis 8624771 Nov 5 04:05 ras_mans_edges_1572923121.xml -rw-rw-r-- 1 icanis icanis 70223531 Nov 4 04:08 ras_mans_texts_1572836929.xml -rw-rw-r-- 1 icanis icanis 8624778 Nov 4 04:08 ras_mans_edges_1572836929.xml The edges matrices don't seem to change too much. icanis@helos:~/icanis/input$ grep psa1752 ras_mans_edges_* icanis@helos:~/icanis/input$ Solomon is not there. Danny is icanis@helos:~/icanis/input$ grep pca462 ras_mans_edges_1573182608.xml <edge from="pbl118" to="pca462" length="0.666666666666667"/> <edge from="pca1004" to="pca462" length="1"/> <edge from="pca462" to="pcl16" length="0.4"/> <edge from="pca462" to="pde1008" length="2"/> <edge from="pca462" to="pdr61" length="0.2"/> <edge from="pca462" to="pen79" length="0.12"/> <edge from="pca462" to="per126" length="0.5"/> <edge from="pca462" to="pes101" length="0.0769230769230769"/> <edge from="pca462" to="pfa375" length="1"/> <edge from="pca462" to="pfe372" length="1.5"/> <edge from="pca462" to="pme555" length="0.166666666666667"/> <edge from="pca462" to="pme591" length="3"/> <edge from="pca462" to="pra263" length="0.272727272727273"/> <edge from="pca462" to="pri349" length="1"/> <edge from="pca462" to="pse638" length="0.666666666666667"/> <edge from="pca462" to="ptc17" length="0.333333333333333"/> <edge from="pca462" to="pva229" length="0.109090909090909"/> <edge from="pca462" to="pva653" length="0.545454545454546"/> <edge from="pca462" to="pva685" length="2"/> <edge from="pca462" to="pve127" length="2"/> <edge from="pca462" to="pve344" length="0.25"/> <edge from="pca462" to="pwo109" length="1"/> icanis@helos:~/icanis/input$ The input update is obviously broken. The input script uses icanis@helos:~$ grep per_dir perl/compose_ras_icanis_input my $per_dir="/home/adrepec/RePEc/remo/per/pers/"; This is NFS linked to helos on aigtu icanis@helos:~$ ls -ld /home/adrepec lrwxrwxrwx 1 root root 25 May 25 2017 /home/adrepec -> ../mnt/aigtu/home/adrepec There are psas in there icanis@helos:~$ ls /home/adrepec/RePEc/remo/per/pers/s/psa17* /home/adrepec/RePEc/remo/per/pers/s/psa1700.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1701.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1702.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1703.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1704.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1705.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1706.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1707.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1708.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1709.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1710.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1711.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1712.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1713.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1714.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1715.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1716.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1717.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1718.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1720.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1721.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1722.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1723.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1724.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1725.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1726.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1727.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1728.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1729.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1730.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1731.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1732.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1733.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1734.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1735.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1736.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1737.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1738.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1739.rdf /home/adrepec/RePEc/remo/per/pers/s/psa173.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1740.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1741.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1742.rdf /home/adrepec/RePEc/remo/per/pers/s/psa1743.rdf /home/adrepec/RePEc/remo/per/pers/s/psa177.rdf But no psa1752 icanis@helos:~$ ls /home/adrepec/RePEc/remo/per/pers/s/psa1752* So CollEc is not to blame, the input is missing. -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

2 1

katri --> helos
by Thomas Krichel 07 Aug '19

07 Aug '19

This is finally finished. The new server is an Hetzner EX-52. There is no Debian for this hardware yet. I went with Ubuntu. Some stats root@helos ~ # free total used free shared buff/cache available Mem: 65877048 2862536 12860392 2348 50154120 62276476 Swap: 33521660 0 33521660 root@helos ~ # df | grep md2 /dev/md2 7717425284 1773026772 5555391392 25% / I run 9 update scripts on amorp (via NFS) and 9 on helos. Maybe I should run more on helos. I put together a stats script for the path files. count: 42994 age: 0.000497685185185185 51.1333999371576 79.2213541666667 This last line has min, mean, and max ago of files in days. I am shocked about the count. Is this correct? Can we really have close to 43k in our largest component? -- Cheers, Thomas Krichel http://openlib.org/home/krichel skype:thomaskrichel

2 4

Inconsistent data
by Christian Zimmermann 04 Jul '19

04 Jul '19

There is a problem either in the file at ftp://ftp.repec.org/CollEc/CollEc.txt or with what is dispayed on the site. The numbers do not match. For example for me: site: closeness 4943 4.8307 betweenness 918 7.87024 file: 4.98787 5804 7.25581 937 The difference is larger for people that are complaining. Christian Zimmermann FIGUGEGL! Economic Research Federal Reserve Bank of St. Louis P.O. Box 442 St. Louis MO 63166-0442 USA https://ideas.repec.org/zimm/ @CZimm_economist

2 1

Incomplete data
by Christian Zimmermann 09 May '19

09 May '19

The ftp://ftp.repec.org/CollEc/CollEc.txt file is incomplete! It stops at pma1077. Christian Zimmermann FIGUGEGL! Economic Research Federal Reserve Bank of St. Louis P.O. Box 442 St. Louis MO 63166-0442 USA https://ideas.repec.org/zimm/ @CZimm_economist

2 9