[RAS] setting up a new RAS

Thomas Krichel krichel at openlib.org
Fri Aug 26 08:55:00 CDT 2011 

  With the latest ACIS version. I have started this today.
  I found some problems that I have fixed. So some of the
  notes are not completely applicable, but it sohuld be
  enough to reproduce the process. 

  It's not complete, but it should Dan get started.
 
  Dan I suggest you try to use the latest BDB version
  you can get. This will create a new set and not 
  use the corrupted ones we have on nebka. 

  I will not get offline, get drunk and chase women,
  but if not successful (rather likely) you can find
  me online later to talk. 

  Installation notes from test system aras at holda.openlib.org

  1. Make DNS entry ras.openlib.org to go to holda

binder at snefru:~$ nslookup ras.openlib.org
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   ras.openlib.org
Address: 85.10.194.76

binder at snefru:~$ nslookup holda.openlib.org
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:   holda.openlib.org
Address: 85.10.194.76

  2. Install libsvm-tools

aptitude install libsvm-tools

  3. Mirror acis from nebka (without RI directory, so save space).

  Make a copy of acis, as mirrored from nebka

aras at holda:~$ mv acis/ acis.org
aras at holda:~$ mkdir acis

  4. Install. Note I made already a new release, so the number will be 
  different here. 

aras at holda:~$ cp ~aacis/public_html/code/ACIS-1.6.50-20110826a.with-libs.tar.gz  .
aras at holda:~$ cp ~aacis/public_html/code/ACIS-1.6.50-20110826a.with-libs.tar.gz  .
aras at holda:~$ tar xzf ACIS-1.6.50-20110826a.with-libs.tar.gz 
aras at holda:~$ cd ACIS-1.6.50-20110826a.with-libs
aras at holda:~/ACIS-1.6.50-20110826a.with-libs$ sh install.sh ~/acis
this is an upgrade
Checking required modules...
  - CGI::Untaint (version 1.25+) is not found
  - String::Similarity (version 1+) is not found
  - AppConfig (version 1.52+) is not found
  - Date::Manip is not found
  - Proc::Daemon (version 0.03+) is not found
  - CGI::Minimal (version 1.24+) is not found
  - String::Approx is not found
installing to /home/aras/acis directory
installing scripts into /home/aras/acis/bin: cit_test.pl conf.pl rid setup setup.cgi_frontend setup.logs-browsing setup.ri_collections setup.ri_local_setup.pm setup.sid_local.pm
this is an upgrade
installing to /home/aras/acis directory
/home/aras/acis/bin/setup: line 11: cd: /home/awho/acis: No such file or directory
You need to have AppConfig perl module installed
unpacked AMF-perl into dir AMF-perl-0.40 AMF-perl-0.40.tar.gz
Now you might want to create main.conf file in /home/aras/acis
Then run bin/setup there.

  There is a mistake on line 11 of setup. I am not sure
  what should replace the line 

homedir=/home/awho/acis

  to

homedir=~/acis

aacis at holda:~/dev/acis$ darcs whatsnew
hunk ./home/bin/setup 3
-homedir=/home/awho/acis
+homedir=~/acis

  Install what I can from Debian

root at holda ~ # aptitude install libcgi-untaint-perl libappconfig-perl libdate-manip-perl libproc-daemon-perl libstring-approx-perl

The following NEW packages will be installed:
  libappconfig-perl libcgi-untaint-perl libdate-manip-perl libproc-daemon-perl libstring-approx-perl libuniversal-require-perl{a} 
  libyaml-syck-perl{a} 
0 packages upgraded, 7 newly installed, 0 to remove and 0 not upgraded.
Need to get 2,949 kB of archives. After unpacking 16.4 MB will be used.
  
  Two packages need to be installed manually. I use ~/usr as prefix.
  The versions can be downloaded from http://acis.openlib.org/code/extra_perl
  Since I am running on holda, which does acis.openlib.org and
  ras.openlib.org, I can just copy

cd /tmp/
tar -xf /home/aacis/public_html/extra_perl/CGI-Minimal-1.29.tar.gz 
cd CGI-Minimal-1.29/
perl Makefile.PL PREFIX=~/usr
make
make test
make install
cd ..
tar -xf /home/aacis/public_html/extra_perl/String-Similarity-1.04.tar.gz 
cd String-Similarity-1.04/
perl Makefile.PL PREFIX=~/usr
make 
make test
make install
cd

  now get main.conf from the orginal

aras at holda:~/acis$ cp ~/acis.org/main.conf main.conf

  then

aras at holda:~/acis$ setup

  fails because the CGI::Minimal not found. Crap. Some
  modules are in share, others in lib. 

aras at holda:~/usr$ echo $PERL5LIB
/home/aras/usr/lib/perl:/home/aras/acis/lib:/home/aras/acis/lib/ARDB
aras at holda:~/usr$ find lib
lib
lib/perl
lib/perl/5.12.4
lib/perl/5.12.4/String
lib/perl/5.12.4/String/Similarity.pm
lib/perl/5.12.4/perllocal.pod
lib/perl/5.12.4/auto
lib/perl/5.12.4/auto/String
lib/perl/5.12.4/auto/String/Similarity
lib/perl/5.12.4/auto/String/Similarity/.packlist
lib/perl/5.12.4/auto/String/Similarity/Similarity.so
lib/perl/5.12.4/auto/String/Similarity/Similarity.bs
lib/perl/5.12.4/auto/CGI
lib/perl/5.12.4/auto/CGI/Minimal
lib/perl/5.12.4/auto/CGI/Minimal/.packlist
aras at holda:~/usr$ find share
share
share/perl
share/perl/5.12.4
share/perl/5.12.4/CGI
share/perl/5.12.4/CGI/Minimal.pm
share/perl/5.12.4/CGI/Minimal
share/perl/5.12.4/CGI/Minimal/Multipart.pm
share/perl/5.12.4/CGI/Minimal/Misc.pm
share/perl/5.12.4/CGI/Minimal.pod

  The way I deal with this is to make "share" an alias for "lib".

aras at holda:~/usr$ rsync -va share/ lib
sending incremental file list
perl/5.12.4/
perl/5.12.4/CGI/
perl/5.12.4/CGI/Minimal.pm
perl/5.12.4/CGI/Minimal.pod
perl/5.12.4/CGI/Minimal/
perl/5.12.4/CGI/Minimal/Misc.pm
perl/5.12.4/CGI/Minimal/Multipart.pm

sent 42452 bytes  received 101 bytes  85106.00 bytes/sec
total size is 42092  speedup is 0.99
aras at holda:~/usr$ rm -rf share
aras at holda:~/usr$ ln -s lib share

  We still need to install additonal modules. 

cd
cd ACIS-1.6.50-20110826a.with-libs
ls -lt
cd AMF-perl-0.40
perl Makefile.PL PREFIX=~/usr
make test
make install
cd ..
ls
cd RePEc-Index-0.38.3
perl Makefile.PL PREFIX=~/usr
make 
make test

  Some tests fail because of the use of plan and because
  Common::Data is not found. These failures are harmless.
  The Common::Data problem is fixed in the update. 

  Install the ReDIF reader

aras at holda:~/ACIS-1.6.50-20110826a.with-libs$ tar xzf ReDIF-perl-2.65.tar.gz 
aras at holda:~/ACIS-1.6.50-20110826a.with-libs$ cd ReDIF-perl-2.65
aras at holda:~/ACIS-1.6.50-20110826a.with-libs/ReDIF-perl-2.65$ perl Makefile.PL PREFIX=~/usr
Preparing to install ReDIF perl package.

Your system has Digest::MD5 module installed.
Do you want to enable MD5 checksum calculation for templates? [yes] 

Do you want ReDIF-perl to check attribute values
for a reasonable length? (early implementation) [no] 

ReDIF home directory is where you store your ReDIF data, your
configuration files for ReDIF-related programs and other related
stuff.  Some programs require it for operation, but FOR ReDIF-perl
INSTALLATION IT IS NOT REQUIRED.  You can set it now and ReDIF-perl
will use it always BY DEFAULT.  Otherwise you will have to pass the
path to ReDIF home by other means to some programs (e.g. through
environment variable REDIFDIR).

Do you wish to set default ReDIF home directory now? [yes] 
What directory to use as the default ReDIF home? [enter for not setting it now] 

Saved lib/ReDIF/setup_conf.pm. Current configuration:

        ReDIF home directory: not set

Checking if your kit is complete...
Looks good
Writing Makefile for ReDIF-perl

aras at holda:~/ACIS-1.6.50-20110826a.with-libs/ReDIF-perl-2.65$ make

  ...

aras at holda:~/ACIS-1.6.50-20110826a.with-libs/ReDIF-perl-2.65$ make tast

  tests fail because of plan, considered harmless

aras at holda:~/ACIS-1.6.50-20110826a.with-libs/ReDIF-perl-2.65$ make install

aras at holda:~/acis$ setup

  fails because the Data::Common is not found. I find it's a 
  problem in the MANIFEST file, that is fixed in a newer version.

aacis at holda:~/dev/acis$ darcs whatsnew
hunk ./MANIFEST 287
+lib/ACIS/Resources/Learn.pm
+lib/ACIS/Resources/Learn/KnownItems.pm
+lib/ACIS/Resources/Learn/Suggested.pm
hunk ./MANIFEST 352
+lib/Common/Data.pm

  make a new version at holda

perl build/dist.pl -f 1.6.50
ls -lt
rm ACIS-1.6.50-20110826a.with-libs.tar.gz
rm ~/public_html/code/ACIS-1.6.50-20110826a.with-libs.tar.gz 
mv ACIS-1.6.50-20110826b.with-libs.tar.gz ~/public_html/code/.

  on the test account, start da capo

aras at holda:~$ rm -rf ACIS-1.6.50-20110826a.with-libs*

  I can untar this because repository and test are running on same box

aras at holda:~$ tar xzf ~aacis/public_html/code/ACIS-1.6.50-20110826b.with-libs.tar.gz 

  When the module RAS.pl is requested, comment this out in acis/main.conf


# /meta/update interface
#require-modules=RAS.pl
#meta-update-clients="wop at 128.252.177.191"
#meta-update-object-fetch-func=RePEc::RAS::object_fetch

  You should now, on setup, see something like

aras at holda:~/acis$ setup
installing additional scripts into /home/aras/acis/bin: apu arpu backup_tables clean-up create_tables dump_email_handle_table events_archiving export_cit_events export_fturls_choices export_name_variations fuzzy_search_table mvrec offline_maintain recover_citation_events recreate_profile_pages reparse_config restore_table_backup rmrec sanitar2 sanitar3 sanitar sanitize_cit_suggestions sanitize_names_table sanitize_redif_files setup_ardb setup_profile_menu sid_base updareq upgrade upgrade_to_1.01 upgrade_to_1.02 upgrade_to_1.04 upgrade_to_1.0 upgrade_to_1.4.111 upgrade_to_1.4.61 upgrade_to_1.4.90 upgrade_to_1.4.95 upgrade_to_1.5.13 upgrade_to_1.5.27 upgrade_to_1.5.34 upgrade_to_1.5.3 upgrade_to_1.5.4 upgrade_to_1.5.53 upgrade_to_1.5.58 upgrade_to_1.5.64 upgrade_to_1.5.70 upgrade_to_1.5.7 upgrade_to_1.6.1 archive_critical
writing /home/aras/WWW/cgi/a.cgi, obsolete
/home/aras/acis/bin/setup: line 133: /home/aras/WWW/cgi/a.cgi: No such file or directory
chmod: cannot access `/home/aras/WWW/cgi/a.cgi': No such file or directory
cp: cannot stat `/home/aras/acis/presentation/ras/style': No such file or directory
cp: cannot stat `/home/aras/acis/presentation/ras/script': No such file or directory
installing log browsing script to /home/aras/WWW/cgi/gol.cgi
/home/aras/acis/bin/setup: line 171: /home/aras/WWW/cgi/gol.cgi: No such file or directory
chmod: cannot access `/home/aras/WWW/cgi/gol.cgi': No such file or directory
DBI connect('rdb','corner',...) failed: Access denied for user 'corner'@'localhost' (using password: YES) at /home/aras/acis/lib/sql_helper.pm line 260
Can't call method "do" without a package or object reference at /home/aras/acis/lib/ACIS/ShortIDs.pm line 357.
configuring ARDB for home dir
DBI connect('rdb','corner',...) failed: Access denied for user 'corner'@'localhost' (using password: YES) at /home/aras/acis/lib/sql_helper.pm line 260
Can't establish database connection at /home/aras/acis/lib/ARDB/Setup.pm line 75.
Can't locate ARDB/Local.pm in @INC (@INC contains: /home/aras/acis/lib /home/aras/usr/lib/perl/5.12.4 /home/aras/usr/lib/perl /home/aras/acis/lib/ARDB /etc/perl /usr/local/lib/perl/5.12.4 /usr/local/share/perl/5.12.4 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.12 /usr/share/perl/5.12 /usr/local/lib/site_perl /usr/local/lib/perl/5.12.3 /usr/local/share/perl/5.12.3 .) at bin/create_tables line 14.
BEGIN failed--compilation aborted at bin/create_tables line 14.
configuring ACIS for home dir
parsing and checking ACIS::Web configuration

 5. Setup the database. The parameters must correspond to what is 
 in acis/main.conf. But from the old one, a change is in order
 Change

metadata-db-name = rdb
sid-db-name      = rdb

  to

metadata-db-name = acis
sid-db-name      = acis

  so only have one database, the distinction between rdb
  and acis is non-sensical. 


root at holda ~ # mysql 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1467
Server version: 5.1.58-1 (Debian)

Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL v2 license

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> create database acis; 
Query OK, 1 row affected (0.19 sec)

mysql> create user 'corner'@'localhost' identified by 'a_password';
Query OK, 0 rows affected (0.28 sec)

mysql> grant all on acis.* to 'corner'@'localhost';
Query OK, 0 rows affected (0.00 sec)

mysql> exit;
Bye

aras at holda:~$ setup
installing additional scripts into /home/aras/acis/bin: apu arpu backup_tables clean-up create_tables dump_email_handle_table events_archiving export_cit_events export_fturls_choices export_name_variations fuzzy_search_table mvrec offline_maintain recover_citation_events recreate_profile_pages reparse_config restore_table_backup rmrec sanitar2 sanitar3 sanitar sanitize_cit_suggestions sanitize_names_table sanitize_redif_files setup_ardb setup_profile_menu sid_base updareq upgrade upgrade_to_1.01 upgrade_to_1.02 upgrade_to_1.04 upgrade_to_1.0 upgrade_to_1.4.111 upgrade_to_1.4.61 upgrade_to_1.4.90 upgrade_to_1.4.95 upgrade_to_1.5.13 upgrade_to_1.5.27 upgrade_to_1.5.34 upgrade_to_1.5.3 upgrade_to_1.5.4 upgrade_to_1.5.53 upgrade_to_1.5.58 upgrade_to_1.5.64 upgrade_to_1.5.70 upgrade_to_1.5.7 upgrade_to_1.6.1 archive_critical
writing /home/aras/WWW/cgi/a.cgi, obsolete
/home/aras/acis/bin/setup: line 133: /home/aras/WWW/cgi/a.cgi: No such file or directory
chmod: cannot access `/home/aras/WWW/cgi/a.cgi': No such file or directory
cp: cannot stat `/home/aras/acis/presentation/ras/style': No such file or directory
cp: cannot stat `/home/aras/acis/presentation/ras/script': No such file or directory
installing log browsing script to /home/aras/WWW/cgi/gol.cgi
/home/aras/acis/bin/setup: line 171: /home/aras/WWW/cgi/gol.cgi: No such file or directory
chmod: cannot access `/home/aras/WWW/cgi/gol.cgi': No such file or directory
created Short-ID tables
configuring ARDB for home dir
Created tables: resources acis:threads acis:cit_sug acis:ft_urls res_creators_bulk acis:citation_events acis:rp_suggestions objects acis:session_history acis:cit_old_sug acis:ft_urls_choices acis:events acis:citations_deleted acis:citations institutions res_creators_separate acis:users acis:apu_queue acis:sysprof acis:cit_doc_similarity acis:records acis:names
configuring ACIS for home dir
parsing and checking ACIS::Web configuration

  This means the tables are create. We don't need a.cgi and
  gol.cgi, because we use the fastCGI approach. The reference
  to them should be removed from a future version of ACIS.

  For good measure, set up .my.cnf at the same time.

  6. Start rid

aras at holda:~$ rid start
going to start RI daemon (/home/aras/acis/bin/control_daemon.pl)
 done

  First 

aras at holda:~$ updareq RePEc / 9999999999999

  Look at it at work. This will only work if you have the
  data directories set up properly in acis.conf.

aras at holda:~/acis/RI$ tail -f update_ch0.log 
RECORD PROCESS: repec:prs:recind:rei_0154-3229_1998_num_84_1_1719|ReDIF-Article 1.0
RECORD NEW: repec:prs:recind:rei_0154-3229_1998_num_84_1_1719
U DATAFILE_FINISH: prs/recind/issue_rei_0154-3229_1998_num_84_1.rdf
U DATAFILE_START: prs/recind/issue_rei_0154-3229_1994_num_70_1.rdf
RECORD PROCESS: repec:prs:recind:rei_0154-3229_1994_num_70_1_1543|ReDIF-Article 1.0
RECORD NEW: repec:prs:recind:rei_0154-3229_1994_num_70_1_1543
RECORD PROCESS: repec:prs:recind:rei_0154-3229_1994_num_70_1_1544|ReDIF-Article 1.0
RECORD NEW: repec:prs:recind:rei_0154-3229_1994_num_70_1_1544
RECORD PROCESS: repec:prs:recind:rei_0154-3229_1994_num_70_1_1545|ReDIF-Article 1.0
RECORD NEW: repec:prs:recind:rei_0154-3229_1994_num_70_1_1545


  7. Setup of web server

  holda is a sensible machine, it runs lighttpd. Ivan knows how
  to set this up, but I will try to do it myself.

  8. Setup crontabs, and housekeeping 

  TBD


  Cheers,

  Thomas Krichel                    http://openlib.org/home/krichel
                                      http://authorprofile.org/pkr1
                                               skype: thomaskrichel

More information about the RAS-run mailing list