Escience software stack

From Tetherless World Wiki

List of installed software on the escience.rpi.edu system.

Production systems

escience is slated to be a Tier 4 (production) machine. Eventually, only production systems will be running on this machine.

svn
trac
apache
mysql
tomcat
- vsto
- opendap

NOTES

Please read the NOTES section for important information about the machine.

Add /opt/bin to your PATH, /opt/lib to your LD_LIBRARY_PATH, /opt/man to your MANPATH.

Any future applications should be installed in /opt and not /usr or /usr/local. TW will be responsible for whatever is installed in /opt and VCC will be responsible for anything installed in /usr.

It was discovered that /opt, /home, /usr, and virtual memory are all on the root partition, and the root partition filled up rather quickly. The way the machine was set up, there was a different, much larger partition at /ext. So we did the following:

Created a /ext/opt directory with a sym link from /opt to /ext/opt
Created a /ext/project directory with a sym link from /project to /ext/project
Created the directory /project/src for production prep. So there is a /project/src/pwest, /project/src/zednik, /project/src/csc, etc...
Created production project directories under /project for any production quality software to be located. For example, /project/vsto (if there was a need for one)
Created user directories under /project/user. Should not do any significant work in your home directory

Currently installed on escience

Software Package	Version	Installed in	Maintainer	Dependencies
svn	1.5.4 (r33841)	/usr	VCC
libcurl	7.15.5	/usr	VCC
flex	2.5.4	/usr	VCC
bison	2.3	/usr	VCC
libxml2	2.6.26	/usr	VCC
readline	5.1	/usr	VCC	ncurses
dejagnu	1.4.4	/usr	VCC	Expect, TCL
gcc	4.1.2	/usr include gfortran	VCC
gmake	3.81	/usr	VCC
autoconf	2.63	/opt ./configure '--prefix=/opt' '--libdir=/opt/lib64'	TW
automake	1.10.2	/opt ./configure '--prefix=/opt' '--libdir=/opt/lib64'	TW
libtool	1.5.26 (tried 2.2.6, but does not work properly)	/opt ./configure '--prefix=/opt' '--libdir=/opt/lib64' '--enable-ltdl-install'	TW
vim/gvim	7.0	/usr	VCC
Apache	2.2.3	/usr see below	TW
php	5.1.6 - php-mysql, php-mbstring, php-mcrypt also installed for phpMyAdmin support	/usr	VCC
mysql client libraries	5.0.45	/usr	VCC
mysql server	5.0.45	/usr	VCC
phpMyAdmin	2.11.9.5 (version 3.x requires php 5.2 or greater)	/opt/var/www/	TW	php-mysql module, mysql server
cppunit	1.12.1	/opt ./configure '--prefix=/opt' '--libdir=/opt/lib64'	TW
netcdf	3.6.3	/opt ./configure '--prefix=/opt' '--libdir=/opt/lib64' '--enable-shared'	TW
libjpeg	6b	/usr	TW
szip	2.1	/opt	TW
hdf4	1.8.2	/opt ./configure 'F77=gfortran' '--prefix=/opt' '--libdir=/opt/lib64' '--enable-production' Also had to hand build the source in mfhdf/libsrc and hdf/src, adding -fPIC to the compile lines, to get 64-bit builds to work. No shared library build for hdf4. And adding to CFLAGS or CXXFLAGS did not work. Was getting the error: /usr/bin/ld: /opt/lib64/libmfhdf.a(mfsd.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC /opt/lib64/libmfhdf.a: could not read symbols: Bad value	TW	libjpeg
hdf5	1.8.2	/opt ./configure 'FC=gfortran' '--prefix=/opt' '--libdir=/opt/lib64' '--enable-cxx' '--enable-fortran' '--enable-shared'	TW	libjpeg
cvs	1.11.22 (5.el5)	/usr	VCC
tcpflow	1.21	/usr	TW
doxygen	1.5.8	/opt ./configure --prefix /opt	TW
graphviz	2.20.3	/opt ./configure --prefix=/opt	TW
emacs	21.4 (20.el5)	/usr	VCC
pellet	1.4	/opt/pellet/pellet-1.4	TW
maven	2.0.9	/opt/maven/apache-maven-2.0.9	TW
tomcat	6.x	/opt/tomcat/apache-tomcat-6.0.18 see below	TW
java JRE	1.6.0_10	/usr/jdk1.6.0_10/jre	VCC
swig	1.3.38	/opt ./configure --prefix=/opt	TW
easy_install for python	setuptools-0.6c9-py2.4	/usr sudo ./setuptools-0.6c9-py2.4.egg --install-dir=/usr/lib64/python2.4/site-packages	TW
Genshi	Genshi-0.5.1-py2.4-linux-x86_64	/usr sudo easy_install --install-dir=/usr/lib64/python2.4/site-packages Genshi	TW
PySQLite	2.5.1	/usr sudo easy_install --install-dir=/usr/lib64/python2.4/site-packages PySQLite	TW
Trac	0.11.3	/usr sudo python ./setup.py install	TW	Genshi and PySQLite
ImageMagick	6.2.8	/usr	VCC
Jena TDB	0.7.2	/opt/TDB-0.7.2 with a symbolic link to that directory at /opt/TDB	TW
Virtuoso	5.0.10	/opt/virtuoso-5.0.10 with a symbolic link to that directory at /opt/virtuoso ./configure CFLAGS="-O2 -m64" '--prefix=/opt/virtuoso-5.0.10' '--enable-perl' '--enable-python' '--enable-imagemagick=/usr' '--with-jdk4=/usr/jdk1.6.0_10'	TW	perl, python, java, imagemagick

Still need to install these software packages

Software Package	Version	Install in	Maintainer	Dependencies
globus	4.2.2 (not released)	/opt/??	Stephan for now
MPICH2	1.1b1	/opt	TWC
ADaM - Data Mining and Image Processing Toolkits	4.0.2	/opt	TWC

Tomcat/Apache installation

Apache

start script is /usr/sbin/apachectl
configuration is located in /etc/httpd/conf/httpd.conf with additional configuration in /etc/httpd/conf.d
cron script to start and stop apache is /etc/init.d/httpd, with the options start, start, restart, status.
logs in /etc/httpd/logs/
modules in /etc/httpd/modules
listening on the escience.rpi.edu ip address (128.113.28.175), port 80 and 443
creating the security certificate for escience.rpi.edu

% openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout server.key -out server.crt

Country Name (2 letter code) [GB]:US
State or Province Name (full name) [Berkshire]:New York
Locality Name (eg, city) [Newbury]:Troy
Organization Name (eg, company) [My Company Ltd]:Rensselaer Polytechnic Institute
Organizational Unit Name (eg, section) []:Tetherless World Constellation
Common Name (eg, your name or your server's hostname) []:escience.rpi.edu
Email Address []:tw-application-l@lists.rpi.edu

Copy these files (server.key and server.crt) to the appropriate directory under /etc/httpd/conf (ssl.key and ssl.crt respectively)

Tomcat for VSTO

tomcat listening on the vsto.rpi.edu ip address (128.113.28.180), port 80 and 443
tomcat installed in /opt/tomcat/tomcat
changed catalina.sh to include setting CATALINA_OPTS, -Dowl_uri=http://escience.rpi.edu/schemas/
vstoapp.war deployed under webapps. All of the code is pointing to the owl files in http://escience.rpi.edu/schemas/. The code changed is under ncar.vsto.auto and ncar.vsto.auto.impl
The code under PortalBase is ncar.jar that gets put under WebContent/WEB-INF/lib

Tomcat for OPeNDAP

tomcat listening on the escience.rpi.edu ip address (128.113.28.175), port 8080 and 8443
tomcat installed in /opt/tomcat/tomcat_opendap
opendap.war deployed under webapps. No modifications made to configuration
OPeNDAP BES is installed and running under /opt/hyrax-1.5 with modules dap, dapcmd, usage, ascii, www, nc, fits and fonc (file out netcdf)

Additional information

/etc/hosts changed so that escience.rpi.edu is not listed with 127.0.0.1
/etc/hosts changed, added escience.rpi.edu for its ip address
/etc/hosts changed, added vsto.rpi.edu for its ip address

Trac Installation

The following is everything that was needed in order to install and run Trac

Downloaded Trac version 0.11.3 from TracDownload
Follow the installlation instructions at TracInstall

Running within httpd

We needed to install the latest and greatest Python, version 2.5. This was so that we could get the latest versions of easy_install (setuptools), Genshi, mod_python.

Trac is installed in the default location, under /usr
the daemon is located at /usr/bin/tracd
There is currently a single project created under /opt/var/trac/tetherless_project
- sudo trac-admin /opt/var/trac/tetherless_project initenv
- can create more projects if needed
currently, basic authentication using htaccess is being used, using the same password file that svn is using. This password file is located under /etc/httpd/conf/svn-auth-file

Trac is running using the mod_python handler in apache. The apache configuration was modified in the following way. First, trac.conf was added to the /etc/httpd/conf.d directory. The file is as follows:

<Location /trac>
   SetHandler mod_python
   PythonInterpreter main_interpreter
   PythonHandler trac.web.modpython_frontend 
   PythonOption TracEnv /opt/var/trac/tetherless_project
   PythonOption TracUriRoot /trac
</Location>

<Location /trac/login>
    AuthType Basic
    AuthName "Tetherless World Trac Server"
    AuthUserFile /etc/httpd/conf/svn-auth-file
    Require valid-user
</Location>

Added to the httpd.conf file (/etc/httpd/conf/httpd.conf), which is loaded after trac.conf, we added the following, which includes svn information:

<VirtualHost *:80>
    ServerName scm.escience.rpi.edu
    <Location /svn>
        DAV svn
        SVNPath /opt/var/svn/tetherless_repository
        AuthzSVNAccessFile /etc/httpd/conf/svn-access-file
        Satisfy Any
        Require valid-user
        AuthType Basic
        AuthName "Tetherless World Subversion repository"
        AuthUserFile /etc/httpd/conf/svn-auth-file
    </Location>
    <Location /trac>
       SetHandler mod_python
       PythonInterpreter main_interpreter
       PythonHandler trac.web.modpython_frontend 
       PythonOption TracEnv /opt/var/trac/tetherless_project
       PythonOption TracUriRoot /trac
    </Location>
    <Location /trac/login>
	AuthType Basic
	AuthName "Tetherless World Trac Server"
	AuthUserFile /etc/httpd/conf/svn-auth-file
	Require valid-user
    </Location>
</VirtualHOst>

To get to trac you can go to scm.escience.rpi.edu/trac or escience.rpi.edu/trac. If we want, we can remove the trac.conf file and just have the scm VirtualHost entry.

In Standalone Mode

At first, trac was run as a standalone server on a specific port, port 8000. To get to this port one had to go to escience.rpi.edu:8000. This is no longer the case (refer to above mod_python configuration.) To run Trac in this standalone mode use the following instructions.

trac is started using the following command:

sudo nohup tracd -s --port=8000 \
     --basic-auth=tetherless_project,/etc/httpd/conf/svn-auth-file,escience.rpi.edu:8000 \
     /opt/var/trac/tetherless_project &

to get to trac, only within the network, browse to http://escience.rpi.edu:8000/ and it will get you to the trac site for the one project, tetherless_project. If there are multiple projects then we wouldn't run with -s (single). When you browse to the specified url it will list all of the projects available.

Escience software stack

From Tetherless World Wiki

Production systems

Contents

NOTES

Currently installed on escience

Still need to install these software packages

Tomcat/Apache installation

Trac Installation

Running within httpd

In Standalone Mode

Views

Personal tools

Tetherless World

links

Internal Pages

reference

Maintenance

Search

Toolbox