1 Archiver Appliance Installation
Archiver appliance[1] is download
here.
user@user:~/Downloads# ls
archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz
1.1 License
The EPICS archiver appliance is distributed under the terms of the
EPICS license.
1.2 System requirements
A recent version of Linux
Definitely 64 bit Linux for production systems. If using RedHat, we should aim for RedHat 6.1
Sun Java JDK 1.8
Definitely the 64 bit version for production systems. We need the JDK, not the JRE.
A recent version of Tomcat 7.x
Preferably apache-tomcat-7.0.61 or later.
A recent version of Tomcat 7.x
Preferably apache-tomcat-7.0.61 or later.
Firefox or Chrome
The management UI works best with a recent version of Firefox or Chrome.
A recent version of MySQL mysql-5.1 or later
If persisting configuration to a database.
A recent version of MySQL mysql-5.1 or later
If persisting configuration to a database.
1.3 System Requirements Installation
JDK 1.8 installation
Sun Java JDK 1.8 is download
here.
Here is the quick download log for all Linux user. In my case JDK 1.8 download the /opt directory.
root@user:/opt# wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2
Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u66-b17/jdk-8u66-linux-x64.tar.gz"
Untar the downloaded file.
root@user:/opt# tar -xzf jdk-8u66-linux-x64.tar.gz
root@user:/opt# ls
jdk1.8.0_66
Here is the installation log for Debian Linux user.
root@user:/opt# update-alternatives --install /usr/bin/java java /opt/jdk1.8.0_66/bin/java 1041
root@user:/opt# update-alternatives --install /usr/bin/javac javac /opt/jdk1.8.0_66/bin/javac 1041
root@user:/opt# update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).
Selection Path Priority Status
------------------------------------------------------------
* 0 /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java 1071 auto mode
1 /opt/jdk1.8.0_66/bin/java 1041 manual mode
2 /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java 1071 manual mode
Press enter to keep the current choice[*], or type selection number: 1
update-alternatives: using /opt/jdk1.8.0_66/bin/java to provide /usr/bin/java (java) in manual mode
root@user:/opt# java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
If you are using CentOS, here is the installation log.
[root@user opt]# alternatives --install /usr/bin/java java /opt/jdk1.8.0_66/bin/java 2
[root@user opt]# alternatives --config java
There are 4 programs which provide ’java’.
Selection Command
-----------------------------------------------
*+ 1 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
2 /usr/lib/jvm/jre-1.6.0-openjdk.x86_64/bin/java
3 /usr/lib/jvm/jre-1.5.0-gcj/bin/java
4 /opt/jdk1.8.0_66/bin/java
Enter to keep the current selection[+], or type selection number: 4
[root@user opt]# alternatives --install /usr/bin/jar jar /opt/jdk1.8.0_66/bin/jar 2
[root@user opt]# alternatives --install /usr/bin/javac javac /opt/jdk1.8.0_66/bin/javac 2
[root@user opt]# alternatives --set jar /opt/jdk1.8.0_66/bin/jar
[root@user opt]# alternatives --set javac /opt/jdk1.8.0_66/bin/javac
[root@user opt]# java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
Tomcat 7.x Download
Tomcat 7.x is download
here.
user@user:~/Downloads# ls
apache-tomcat-7.0.65.tar.gz
archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz
Chrome installation
Chrome is download
here.
Here is the chrome installation log for Debian Linux user.
user@user:~/home/user/Downloads# ls
apache-tomcat-7.0.65.tar.gz
archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz
google-chrome-stable_current_amd64.deb
user@user:~/Downloads$ su
Password:
root@user:/home/user/Downloads# dpkg -i google-chrome-stable_current_amd64.deb
root@user:/home/user/Downloads# aptitude install google-chrome-stable
MySQL installation and setting
Create MySQL user and table is needed for the archiver appliance installation.
Here is the quick installation log for Debian Linux user. Other Linux users install the same name package.
Package : mysql-server
root@user:~# aptitude install mysql-server
MySQL connector download
here, and untar the download file.
user@user:~/home/user/Downloads# ls
apache-tomcat-7.0.65.tar.gz
archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz
google-chrome-stable_current_amd64.deb
mysql-connector-java-5.1.38.tar.gz
user@user:~/home/user/Downloads# tar -xzf mysql-connector-java-5.1.38.tar.gz
user@user:~/home/user/Downloads# ls
apache-tomcat-7.0.65.tar.gz
archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz
google-chrome-stable_current_amd64.deb mysql-connector-java-5.1.38.tar.gz
mysql-connector-java-5.1.38
Here is the create MySQL user and table log for Debian Linux user.
user@user:~$ mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 66
Server version: 5.5.46-0+deb8u1 (Debian)
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type ’help;’ or ’\h’ for help. Type ’\c’ to clear the current input statement.
mysql> create user username@localhost identified by ’password’;
Query OK, 0 rows affected (0.00 sec)
mysql> create database databasename;
Query OK, 0 rows affected (0.00 sec)
mysql> grant all on databasename.* to username@localhost;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
user@user:~$ mysql -u username -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 72
Server version: 5.5.46-0+deb8u1 (Debian)
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type ’help;’ or ’\h’ for help. Type ’\c’ to clear the current input statement.
mysql> show databases;
+-------------------------+
| Database |
+-------------------------+
| information_schema |
| databasename |
+-------------------------+
2 rows in set (0.00 sec)
mysql> use databasename;
Database changed
mysql> exit;
bye
If you are using CentOS, here is the create MySQL user and table log.
[root@user ~]# service mysqld start
Initializing MySQL database: Installing MySQL system tables...
OK
Filling help tables...
OK
To start mysqld at boot time you have to copy
support-files/mysql.server to the right place for your system
PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
/usr/bin/mysqladmin -u root password ’new-password’
/usr/bin/mysqladmin -u root -h node2 password ’new-password’
Alternatively you can run:
/usr/bin/mysql_secure_installation
which will also give you the option of removing the test
databases and anonymous user created by default. This is
strongly recommended for production servers.
See the manual for more instructions.
You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &
You can test the MySQL daemon with mysql-test-run.pl
cd /usr/mysql-test ; perl mysql-test-run.pl
Please report any problems with the /usr/bin/mysqlbug script!
[ OK ]
Starting mysqld:
[ OK ]
[root@user ~]# /usr/bin/mysqladmin -u root password ’password’
[root@user ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.1.73 Source distribution
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type ’help;’ or ’\h’ for help. Type ’\c’ to clear the current input statement.
mysql> create user username@localhost identified by ’password’;
Query OK, 0 rows affected (0.00 sec)
mysql> create database databasename;
Query OK, 1 row affected (0.00 sec)
mysql> grant all on databasename.* to username@localhost;
mysql> show databases;
+-------------------------+
| Database |
+-------------------------+
| information_schema |
| mysql |
| databasename |
+-------------------------+
4 rows in set (0.00 sec)
1.5 Archiver Appliance Installation Using an Install Script
Install script accommodates installations with a ”standard” set of parameters and installs the EPICS archiver appliance on one machine.
If you want to simply test the system and quickly get going, please see the Quickstart section.
1
Make sure you have a recent version of Sun Java 1.8 from Oracle by running java -version.
You should see something like so
user@user:~$ java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)
2
The downloaded installation package to a Linux machine into a brand new folder. This should give you a tar.gz file like archappl vx.x.x.tar.gz. Also, Tomcat package into a brand new folder.
user@user:~$ mkdir archappl
user@user:~$ cd Downloads
user@user:~/Downloads$ mv archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz apache-tomcat-7.0.65.tar.gz ./../archappl
user@user:~/Downloads$ cd ../archappl
user@user:~/archappl$ ls
apache-tomcat-7.0.65.tar.gz archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz
3
Untar the archappl vx.x.x.tar.gz. This should untar into 4 WAR files and a bash script
like so
user@user:~/archappl$ tar -xzf archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz
user@user:~/archappl$ ls
Apache_2.0_License.txt
install_scripts RELEASE_NOTES
apache-tomcat-7.0.65.tar.gz
LICENSE
retrieval.war
archappl_v0.0.1_SNAPSHOT_23-September-2015T07-41-14.tar.gz mgmt.war
sample_site_specific_content
engine.war
NOTICE
etl.war
quickstart.sh
4
Set JAVA HOME to point to a 1.8 JDK
user@user:~/archappl$ export JAVA_HOME=/opt/jdk1.8.0.66
5
Run the install script like so
user@user:~/archappl$ cd install_scripts
user@user:~archappl/install_scripts$ ./single_machine_install.sh
This should start the install script.
6
Soon you can see the following window.
 |
| Pick a folder for create the Tomcat instance |
It is recommended that you select an empty folder. In my case, create a archiver appliance
folder and pick this folder.
7
Soon you can see the following window.
 |
| Pick a Tomcat distribution (tar.gz) file |
Select a Tomcat distribution (tar.gz) file.
8
Soon you can see the following window.
 |
| Pick a mysql client jar |
Select a mysql client jar file.
9
Soon you can see the following window.
 |
| Question about ARCHAPPL APPLIANCES environment variable |
In my case, select the YES button. The appliance.xml file is automatically created in the selected installation folder. And it is automatically export ARCHAPPL APPLIANCES. Of course, after the installation you can modify this file. If you want generate appliance.xml, following paragraph is the way for create appliance.xml and export appliance.xml.
The appliances.xml is a file that lists all the appliances in a cluster of archiver appliance. While it is not necessary to point to the same physical file, the contents are expected to be identical across all appliances in the cluster. The details of the file are outlined in the ConfigService javadoc. A sample appliances.xml with two appliances looks like
<appliances>
<appliance>
<identity>appliance0</identity>
<cluster_inetport>machinename:16670</cluster_inetport>
<mgmt_url>http://machinename:17665/mgmt/bpl</mgmt_url>
<engine_url>http://machinename:17666/engine/bpl</engine_url>
<etl_url>http://machinename:17667/etl/bpl</etl_url>
<retrieval_url>http://machinename:17668/retrieval/bpl</retrieval_url>
<data_retrieval_url>http://machinename:17668/retrieval</data_retrieval_url>
</appliance>
</appliances>
appliances.xml (END)
The archiver appliance looks at the environment variable ARCHAPPL APPLIANCES for the location of the appliances.xml file. Use an export statement like so
export ARCHAPPL_APPLIANCES=/location/appliances.xml
10
Soon you can see the following window.
 |
| MySQL information input window |
Enter a user name, password, databasename you set in MySQL. If not set, check the MySQL installation and setting section.
11
Soon you can see the following window.
 |
| Question about MySQL database table |
The archiver appliance ships with DDL, for MySQL, this is a file called archappl mysql.sql that is located in /home/user/archappl/install scripts/archappl mysql.sql. Execute this script in you newly created schema. We select the YES button.
There should be at least these tables
- PVTypeInfo - This table stores the archiving parameters for the PVs
- PVAliases - This table stores EPICS alias mappings
- ExternalDataServers - This table stores information about external data servers.
- ArchivePVRequests - This table stores archive requests that are still pending.
mysql> show tables;
+-----------------------+
| Tables_in_databasename|
+-----------------------+
| ArchivePVRequests |
| ExternalDataServers |
| PVAliases |
| PVTypeInfo |
+-----------------------+
4 rows in set (0.00 sec)
12
Soon you can see the following window.
 |
| Question about policies.py file |
We did not create the policies.py file. We choose the No button. For more mention is back to the policies.py file.
13
Soon you can see the following window.
 |
| Done with the installation window |
Congratulations. Now just add a few step to complete installation.
14
Setting up Apache Commons Daemon
Editing the /apache-tomcat-7-0-65/conf/server.xml file to change the ports to better suit your installation
- By default, the connector port for the HTTP connector is set to 8080. Change this to the port used by the mgmt webapp for this appliance, in this example, 17665.
<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
<Connector port="17665" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />
- Remove/comment out the sections for the AJP connector.
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<!--Connector port="8009" protocol="AJP/1.3" redirectPort="8443" /-->
- At the end, there should be two ports active in the /apache-tomcat-7-0-65/conf/server.xml file, one for the HTTP connector and the other for the SHUTDOWN command.
<Server port="8005" shutdown="SHUTDOWN">
You can ignore following paragraph about log4j.properties. The log4j.properties file is automatically created in apache-tomcat-7-0-65/lib folder. However, if you want a personal log settings, look in the following paragraphs.
Setting the appropriate log4j configuration level by creating/editing the apache-tomcat-7-0-65/lib/log4j.properties. Here’s a sample that logs exceptions and errors with one exception - log messages logged to the config namespace are logged at INFO level.
# Set root logger level and its only appender to A1.
log4j.rootLogger=ERROR, A1
log4j.logger.config.org.epics.archiverappliance=INFO
log4j.logger.org.apache.http=ERROR
# A1 is set to be a DailyRollingFileAppender
log4j.appender.A1=org.apache.log4j.DailyRollingFileAppender
log4j.appender.A1.File=arch.log
log4j.appender.A1.DatePattern=’.’yyyy-MM-dd
# A1 uses PatternLayout.
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
15
Setting up sampleStartup.sh
Editing the /archiver appliance/sampleStartup.sh
- Setting up EPICS environment.
Default
source /opt/local/setEPICSEnv.sh
EPICS environment position.
source /home/user/epics/R3.14.12.5/setEpicsEnv.sh
Default
export ARCHAPPL_SHORT_TERM_FOLDER=/arch/sts/ArchiverStore
export ARCHAPPL_MEDIUM_TERM_FOLDER=/arch/mts/ArchiverStore
export ARCHAPPL_LONG_TERM_FOLDER=/arch/lts/ArchiverStore
Storage environment variables example
export ARCHAPPL_SHORT_TERM_FOLDER=/home/user/archiver_appliance/arch/STS
export ARCHAPPL_MEDIUM_TERM_FOLDER=/home/user/archiver_appliance/arch/MTS
export ARCHAPPL_LONG_TERM_FOLDER=/home/user/archiver_appliance/arch/LTS
Congratulations. Archiver appliance installation is now complete.
16
Here is how to use the script.
Usage: ./sampleStartup.sh {start|stop|restart}
Run the archive appliance like so
user@user:~/archiver_appliance$ ./sampleStartup.sh start
Starting tomcat at location /home/user/archiver_appliance/mgmt
Using 64 bit versions of libraries
~/archiver_appliance/mgmt/logs ~/archiver_appliance
~/archiver_appliance
Starting tomcat at location /home/user/archiver_appliance/engine
Using 64 bit versions of libraries
~/archiver_appliance/engine/logs ~/archiver_appliance
~/archiver_appliance
Starting tomcat at location /home/user/archiver_appliance/etl
Using 64 bit versions of libraries
~/archiver_appliance/etl/logs ~/archiver_appliance
~/archiver_appliance
Starting tomcat at location /home/user/archiver_appliance/retrieval
Using 64 bit versions of libraries
~/archiver_appliance/retrieval/logs ~/archiver_appliance
~/archiver_appliance
Bibliography
[1] The EPICS Archiver Appliance : “archiver appliance”http://slacmshankar.github.io/epicsarchiver_docs/index.html (2016.01.06)
[2] Sang-il Lee(2014), “Archiver Appliance 구성”, RISP control group technical document
