EPICS Archiver Appliance Installation Using an Install Script Log [Raspberry Pi]

1 Archiver Appliance Installation

Archiver appliance[1] is download here.

pi@raspberrypi ~/archappl $ ls
archappl_v0.0.1_SNAPSHOT_03-November-2015T18-04-14.tar.gz

1.1 License

The EPICS archiver appliance is distributed under the terms of the EPICS license.

1.2 System requirements

A recent version of Linux

Definitely 64 bit Linux for production systems. If using RedHat, we should aim for RedHat 6.1

Sun Java JDK 1.8

Definitely the 64 bit version for production systems. We need the JDK, not the JRE.
A recent version of Tomcat 7.x
Preferably apache-tomcat-7.0.61 or later.

A recent version of Tomcat 7.x

Preferably apache-tomcat-7.0.61 or later.

Firefox or Chrome

The management UI works best with a recent version of Firefox or Chrome.
A recent version of MySQL mysql-5.1 or later
If persisting configuration to a database.

A recent version of MySQL mysql-5.1 or later

If persisting configuration to a database.

1.3 System Requirements Installation

JDK 1.8 installation

Raspberry Pi already using oracle-java8-jdk.

pi@raspberrypi ~ $ java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) Client VM (build 25.0-b70, mixed mode)

If you do not have JDK8, Here is the quick installation log.

pi@raspberrypi ~ $ sudo aptitude istall oracle-java8-jdk

Tomcat 7.x Download

Tomcat 7.x is download here.

 pi@raspberrypi ~/archappl $ ls
apache-tomcat-7.0.65.tar.gz 

archappl_v0.0.1_SNAPSHOT_03-November-2015T18-04-14.tar.gz

MySQL installation and setting

Create MySQL user and table is needed for the archiver appliance installation.

Here is the quick installation log.

Package : mysql-server

pi@raspberrypi ~ $ sudo aptitude install mysql-server

MySQL connector download here, and untar the download file.

pi@raspberrypi ~/archappl $ ls
apache-tomcat-7.0.65.tar.gz                                mysql-connector-java-5.1.38
archappl_v0.0.1_SNAPSHOT_03-November-2015T18-04-14.tar.gz

mysql-connector-java-5.1.38.tar.gz

Here is the create MySQL user and table log.

pi@raspberrypi ~ $ mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 66
Server version: 5.5.47-0+deb7u1 (Debian)
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type ’help;’ or ’\h’ for help. Type ’\c’ to clear the current input statement.
mysql> create user username@localhost identified by ’password’;
Query OK, 0 rows affected (0.00 sec)
mysql> create database databasename;
Query OK, 0 rows affected (0.00 sec)
mysql> grant all on databasename.* to username@localhost;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye
pi@raspberrypi ~ $ mysql -u username -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 72
Server version: 5.5.46-0+deb8u1 (Debian)
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type ’help;’ or ’\h’ for help. Type ’\c’ to clear the current input statement.
mysql> show databases;
+-------------------------+
| Database                              |
+-------------------------+
| information_schema          |
| databasename                     |
+-------------------------+
2 rows in set (0.00 sec)
mysql> use databasename;
Database changed
mysql> exit;
bye

1.5 Archiver Appliance Installation Using an Install Script

Install script accommodates installations with a ”standard” set of parameters and installs the EPICS archiver appliance on one machine.

1

Make sure you have a recent version of Sun Java 1.8 from Oracle by running java -version.

You should see something like so

pi@raspberrypi ~ $ java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) Client VM (build 25.0-b70, mixed mode)

2

The downloaded installation package to a Linux machine into a brand new folder. This should give you a tar.gz file like archappl vx.x.x.tar.gz. Also, Tomcat package into a brand new folder.

 pi@raspberrypi ~/archappl $ mkdir archappl_install

pi@raspberrypi ~/archappl $ ls
apache-tomcat-7.0.65.tar.gz                                mysql-connector-java-5.1.38
archappl_install                                           mysql-connector-java-5.1.38.tar.gz
archappl_v0.0.1_SNAPSHOT_03-November-2015T18-04-14.tar.gz

3

Untar the archappl vx.x.x.tar.gz. This should untar into 4 WAR files and a bash script

like so

pi@raspberrypi ~/archappl $ tar -xzf archappl_v0.0.1_SNAPSHOT_03-November-2015T18-04-14.tar.gz
pi@raspberrypi ~/archappl $ ls
Apache_2.0_License.txt                                     mgmt.war
apache-tomcat-7.0.65.tar.gz                                mysql-connector-java-5.1.38
archappl_install                                           mysql-connector-java-5.1.38.tar.gz
archappl_v0.0.1_SNAPSHOT_03-November-2015T18-04-14.tar.gz  NOTICE
engine.war                                                 quickstart.sh
etl.war                                                    RELEASE_NOTES
install_scripts                                            retrieval.war
LICENSE                                                    sample_site_specific_content

4

Set JAVA HOME to point to a 1.8 JDK

pi@raspberrypi ~/archappl $ cd install_scripts/pi@raspberrypi ~/archappl $ export JAVA_HOME=/usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt

5

Run the install script like so

pi@raspberrypi ~/archappl/install_scripts $ ./single_machine_install.sh

This should start the install script.

6

Soon you can see the following window.

It is recommended that you select an empty folder. In my case, create a archiver appliance
folder and pick this folder.
If you starting X applications from the terminal and the warnings that follow

** (zenity:24316): WARNING **: Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-H7CXbpqO6z: Connection refused

Do not worry. The accessibility warning is a known bug with an easy workaround if you don't use any accessibility feature:

pi@raspberrypi ~ $ export NO_AT_BRIDGE=1

Now, troubleshooting is finished.

7

Soon you can see the following window.

Select a Tomcat distribution (tar.gz) file.
If you see this screen, it is something wrong.

To start up a Java program as a daemon using Jsvc, we’ll need to do a few things
Just install jsvc package. And start the jsvc.

pi@raspberrypi ~ $ aptitude install jsvc

And you can see another error.

/usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt//include/jni.h:45:20: fatal error: jni_md.h: No such file or directory
compilation terminated.

Tomcat can not find jni_md.h header file.
Here is the troubleshooting log.

 pi@raspberrypi ~ $ sudo find / -name jni_md.h
/usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt/include/linux/jni_md.h

 pi@raspberrypi ~ $ cd /usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt/include/linux
pi@raspberrypi /usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt/include/linux $ ls
jawt_md.h  jni_md.h
pi@raspberrypi /usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt/include/linux $ cp jni_md.h ./..

pi@raspberrypi /usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt/include/linux $ cd ..
pi@raspberrypi /usr/lib/jvm/jdk-8-oracle-arm-vfp-hflt/include $ ls
classfile_constants.h  jawt.h  jdwpTransport.h  jni.h  jni_md.h  jvmticmlr.h  jvmti.h  linux

8

Soon you can see the following window.

 Select a mysql client jar file.

9

Soon you can see the following window.

 In my case, select the YES button. The appliance.xml file is automatically created in the selected installation folder. And it is automatically export ARCHAPPL APPLIANCES. Of course, after the installation you can modify this file. If you want generate appliance.xml, following paragraph is the way for create appliance.xml and export appliance.xml.
The appliances.xml is a file that lists all the appliances in a cluster of archiver appliance. While it is not necessary to point to the same physical file, the contents are expected to be identical across all appliances in the cluster. The details of the file are outlined in the ConfigService javadoc. A sample appliances.xml with two appliances looks like

<appliances>
<appliance>
<identity>appliance0</identity>
<cluster_inetport>machinename:16670</cluster_inetport>
<mgmt_url>http://machinename:17665/mgmt/bpl</mgmt_url>
<engine_url>http://machinename:17666/engine/bpl</engine_url>
<etl_url>http://machinename:17667/etl/bpl</etl_url>
<retrieval_url>http://machinename:17668/retrieval/bpl</retrieval_url>
<data_retrieval_url>http://machinename:17668/retrieval</data_retrieval_url>
</appliance>
</appliances>
appliances.xml (END)

The archiver appliance looks at the environment variable ARCHAPPL APPLIANCES for the location of the appliances.xml file. Use an export statement like so

export ARCHAPPL_APPLIANCES=/location/appliances.xml

10

Soon you can see the following window.

MySQL information input window

 Enter a user name, password, databasename you set in MySQL. If not set, check the MySQL installation and setting section.

11

Soon you can see the following window.

Question about MySQL database table

 The archiver appliance ships with DDL, for MySQL, this is a file called archappl mysql.sql that is located in /home/user/archappl/install scripts/archappl mysql.sql. Execute this script in you newly created schema. We select the YES button.
There should be at least these tables

• PVTypeInfo - This table stores the archiving parameters for the PVs
• PVAliases - This table stores EPICS alias mappings
• ExternalDataServers - This table stores information about external data servers.
• ArchivePVRequests - This table stores archive requests that are still pending.

mysql> show tables;
+-----------------------+
| Tables_in_databasename|
+-----------------------+
| ArchivePVRequests           |
| ExternalDataServers        |
| PVAliases                          |
| PVTypeInfo                       |
+-----------------------+
4 rows in set (0.00 sec)

12

Soon you can see the following window.

 We did not create the policies.py file. We choose the No button. For more mention is back to the policies.py file.

13

Soon you can see the following window.

Congratulations. Now just add a few step to complete installation.

14

Setting up Apache Commons Daemon

Editing the /apache-tomcat-7-0-65/conf/server.xml file to change the ports to better suit your installation

• By default, the connector port for the HTTP connector is set to 8080. Change this to the port used by the mgmt webapp for this appliance, in this example, 17665.

<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />

 <Connector port="17665" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8443" />

•  Remove/comment out the sections for the AJP connector.

<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />

 <!--Connector port="8009" protocol="AJP/1.3" redirectPort="8443" /-->

•  At the end, there should be two ports active in the /apache-tomcat-7-0-65/conf/server.xml file, one for the HTTP connector and the other for the SHUTDOWN command.

<Server port="8005" shutdown="SHUTDOWN">

You can ignore following paragraph about log4j.properties. The log4j.properties file is automatically created in apache-tomcat-7-0-65/lib folder. However, if you want a personal log settings, look in the following paragraphs.

Setting the appropriate log4j configuration level by creating/editing the apache-tomcat-7-0-65/lib/log4j.properties. Here’s a sample that logs exceptions and errors with one exception - log messages logged to the config namespace are logged at INFO level.

# Set root logger level and its only appender to A1.
log4j.rootLogger=ERROR, A1
log4j.logger.config.org.epics.archiverappliance=INFO
log4j.logger.org.apache.http=ERROR
# A1 is set to be a DailyRollingFileAppender
log4j.appender.A1=org.apache.log4j.DailyRollingFileAppender
log4j.appender.A1.File=arch.log
log4j.appender.A1.DatePattern=’.’yyyy-MM-dd
# A1 uses PatternLayout.
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n

15

Setting up sampleStartup.sh
Editing the /archiver appliance/sampleStartup.sh

• Setting up EPICS environment.

Default

source /opt/local/setEPICSEnv.sh

EPICS environment position.

source /home/user/epics/R3.14.12.5/setEpicsEnv.sh

• Setting up storage. 

Default

export ARCHAPPL_SHORT_TERM_FOLDER=/arch/sts/ArchiverStore
export ARCHAPPL_MEDIUM_TERM_FOLDER=/arch/mts/ArchiverStore
export ARCHAPPL_LONG_TERM_FOLDER=/arch/lts/ArchiverStore

Storage environment variables example

export ARCHAPPL_SHORT_TERM_FOLDER=/home/user/archiver_appliance/arch/STS
export ARCHAPPL_MEDIUM_TERM_FOLDER=/home/user/archiver_appliance/arch/MTS
export ARCHAPPL_LONG_TERM_FOLDER=/home/user/archiver_appliance/arch/LTS

• Setting up JAVA_OPTS

Default

export JAVA_OPTS="-XX:MaxPermSize=128M -XX:+UseG1GC -Xmx4G -Xms4G -ea"

Raspberry Pi 2 have only 1G memory. So, we need to change heap size.

Be modified to suit you.

 Congratulations. Archiver appliance installation is now complete.

16

Here is how to use the script.

Usage: ./sampleStartup.sh {start|stop|restart}

Run the archive appliance like so

user@user:~/archiver_appliance$ ./sampleStartup.sh start
Starting tomcat at location /home/user/archiver_appliance/mgmt
Using 64 bit versions of libraries
~/archiver_appliance/mgmt/logs ~/archiver_appliance
~/archiver_appliance
Starting tomcat at location /home/user/archiver_appliance/engine
Using 64 bit versions of libraries
~/archiver_appliance/engine/logs ~/archiver_appliance
~/archiver_appliance
Starting tomcat at location /home/user/archiver_appliance/etl
Using 64 bit versions of libraries
~/archiver_appliance/etl/logs ~/archiver_appliance
~/archiver_appliance
Starting tomcat at location /home/user/archiver_appliance/retrieval
Using 64 bit versions of libraries
~/archiver_appliance/retrieval/logs ~/archiver_appliance
~/archiver_appliance

Bibliography

[1] The EPICS Archiver Appliance : “archiver appliance”http://slacmshankar.github.io/epicsarchiver_docs/index.html (2016.01.06)
[2] Sang-il Lee(2014), “Archiver Appliance 구성”, RISP control group technical document
[3] UNIX & LINUX : “Couldn't connect to accessibility bus” http://unix.stackexchange.com/questions/230238/starting-x-applications-from-the-terminal-and-the-warnings-that-follow(2016.02.05)