Cannot access Spotfire server - no Web interface (only SSH works)

[Georgi Koemdzhiev]

Recently one of our Spotfire environments got down out of a sudden (not sure what happened). When I try to open the server's web interface (i.e. access "Notes & Services") I get that:

 

When I SSHed to it and ranps -ef | grep tomcatit listed only one process:

user 27247 26393 0 11:33 pts/1 00:00:00 grep --color=auto tomcatWhen I did the same in a working Spotfire server environment it showed 2 processes (including the one above):

root 1623 1 1 Nov02 03:15:21 /home/ec2-user/tss/jdk/jre/bin/java -Djava.util.logging.config.file=/home/ec2-user/tss/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -server -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -Xms512M -Xmx4096M -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dcom.sun.management.jmxremote -Dorg.apache.catalina.session.StandardSession.ACTIVITY_CHECK=true -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector -Dignore.endorsed.dirs= -classpath /home/ec2-user/tss/tomcat/bin/bootstrap.jar:/home/ec2-user/tss/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/home/ec2-user/tss/tomcat -Dcatalina.home=/home/ec2-user/tss/tomcat -Djava.io.tmpdir=/home/ec2-user/tss/tomcat/temp org.apache.catalina.startup.Bootstrap start

user 17726 17677 0 13:38 pts/2 00:00:00 grep --color=auto tomcatSo what I did is I copied the command (from the working ENV) that was missing and executed it in the broken ENV as root:

sudo /home/user/tss/jdk/jre/bin/java -Djava.util.logging.config.file=/home/user/tss/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -server -XX:+AlwaysPreTouch -XX:+UseG1GC -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -Xms512M -Xmx4096M -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Dcom.sun.management.jmxremote -Dorg.apache.catalina.session.StandardSession.ACTIVITY_CHECK=true -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector -Dignore.endorsed.dirs= -classpath /home/user/tss/tomcat/bin/bootstrap.jar:/home/user/tss/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/home/user/tss/tomcat -Dcatalina.home=/home/user/tss/tomcat -Djava.io.tmpdir=/home/user/tss/tomcat/temp org.apache.catalina.startup.Bootstrap startAfter that I was able to access the Web interface, well sort of as I was getting this error:

 

When I checked the console output after I saw the above error on the Web interface I saw that error:

14-Nov-2019 13:46:39.443 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 52682 ms

14-Nov-2019 13:47:49.187 SEVERE [http-nio-80-exec-6] org.apache.catalina.core.StandardWrapperValve.invoke Servlet.service() for servlet [default] in context with path [/spotfire] threw exception [Filter execution threw an exception] with root cause

java.lang.IllegalArgumentException: Could not resolve placeholder 'SPOTFIRE_AUTH_CONFIG' in value "file:${SPOTFIRE_AUTH_CONFIG}/spotfire-auth.properties"

at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:172)

at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:124)

at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:237)

at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:211)

at org.springframework.core.env.AbstractEnvironment.resolveRequiredPlaceholders(AbstractEnvironment.java:571)

at org.springframework.context.annotation.ConfigurationClassParser.processPropertySource(ConfigurationClassParser.java:451)

at org.springframework.context.annotation.ConfigurationClassParser.doProcessConfigurationClass(ConfigurationClassParser.java:272)

at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:245)

at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:202)

[Jens Borgland]

That last part there looks like some custom authentication component or such

[Georgi Koemdzhiev]

Yes, you are correct. We are using a custom authenticator to authenticate our users 

[Jens Borgland]

Right. The stack trace isn't complete so it's hard to tell exactly what goes wrong but it looks like that parameter (SPOTFIRE_AUTH_CONFIG) isn't set.

[Georgi Koemdzhiev]

It turned out that somebody has started the tomcat server without root. That's why our environment variable "SPOTFIRE_AUTH_CONFIG" was not "recognised". As soon as I executed:

/etc/init.d/tss-10.0.1stop|start

 

Everything worked fine.