{"id":1608,"date":"2014-05-19T11:20:27","date_gmt":"2014-05-19T10:20:27","guid":{"rendered":"http:\/\/www.xenonique.co.uk\/blog\/?p=1608"},"modified":"2014-05-20T11:42:20","modified_gmt":"2014-05-20T10:42:20","slug":"staying-alive-with-atlassian-stash","status":"publish","type":"post","link":"https:\/\/www.xenonique.co.uk\/blog\/2014\/05\/19\/staying-alive-with-atlassian-stash\/","title":{"rendered":"Staying Alive with Atlassian Stash"},"content":{"rendered":"

Don’t get me wrong, I like the simplicity of the user interface in Atlassian Stash, what is too painful for me is the networking configuration between it and Apache HTTPD server. It really should be very simple, but I find that mod_proxy drops more connections than a butterfingered wicket keeper on the field.<\/p>\n

Apparently, this configuration in Mod Proxy should work e.g \/etc\/apache2\/mods_available\/proxy.conf<\/p>\n

\r\n<IfModule mod_proxy.c>\r\nProxyPass        \/stash https:\/\/localhost:7990\/stash keepalive=On connectiontimeout=300 timeout=300\r\nProxyPassReverse \/stash https:\/\/localhost:7990\/stash keepalive=On\r\n<\/IfModule>\r\n<\/pre>\n
This is based on the official Atlassian Stash documents, the difference is I have added keepalive=On<\/strong><\/code>. The HTTP connection should be kept alive for the long time.<\/p>\n
And yes, I have already looked around the official Apache documents on mod_proxy<\/a> and played around with the connectiontimeout<\/code>, timeout<\/code> and ttl<\/code> parameters to no avail. I found the keepalive<\/code><\/strong> parameter and that helped alot.<\/p>\n
I also found that on my virtual environment that is provided by my hosting company that Tomcat tooks a very long time to launch. I suspected also my hosting company VPS (virtual private server) was closing connections from time to time, but when I asked them, they answered that they do not do that with their customer’s policy. So the apache is losing the HTTP connection or the Tomcat server is dropping it.<\/p>\n
Ok, I have seen the product warnings in the actual output of the shell script file:<\/p>\n
\r\n... Detecting JVM PermGen support...\r\nPermGen switch is supported. Setting to 256m\r\n\r\nUsing STASH_HOME:      \/etc\/acme\/protect\/innocent\r\nUsing CATALINA_BASE:   \/opt\/atlassian-stash-2.10.1\r\nUsing CATALINA_HOME:   \/opt\/atlassian-stash-2.10.1\r\nUsing CATALINA_TMPDIR: \/opt\/atlassian-stash-2.10.1\/temp\r\nUsing JRE_HOME:        <strong>\/opt\/jdk1.7.0_51<\/strong>\r\n...\r\nSuccess! You can now use Stash at the following address:https:\/\/localhost:7990\/\r\n<strong>If you cannot access Stash at the above location within 3 minutes<\/strong>, or encounter any other issues starting or stopping Atlassian Stash, please see the troubleshooting guide at:\r\nhttps:\/\/confluence.atlassian.com\/display\/STASHKB\/Troubleshooting+Installation\r\n<\/pre>\n
When I checked the logs of the embedded Tomcat server with the Atlassian Stash product, I see very slow times:<\/p>\n
\r\n<strong>INFO: Starting Servlet Engine: Apache Tomcat\/7.0.47<\/strong>\r\n<strong>May 19, 2014 9:45:53 AM org.apache.catalina.core.AprLifecycleListener init<\/strong>\r\nINFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: \/opt\/atlassian-stash-2.10.1\/lib\/native:\/magic\/etc\/acme\/protect\/git\/stash\/repository\/lib\/native\r\nMay 19, 2014 9:45:53 AM org.apache.tomcat.util.digester.SetPropertiesRule begin\r\nWARNING: [SetPropertiesRule]{Server\/Service\/Engine\/Host} Setting property 'xmlValidation' to 'false' did not find a matching property.\r\nMay 19, 2014 9:45:53 AM org.apache.tomcat.util.digester.SetPropertiesRule begin\r\nWARNING: [SetPropertiesRule]{Server\/Service\/Engine\/Host} Setting property 'xmlNamespaceAware' to 'false' did not find a matching property.\r\nMay 19, 2014 9:45:55 AM org.apache.coyote.AbstractProtocol init\r\nINFO: Initializing ProtocolHandler ["http-bio-7990"]\r\nMay 19, 2014 9:45:55 AM org.apache.catalina.startup.Catalina load\r\n<strong>INFO: Initialization processed in 3734 ms<\/strong>\r\nMay 19, 2014 9:45:55 AM org.apache.catalina.core.StandardService startInternal\r\nINFO: Starting service Catalina\r\nMay 19, 2014 9:45:55 AM org.apache.catalina.core.StandardEngine startInternal\r\nINFO: Starting Servlet Engine: Apache Tomcat\/7.0.47\r\nMay 19, 2014 9:49:01 AM org.apache.coyote.AbstractProtocol start\r\nINFO: Starting ProtocolHandler ["http-bio-7990"]\r\nMay 19, 2014 9:49:01 AM org.apache.catalina.startup.Catalina start\r\n<strong>INFO: Server startup in 186153 ms<\/strong>\r\nMay 19, 2014 9:49:21 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate\r\nINFO: Initiating Jersey application, version 'Jersey: 1.8-atlassian-12 07\/11\/2013 01:08 AM'\r\nMay 19, 2014 9:49:21 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate\r\nINFO: Initiating Jersey application, version 'Jersey: 1.8-atlassian-12 07\/11\/2013 01:08 AM'\r\nMay 19, 2014 9:49:21 AM com.sun.jersey.api.wadl.config.WadlGeneratorLoader loadWadlGenerator\r\nINFO: Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorApplicationDoc\r\nMay 19, 2014 9:49:21 AM com.sun.jersey.api.wadl.config.WadlGeneratorLoader loadWadlGenerator\r\nINFO: Loading wadlGenerator com.sun.jersey.server.wadl.generators.WadlGeneratorGrammarsSupport\r\n<strong>May 19, 2014 9:49:21 AM com.sun.jersey.api.wadl.config.WadlGeneratorLoader loadWadlGenerator<\/strong>\r\nINFO: Loading wadlGenerator com.atlassian.plugins.rest.doclet.generators.resourcedoc.AtlassianWadlGeneratorResourceDocSupport\r\n<\/pre>\n
There is a clue here, inspect the timing between the WADL generation and the actually server up time. It is 4 minutes longer than necessary. Why is this WADL generation not pre-generated or cached on each server run into a database? There is should be 4 minutes start up time on a Tomcat server for an enterprise application in 2014!<\/p>\n
So I have to two issues with Stash. First it runs like a dog (“a slow dog, I have know many fast real dogs, mongrels that can just tear up the track”) and second it has a long start up time. It is not emotional but a mystery. Stash is not quite reliable for me to recommend it to others or to clients.<\/p>\n
Postscript<\/h3>\n
I am currently using the following settings in the proxy.conf<\/code><\/p>\n
\r\n<IfModule mod_proxy.c>\r\nProxyPass        \/stash https:\/\/localhost:7990\/stash keepalive=On connectiontimeout=300 timeout=300\r\nProxyPassReverse \/stash https:\/\/localhost:7990\/stash \r\n<\/IfModule>\r\n<\/pre>\n
And I have two system cron jobs to reboot Apache and Stash every few hours, which works something like this:<\/p>\n
\r\n10 8,11,13,15,19 * * * \/etc\/init.d\/apache2 restart\r\n08 8,11,13,15,19 * * * \/etc\/init.d\/Stash.sh force-kill-restart\r\n<\/pre>\n
I will file a ticket with Atlassian support, because I should have to continuously reboot services like this.
\nFirst, I will upgrade to the very latest version of Stash 2.12<\/a>.
\nI just tried to upgrade from 2.10.1 to 2.12.3. I backed out of it, because I need access to the special port and I cannot do it from here remotely.<\/p>\n","protected":false},"excerpt":{"rendered":"
Don’t get me wrong, I like the simplicity of the user interface in Atlassian Stash, what is too painful for me is the networking configuration between it and Apache HTTPD server. It really should be very simple, but I find that mod_proxy drops more connections than a butterfingered wicket keeper on the field. Apparently, this […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[80,174,81,175,82,139],"tags":[],"_links":{"self":[{"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/posts\/1608"}],"collection":[{"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/comments?post=1608"}],"version-history":[{"count":9,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/posts\/1608\/revisions"}],"predecessor-version":[{"id":1615,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/posts\/1608\/revisions\/1615"}],"wp:attachment":[{"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/media?parent=1608"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/categories?post=1608"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.xenonique.co.uk\/blog\/wp-json\/wp\/v2\/tags?post=1608"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}