Error after upgrade to 6.9.1

Hi,
after the upgrade to version 6.9.1 platform, I have an issue on one of the server.
app-core cannot be started any more…
I think the main reason is this:
Caused by: java.sql.SQLException: Unknown column ‘VALUE’ in ‘field list’ Query: select NAME, VALUE from SYS_CONFIG Parameters: []

The interesting thing is, that the DB got updated and connection to the same repository from another machine works fine.
Also the deployment to QA and Prod, including DB update was fine.
Only the Dev server is not working any more.
I did the buildWar several times, but not beeing able to get the installation running.

Roland

and here is the complete log entry:

SEVERE: Exception sending context initialized event to listener instance of class [com.haulmont.cuba.core.sys.AppContextLoader]
java.lang.RuntimeException: Error loading DB-stored app properties cache
	at com.haulmont.cuba.core.app.ConfigStorage.loadCache(ConfigStorage.java:135)
	at com.haulmont.cuba.core.app.ConfigStorage.getDbProperty(ConfigStorage.java:106)
	at com.haulmont.cuba.core.sys.ConfigPersisterImpl.getProperty(ConfigPersisterImpl.java:44)
	at com.haulmont.cuba.core.config.ConfigGetter.getProperty(ConfigGetter.java:110)
	at com.haulmont.cuba.core.config.ConfigGetter.getProperty(ConfigGetter.java:95)
	at com.haulmont.cuba.core.config.ConfigGetter.invoke(ConfigGetter.java:67)
	at com.haulmont.cuba.core.config.ConfigHandler.invoke(ConfigHandler.java:79)
	at com.sun.proxy.$Proxy388.getBruteForceProtectionEnabled(Unknown Source)
	at com.haulmont.cuba.security.app.BruteForceProtectionBean.isBruteForceProtectionEnabled(BruteForceProtectionBean.java:149)
	at com.haulmont.cuba.security.auth.checks.BruteForceUserCredentialsChecker.onAuthenticationFailure(BruteForceUserCredentialsChecker.java:71)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:256)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:177)
	at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:140)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393)
	at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347)
	at com.haulmont.cuba.core.sys.EventsImpl.publish(EventsImpl.java:33)
	at com.haulmont.cuba.security.auth.AuthenticationManagerBean.publishAuthenticationFailed(AuthenticationManagerBean.java:357)
	at com.haulmont.cuba.security.auth.AuthenticationManagerBean.authenticateInternal(AuthenticationManagerBean.java:277)
	at com.haulmont.cuba.security.auth.AuthenticationManagerBean.login(AuthenticationManagerBean.java:115)
	at com.haulmont.cuba.security.auth.AnonymousSessionHolder.loginAnonymous(AnonymousSessionHolder.java:75)
	at com.haulmont.cuba.security.auth.AnonymousSessionHolder.initializeAnonymousSession(AnonymousSessionHolder.java:66)
	at com.haulmont.cuba.security.auth.AnonymousSessionHolder.applicationStarted(AnonymousSessionHolder.java:45)
	at com.haulmont.cuba.core.sys.AppContext.startContext(AppContext.java:239)
	at com.haulmont.cuba.core.sys.AppContext$Internals.startContext(AppContext.java:302)
	at com.haulmont.cuba.core.sys.AbstractWebAppContextLoader.contextInitialized(AbstractWebAppContextLoader.java:83)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
	at org.apache.catalina.manager.ManagerServlet.start(ManagerServlet.java:1309)
	at org.apache.catalina.manager.HTMLManagerServlet.start(HTMLManagerServlet.java:665)
	at org.apache.catalina.manager.HTMLManagerServlet.doPost(HTMLManagerServlet.java:211)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.filters.CsrfPreventionFilter.doFilter(CsrfPreventionFilter.java:136)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:595)
	at org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:348)
	at org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:52)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: Unknown column 'VALUE' in 'field list' Query: select NAME, VALUE from SYS_CONFIG Parameters: []
	at com.haulmont.bali.db.QueryRunner.rethrow(QueryRunner.java:462)
	at com.haulmont.bali.db.QueryRunner.query(QueryRunner.java:320)
	at com.haulmont.bali.db.QueryRunner.query(QueryRunner.java:411)
	at com.haulmont.bali.db.QueryRunner.query(QueryRunner.java:386)
	at com.haulmont.bali.db.QueryRunner.query(QueryRunner.java:430)
	at com.haulmont.cuba.core.app.ConfigStorage.loadCache(ConfigStorage.java:123)
	... 69 more

Jun 20, 2018 5:37:41 AM org.apache.catalina.core.StandardContext startInternal
SEVERE: One or more listeners failed to start. Full details will be found in the appropriate container log file
Jun 20, 2018 5:37:41 AM org.apache.catalina.core.StandardContext startInternal
SEVERE: Context [/app_ps-core] startup failed due to previous errors
Jun 20, 2018 5:37:41 AM org.apache.catalina.core.StandardContext listenerStop
SEVERE: Exception sending context destroyed event to listener instance of class [com.haulmont.cuba.core.sys.AppContextLoader]
java.lang.IllegalStateException: BeanFactory not initialized or already closed - call 'refresh' before accessing beans via the ApplicationContext
	at org.springframework.context.support.AbstractRefreshableApplicationContext.getBeanFactory(AbstractRefreshableApplicationContext.java:170)
	at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1090)
	at com.haulmont.cuba.core.sys.AbstractWebAppContextLoader.contextDestroyed(AbstractWebAppContextLoader.java:103)
	at org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:4792)
	at org.apache.catalina.core.StandardContext.stopInternal(StandardContext.java:5429)
	at org.apache.catalina.util.LifecycleBase.stop(LifecycleBase.java:226)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:154)
	at org.apache.catalina.manager.ManagerServlet.start(ManagerServlet.java:1309)
	at org.apache.catalina.manager.HTMLManagerServlet.start(HTMLManagerServlet.java:665)
	at org.apache.catalina.manager.HTMLManagerServlet.doPost(HTMLManagerServlet.java:211)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:661)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.filters.CsrfPreventionFilter.doFilter(CsrfPreventionFilter.java:136)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:595)
	at org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:348)
	at org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:52)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:650)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

Jun 20, 2018 5:37:41 AM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
WARNING: The web application [app_ps-core] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.util.TimerThread.mainLoop(Timer.java:552)
 java.util.TimerThread.run(Timer.java:505)
Jun 20, 2018 5:37:41 AM org.apache.catalina.loader.WebappClassLoaderBase clearReferencesThreads
WARNING: The web application [app_ps-core] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
 java.lang.Object.wait(Native Method)
 java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
 com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:43)
Jun 20, 2018 5:37:41 AM org.apache.catalina.core.ApplicationContext log
INFO: HTMLManager: list: Listing contexts for virtual host 'localhost'
Jun 20, 2018 5:38:09 AM org.apache.catalina.loader.WebappClassLoaderBase checkStateForResourceLoading
INFO: Illegal access: this web application instance has been stopped already. Could not load [com.haulmont.cuba.core.sys.persistence.MysqlDbTypeConverter]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [com.haulmont.cuba.core.sys.persistence.MysqlDbTypeConverter]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1311)
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1299)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1158)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
	at com.haulmont.bali.util.ReflectionHelper.loadClass(ReflectionHelper.java:62)
	at com.haulmont.cuba.core.sys.persistence.DbmsSpecificFactory.create(DbmsSpecificFactory.java:76)
	at com.haulmont.cuba.core.sys.persistence.DbmsSpecificFactory.create(DbmsSpecificFactory.java:66)
	at com.haulmont.cuba.core.sys.persistence.DbmsSpecificFactory.getDbTypeConverter(DbmsSpecificFactory.java:54)
	at com.haulmont.cuba.core.sys.PersistenceImpl.getDbTypeConverter(PersistenceImpl.java:93)
	at com.haulmont.cuba.core.app.ServerInfo.updateCurrentServer(ServerInfo.java:170)
	at com.haulmont.cuba.core.app.ServerInfo$1.run(ServerInfo.java:130)
	at java.util.TimerThread.mainLoop(Timer.java:555)
	at java.util.TimerThread.run(Timer.java:505)

Jun 20, 2018 5:38:09 AM org.apache.catalina.loader.WebappClassLoaderBase checkStateForResourceLoading
INFO: Illegal access: this web application instance has been stopped already. Could not load [com.haulmont.cuba.core.sys.persistence.MysqlDbTypeConverter]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [com.haulmont.cuba.core.sys.persistence.MysqlDbTypeConverter]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1311)
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1299)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1158)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
	at com.haulmont.bali.util.ReflectionHelper.loadClass(ReflectionHelper.java:62)
	at com.haulmont.cuba.core.sys.persistence.DbmsSpecificFactory.create(DbmsSpecificFactory.java:80)
	at com.haulmont.cuba.core.sys.persistence.DbmsSpecificFactory.create(DbmsSpecificFactory.java:66)
	at com.haulmont.cuba.core.sys.persistence.DbmsSpecificFactory.getDbTypeConverter(DbmsSpecificFactory.java:54)
	at com.haulmont.cuba.core.sys.PersistenceImpl.getDbTypeConverter(PersistenceImpl.java:93)
	at com.haulmont.cuba.core.app.ServerInfo.updateCurrentServer(ServerInfo.java:170)
	at com.haulmont.cuba.core.app.ServerInfo$1.run(ServerInfo.java:130)
	at java.util.TimerThread.mainLoop(Timer.java:555)
	at java.util.TimerThread.run(Timer.java:505)

Jun 20, 2018 5:38:09 AM org.apache.catalina.loader.WebappClassLoaderBase checkStateForResourceLoading
INFO: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.WarnStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.WarnStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1311)
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1299)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1158)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
	at ch.qos.logback.classic.LoggerContext.noAppenderDefinedWarning(LoggerContext.java:186)
	at ch.qos.logback.classic.Logger.callAppenders(Logger.java:264)
	at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421)
	at ch.qos.logback.classic.Logger.filterAndLog_1(Logger.java:398)
	at ch.qos.logback.classic.Logger.error(Logger.java:526)
	at com.haulmont.cuba.core.app.ServerInfo.updateCurrentServer(ServerInfo.java:194)
	at com.haulmont.cuba.core.app.ServerInfo$1.run(ServerInfo.java:130)
	at java.util.TimerThread.mainLoop(Timer.java:555)
	at java.util.TimerThread.run(Timer.java:505)

Exception in thread "ServerInfoTimer" java.lang.NoClassDefFoundError: ch/qos/logback/core/status/WarnStatus
	at ch.qos.logback.classic.LoggerContext.noAppenderDefinedWarning(LoggerContext.java:186)
	at ch.qos.logback.classic.Logger.callAppenders(Logger.java:264)
	at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:421)
	at ch.qos.logback.classic.Logger.filterAndLog_1(Logger.java:398)
	at ch.qos.logback.classic.Logger.error(Logger.java:526)
	at com.haulmont.cuba.core.app.ServerInfo.updateCurrentServer(ServerInfo.java:194)
	at com.haulmont.cuba.core.app.ServerInfo$1.run(ServerInfo.java:130)
	at java.util.TimerThread.mainLoop(Timer.java:555)
	at java.util.TimerThread.run(Timer.java:505)
Caused by: java.lang.ClassNotFoundException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.WarnStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1301)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1158)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1119)
	... 9 more
Caused by: java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [ch.qos.logback.core.status.WarnStatus]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1311)
	at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1299)
	... 11 more

found the reason: because we are using different build.gradle files, only in the actually used one the following entry got updated:ext.cubaVersion = '6.9.1’
in all other it still contained the old setting ext.cubaVersion = ‘6.8.8’ and so the wrong classes got into the build…
We will change to use only ony build.gradle file but with different sections like buildWar - buildWarDev - buildWarQA…