Firefox stuck on Cuba: IllegalArgumentException: Control character in cookie value or attribute

Hello,

I have a CUBA App developed with studio ver 6.5.5, deployed on a cloud Linux machine (Ubuntu 16.04) where I installed Apache2 as reverse proxy to Tomcat8.

When I connect with Firefox I often get the following warnings:


Oct 16, 2017 9:10:25 PM org.atmosphere.cpr.DefaultBroadcaster addAtmosphereResource
WARNING: Duplicate resource 31fcac69-5738-4acd-ade6-a5fe272072fe. Could be caused by a dead connection not detected by your server. Replacing the old one with the fresh one

2017-10-16 21:03:32.887 WARN  [ajp-nio-127.0.0.1-8009-exec-2] com.haulmont.cuba.web.sys.CubaApplicationServlet - Too long request processing [9829 ms]: ip=xx.xx.xx.xx, url=/app/UIDL/

Oct 16, 2017 8:57:02 PM org.apache.coyote.ajp.AbstractAjpProcessor process
SEVERE: Error processing request
java.lang.IllegalArgumentException: Control character in cookie value or attribute.
        at org.apache.tomcat.util.http.LegacyCookieProcessor.isHttpSeparator(LegacyCookieProcessor.java:733)
        at org.apache.tomcat.util.http.LegacyCookieProcessor.getTokenEndPosition(LegacyCookieProcessor.java:716)
        at org.apache.tomcat.util.http.LegacyCookieProcessor.processCookieHeader(LegacyCookieProcessor.java:555)
        at org.apache.tomcat.util.http.LegacyCookieProcessor.parseCookieHeader(LegacyCookieProcessor.java:276)
        at org.apache.catalina.connector.Request.parseCookies(Request.java:3032)
        at org.apache.catalina.connector.Request.getServerCookies(Request.java:2048)
        at org.apache.catalina.connector.CoyoteAdapter.parseSessionCookiesId(CoyoteAdapter.java:1212)
        at org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:921)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:517)
        at org.apache.coyote.ajp.AbstractAjpProcessor.process(AbstractAjpProcessor.java:868)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:666)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1504)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1460)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)

When the last Error occurs I can’t Login to CUBA no matter what. Only option is to clear the Firefox History

It happens with Chrome as well, so it’s not browser dependent

Did you try accessing tomcat directly, without apache in the middle?
What’s happening is probably apache reverse proxy module sending affinity cookie that contains non-ascii character, and this usually cause problems with tomcat.
I never use apache as a reverse proxy, only nginx, and I don’t remember having such problems.
Paolo

See for example: https://www.jahia.com/home/community/forum/general/jahia-in-english/illegalargumentexception-control.html

Thank you Paolo for the answer,
Indeed everything works ok if I access tomcat directly
I’ve been investigating a bit and the issue seems related to mod_jk connector and how cookies are managed between the two
The Tomcat8 support suggested to use a different Cookie Processor in the application context.
I put the following:

<CookieProcessor className="org.apache.tomcat.util.http.Rfc6265CookieProcessor" />

In tomcat context.xml
Now I get this kind of errors:

Oct 18, 2017 5:58:40 PM org.apache.tomcat.util.http.parser.Cookie logInvalidHeader
INFO: A cookie header was received &#91;^@^@^Qgzip, deflate, br^@�^M^@^\[url=https://report-haccp.it/app/]https://some-where.com/app/[/url]^@�     ^@hJSESSIONID=DF8F3C4F8C1FF6CFE5202DFC43D7132&#93; that contained an inalid cookie. That cookie will be ignored.Note: further occurrences of this error will be logged at DEBUG level.

Do you have any clue?
You said nginx can reverse proxy with no fuzz, but I’m no expert in nginx is it easy to setup?

Based on my own experience, nginx is easier to configure as a reverse proxy than Apache, but YMMV :wink: Keep in mind that nginx has been designed for this from the ground up, and that’s a matter of fact…

I can’t share a simple example here, as my proxy config files are complex, and include things like SSL termination, caching, etc.
But you can find a lot of examples on the net, for example on the DigitalOcean docs (they have a large collection of nginx articles there).

Paolo

Ok, I’m getting a deeper insight in the issue…

I set up a custom logger in Apache to show cookies. When I refresh a CUBA page this is what I get in the “cookie” logger:

xxx.xxx.xxx.xxx - - [19/Oct/2017:10:19:07 +0200] “GET /app/VAADIN/vaadinBootstrap.js?v2017_10_05_11_04 HTTP/1.1” 304 - “https://www.somewebsite.com/app/” “Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:56.0) Gecko/20100101 Firefox/56.0” “JSESSIONID=1B7110DEE1770C6D16FD3725D62CA94A; JSESSIONID=11967D61DBE46703C77E799EBD3D61D9”

Which seems normal to me, but when I check catalina.out:

INFO: A cookie header was received [� ^@XJSESSIONID=1B7110DEE1770C6D16FD3725D62CA94A; JSESSIONID=11967D61DBE46703C77E799EBD3D] that contained an invalid cookie. That cookie will be ignored.Note: further occurrences of this error will be logged at DEBUG level.

So I get random unicodes in the cookie header, sometimes pieces of the URL… that’s incredible

Hi,

It seems that Apache does not understand encoding/charset of Tomcat 8 output, unfortunately, we haven’t seen this error before. Do you use latest version of Apache Server and mod_jk ?

Unfortunately I can’t help with apache, because as I said before I never used it as a reverse proxy frontend, and surely I’ll never do :wink:

I suggest to stop wasting time with it, and try an nginx setup on a test machine (local VM or docker, the latter gives you tons of pre-made configurations that you only need to tweak a bit, but requires some prior knowledge of the containerization concepts)

Bye
Paolo

my conf is:

Tomcat 8.0.32 Java 1.8.0_131 Ubuntu 16.04.2 LTS

Apache2 2.4.18 in reverse proxy for Tomcat using mod_jk 1:1.2.41-1 connector

I’ll try to update everything
I thought it could be depending on mod_rewrite as well so I did some tweaking there too… now I have the following rules:


<VirtualHost *:80>
...
RewriteEngine on
RewriteCond %{SERVER_NAME} =example.com [OR]
RewriteCond %{SERVER_NAME} =www.example.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

and


<VirtualHost *:443>
...
RewriteEngine on
RewriteRule ^/$ /app [R]
</VirtualHost>

It looks like Chrome is now much less prone to this issue though it does still happen… while Firefox is hanging each and every time I refresh a page

@Paolo thank you I’ll give it a try

Ok one solution is to switch to mod_proxy… so the problem seems related to ajp13.
I checked packets passing on port 8009 with tcpdump and wireshark and they look alright, so it’s either mod_jk or Tomcat I guess