Is it possible to use non vaadin component, like this https://www.jstree.com/ with cuba?
Yes, you can integrate jstree into CUBA application. CUBA uses jquery 1.11.3 and jstree requires jquery 1.9.0 or greater, so it is compatible.
Follow this tutorial to integrate your component: https://doc.cuba-platform.com/manual-6.1/js_library_sample.html
It seems that jstree features are the same as CUBA Tree, you can style CUBA Tree and add icons for items using IconProvider. So perhaps you don’t need to use jstree.