Filter non-peristent entities

Ideas
#1

Hello

It would be nice to be able to filter a DTO using the CUBA platform filter.
Is there a plan to “decouple the generic Filter component from JPQL queries” in the next half year?
Maybe for JMIX only?

Kind regards,
Mladen

#2

Hi Mladen,

I’ve created an issue for Jmix: Filter component should work with DTO entities · Issue #709 · jmix-framework/jmix · GitHub

Could you describe your use case, how would you like to use generic filter with DTOs?

Regards,
Konstantin

#3

Hello dear Konstantin, thanks for opening the issue - here is my answer to your question.

At a high level, I would like the generic filtering to have equivalent functionality when filtering the non-persistent JPA entity. Bonus is if the generated user interface can be the same, or look the same, with abilities to edit and save the filter, make it global, default, etc - just like it is now used to filter the JPA entities.

Consider this use case:

  • use the external data source, such as postcode/address finder service
  • you get the initial result into a DTO, present it via a lookup screen
  • but the result set is large - maybe there are 100-200 addresses, so it requires the user to scroll a lot
  • you want to give an option to the user to further filter the result set to pick the address on the lookup screen
  • you can write some filtering on your own, and it can become tedious - would it not be nice not to, and get it out of the box just like with the usual browse/lookup screens

How to do it? Some research is needed.
(I may not be correct to the detail here)
As it is now, we have a dataloader with base query, that is modified by JPQL where clauses generated as a result of the filter. So, it’s not that it is filtering, it’s a re-query with an equivalent result.
A new result is then passed to the collection container, and then it’s passed to the table UI component to display.

It looks like one would need to implement filtering on the result collection.
(The table has the ability to sort the result by column - is it also manipulating “order by” or is it operating on the collection itself? - maybe extend that)

Is there anything that can JPQL an in-memory object? Seems no.
Some trick with HSQL database - no, memory costly, and you probably can’t generate and load an entity during runtime because can’t hot-swap loaded JPA entity model.
Doing something nasty to Eclipselink is not an option, because one will lose compatibility if something changes.

If entity collection is a JPA entity, proceed as now, if it is non-persistent, filter the collection.
Maybe there is already a framework for that (Eclipse, Apache, Guava), or one can filter collection (Map, List, Set) using stream filtering.
Is there a way to reuse the current filter UI to generate query conditions as Konstantin said?
Can the new filter be also applied elsewhere, for more generic purposes, to filter a collection?
Consult the Platform team, they were recently asking about components such as address finders, email checkers, etc.
Can it be applied to REST API addon - filter the result.

Hope this helps,
Mladen

#4

Thanks for your considerations.

Actually, in Jmix the generic filter component doesn’t directly modify the JPQL, instead it creates a tree of conditions and sets it to LoadContext. So a developer can get the conditions from LoadContext passed to loadDelegate or even deeper in a custom DataStore and interpret the conditions as needed, for example to query an external service with a specific set of parameters.

The problem with the filter in Jmix now is that it doesn’t show attributes of DTO in its UI, which is I think relatively easy to fix.

However, I still doubt that generic filter is a good choice for working with arbitrary external services like REST endpoints. Such services usually have a limited set of parameters, while generic filter gives users too much freedom in expressing filter conditions: attributes, various operations, grouping. So in most cases a developer will not be able adequately map filter conditions to actual requests.

#5

I stand corrected regarding the generic filter inner working.

Regarding the generic filter giving users too much freedom, maybe there is a way to limit the available attributes, or even lock the filter operations?

#6

Yes you can restrict the list of attributes, but not operations.
We could introduce a parameter to restrict the filter with design-time configurations only, but then a benefit of using the generic filter over of a hardcoded set of filtering fields would be minimal.

#7

Hey @krivopustov, we are working on a project which is building a lot of Dtos as we have a lot of entity related complexities and that’s why we are using the DTO strategy.
We would like to get the filter benefits which are coming out of the box for entities but seems like for dtos is not possible yet.
If you have any work around to show us so we can achieve it would be great.

Best Regards,
Dzhuliyan Ovcharov

#8

Hi Dzhuliyan,

Are you sure you will be able to properly handle all conditions generated by the filter?

When you use filter with JPA entities, the framework converts generated conditions to JPQL query for you (see ConditionJpqlGenerator.java). What are you going to do with these conditions for a DTO?