Maps add-on with large data set

Add-ons
#1

Hi,
I would like to ask what is the best approach to display a large data set of locations using the maps add-on. I have a request to display a set of approx. 20.000 location points (ports +airports) around the world. How could I implement this feature using zoom in /out events? I understand that loading all these locations on map’s initialization process is not accepted in terms of performance so I would like to find a way to load gradually this data.

Regards,
George

#3

Hi, @papageor

The maps addon supports clustering, but it is performed on the client side, which means that first the data is loaded and only after that it is being grouped into clusters. This approach is fine for not very large datasets, but for massive datasets it is needed to perform pre-clustering on the server side.

For pre-clustering on the server side you need to use a spatial database. For example, it could be PostreSQL with PostGIS extension. It provides such tools as geospatial indexes and functions, which solve such tasks as querying geometries inside the given bounds.
Make sure to use an appropriate JPA converters for your spatial DB types, since the add-on comes with the default WKT converters which are unsuitable for geo-indexing. If you choose PostGIS, you can find JPA converters for its geo-types here.
As an alternative to spatial databases you can try using in-memory spatial indexes provided by JTS library.

As you involved a spatial database, you can implement a service that will query weighted points (points with numeric values) showing how many real points are inside a certain area of a map. For example, it could be achieved using quadkeys. For a weighted point you can create a non-persistent entity like this:

@MetaClass(name = "test_WeightedPoint")
public class WeightedPoint extends BaseUuidEntity {
    @MetaProperty(datatype = "GeoPoint")
    @Convert(converter = CubaPointPostgisConverter.class)
    @Geometry
    protected Point point;

    @MetaProperty
    protected Integer weight;

    ...
}

Finally, you can specify 2 layers on a map: one for pre-clustered points (for lower zoom levels) and another one for real points (for higher zoom levels). For example, like this:

<maps:layers>
    ...
    <maps:vector id="preClusteringLayer" dataContainer="weightedPointsDc" maxZoom="10">
        <maps:cluster weightProperty="weight"/>
    </maps:vector>
    <maps:vector id="locations" dataContainer="locationsDc" minZoom="11">
        <maps:cluster/>
    </maps:vector>
</maps:layers>

Note that for preClusteringLayer we have specified weightProperty which means that we use points weights for clustering instead of their count.
You can refresh these layers on each map zoom/drag by subscribing to the corresponding event in a screen controller:

    @Subscribe("map")
    public void onMapMoveEnd(GeoMap.MoveEndEvent event) {
        GeoMap.Bounds bounds = event.getBounds();
        
        //load points for the current bounds.
    }

Regards,
Gleb

#5

Hi Gleb,
Thank you for your reply. I will try to use the first approach with PostGIS.

Regards,
George