[OSM-dev] A PostgreSQL extension for real-time simplification of objects in OSM API database

[RTOSM DOOPAS]

The simplified line is not 'saved' in the database, It is assembled
from selected nodes on-the-fly.

The rtosm extension will add several tables to the OSM API database
and do some computation on the ways and relations to derive a weight
for each node. The weight of each node will be used to determine which
node can be selected to assemble a way or relation.

The rtosm query processing is much like the original API v0.6 map call:

The original map call:

1. First the bounding box (bbox) is checked to make sure that it is sane.
2. All nodes in the bbox are searched,
3. All the ways that reference those nodes are found.
4. All Nodes that are referenced by those ways are fetched and added
to the list of nodes.
5. All the relations that reference the already found nodes and ways
are fetched.
6. All the nodes and ways that are referenced by those ways are fetched.
7. Finally all the xml is returned.

The rtosm map call (not that exact, but the framework is clear):

1. no need to check the bbox size, but need the user specified how
many nodes he/her want received to represent the bbox area.
2. A top k query in the bbox is issued to select the k nodes according
to their weight.
3.  All the ways that reference those nodes are found.
4.  All Nodes that are ancestor node (ancestor node is a concept in
tree structure which is used to represent a spatial object such as way
or relation) of already found nodes are found.
5. Ways/relations are assembled by found nodes. They are the simplified objects.
6. send back the  simplified objects to request.

.

And yes, the ST_SimplifyPreserveTopology can simplify an object by
applying Douglas-Peucker algorithm. But it needs firstly to fetch the
whole object from storage before the alogrithm can pick some nodes
from it. It cost too much to  read a large object from storage. The
better solution is to fetch only needed nodes with enough weight.

rtosm is an extension for Postgresql database without a PostGIS extension.

On Wed, Nov 16, 2016 at 5:52 PM, Darafei "Komяpa" Praliaskouski
<me at komzpa.net> wrote:
> Yes, but after you do such a select, you can't save this simplified line
> back to OSM.
>
> Is it different from
>
> SELECT tags,
> ST_AsGeoJSON(ST_SimplifyPreserveTopology(ST_RemoveRepeatedPoints(way, 1000),
> 10000)) from planet_osm_polygon where name='United States of America' and
> boundary='administrative';
>
> In osm2pgsql imported with --hstore-all key?
>
> ср, 16 нояб. 2016 г. в 7:30, RTOSM DOOPAS <rtosmdb at gmail.com>:
>>
>> Yes, osm2pgsql/imposm are very useful tool.
>>
>> While the osm2pgsql and imposm are used to import data from API
>> Database to a PostGIS instance. With the PostGIS, user can perform
>> tile rendering or geospatial computing.
>>
>> The purpose of the osm2pgsql/imposm is different. rtosm aims to make
>> the API database more flexible and versatile to have the ability to
>> answer following queries :
>>
>> "select the national boundary of U.S. in 1000 (or any other number,
>> you name it) nodes and tell the simplification error of it"
>>
>> So user can draw any huge features in real-time in browser or JOSM
>> with its simplified version. the online data editing and online data
>> viewing can be more efficient.
>>
>> let's take an example, The relation with ID 6038068 (islands of
>> Britain)  is comprised by 4596 ways and 634414 nodes. without
>> simplification, even with a naive simplification which only preserve
>> the start and end point of of each way will result in 4596 nodes to
>> represent the feature, it is hard to visualized and manipulated with
>> so many nodes retrieved from database. The rtosm can simplify the
>> relation with any number of nodes.
>>
>> In short, rtosm is about to extend the OSM API to make it support
>> viewing and editing data at any scale directly from database.
>>
>> On Wed, Nov 16, 2016 at 6:36 AM, Darafei "Komяpa" Praliaskouski
>> <me at komzpa.net> wrote:
>> > Hi!
>> >
>> > Have you tried osm2pgsql and/or imposm? What is the reason to do it over
>> > API
>> > db schema?
>> >
>> > вт, 15 нояб. 2016 г. в 19:27, RTOSM DOOPAS <rtosmdb at gmail.com>:
>> >>
>> >> Hi,
>> >>
>> >>
>> >>
>> >> I have written a small extension for OSM API database to offer the
>> >> functionality of real-time simplification of objects (ways, relations)
>> >> by
>> >> node filtering during the query processing. The idea behind it is as
>> >> followings:
>> >>
>> >>
>> >> 1. For each spatial object such as way or relation, attach weight value
>> >> to
>> >> each component node by geometric computation.
>> >>
>> >>
>> >> 2. In processing query, only retrieve nodes with the top k weight.
>> >>
>> >>
>> >> 3. Assembly the simplified objects with filtered nodes.
>> >>
>> >>
>> >>
>> >> In fact, there are some really complex concepts, algorithms and data
>> >> structures in the extension, but the essential idea is that simple.
>> >>
>> >>
>> >> With this extension, The OSM API database will have the ability to
>> >> answer
>> >> windowing query with arbitrary size by a “top k” operator to limit the
>> >> output volume of the results.
>> >>
>> >>
>> >> The purpose of the extension is to make the OSM API database can serve
>> >> data in any scale like the tile server can do.
>> >>
>> >>
>> >>  The codes in the extension is a bit of preliminary. However, to make
>> >> the
>> >> idea work, Not only the code need to be examined and tested, but also
>> >> the
>> >> database must be “normalized” to take the advantages of this extension.
>> >>
>> >>
>> >> The code is hosted in github, https://github.com/rtosm/rtosm
>> >>
>> >>
>> >> The "rtosm" means a database for real-time openstreetmap. Welcome to
>> >> comment.
>> >>
>> >> _______________________________________________
>> >> dev mailing list
>> >> dev at openstreetmap.org
>> >> https://lists.openstreetmap.org/listinfo/dev