[OSM-dev] Suggestion about design of OSM data structure

[Steve Coast]

Chien

Best to discuss this on dev@

A lot of what you talk about is in svn already.

On 26 Jul 2007, at 08:51, Chien Nguyen Khac wrote:

> Hi,
>
> I have one suggestion about design of OSM data structure. Here is  
> my suggestion:
>
> I have studied osm data stucture.
> osm data stucture contains 3 main components (nodes,segments,ways):
> nodes (id, lat, lon, tag)
> segments (id, fromnodeid, tonodeid)
> ways (id, segmentid[0],segmentid[1],...,segmentid[n], tag)
>
>
> Actually, to store a map data we need only 2 components: nodes and  
> ways.
> nodes (id, lat, lon, tag)
> ways (id, nodeid[0],nodeid[1],...,nodeid[n], tag)
>
> I wonder why osm foundation introduce segment component in between  
> node and way component? So I read the osm's wiki to find the  
> answer.
> osm foundation said that they need to use segment component because  
> ways can share segment!
> Well, it may be true in theory but in the real life how many  
> segments have been shared among ways? I guess not so many.
>
> So, by introducing segment component in between node and way to  
> save some spaces (sharing segments between ways), osm foundation  
> make osm data structure design waste a lot of space when they  
> output the osm data to planet.osm in xml format!
>
> Users (who use planet.osm like me) need to write our own program  
> (perl, java or whatever language) to process a 4GB planet.osm text  
> file. And this 4GB will grow up very fast in the near furture.
>
> Furthermore, the current osm data structure design  
> (nodes,segments,ways) make thing very dificult to process  
> planet.osm xml file.
>
> Forexample: if I want to get the data from London city from  
> planet.osm, what I need to do?
> My program need to do the following tasks:
>
> 1. Open and read planet.osm xml file
> 2. Trim all the nodes that are outside London's bounds -->  
> write result to file
> 3. Trim all segments that are outside London's bounds -->  
> write result to file
> 4. Trim all ways that are outside London's bounds --> write  
> result to file
>
>
> Task 2 is very simple, just check the node's lat and lon within  
> the bounds --> after task 2 finish we have a set of nodes within  
> bounds
> Task 3 is a little bit complicated, we need to check segment's  
> fromnodeid and tonodeid against the set of nodes within bounds -- 
> >  after task 3 finish we have a set of segments within bounds
> Task 4 is extremely complicated, we need to check way's segment  
> against the set of segments within bounds.
>
>
> If we can implement a new data structure like this:
> nodes (id, lat, lon, tag)
> ways (id, nodeid[0],nodeid[1],...,nodeid[n], tag)
>
> Then we can save a lof of space in planet.osm xml file and it is  
> also easy for other programs to process planet.osm file.
>
> Other programs just need to do fewer tasks:
> 1. Open and read planet.osm xml file
> 2. Trim all the nodes that are outside London's bounds -->  
> write result to file
> 3. Trim all ways that are outside London's bounds --> write  
> result to file
>
> The program will require less memory and run faster because it does  
> not need to allocate memory to store and process segment components.
>
>
>
>
> You can reply to my post here:
> http://osm.na1400.info/viewtopic.php?id=231
>
> Regards,
>
>
>
> ______________________________________________________________________ 
> ______________
> Choose the right car based on your needs.  Check out Yahoo! Autos  
> new Car Finder tool.
> http://autos.yahoo.com/carfinder/
>

have fun,

SteveC | steve at asklater.com | http://www.asklater.com/steve/