[Legal-general] Proposal For OSMPD Data Repository Folder Structure

Fri Nov 7 07:51:04 GMT 2008

Sounds good and it is a start. So, we don't need any database to start the
PD project. Great!

How do we ensure that two contributors don't modify the same osm file at the
same time? Do we need a file locking mechanism?

I suppose this would be a temporary solution for GPX files and OSM files
until we get a real OSM server running?

- Kari

2008/11/7 Sunburned Surveyor <sunburned.surveyor at gmail.com>

> Joseph wrote: "Heh - not many people will comment for this reason:
> http://www.bikeshed.com/"
>
> Understood. I was just trying to build some consensus before making
> unilateral moves. That got me into trougle with the mailing list idea.
> :]
>
> Joseph wrote: "How many directories does that create in the repository? 1/4
> of a
> degree per makes it sound like a lot."
>
> Since there are 360 degrees of longitude we are talking about (in
> theory) <4*360> 1080 longitude folders. Each longitude folder would
> have (in theory) <180*4> 720 latitude folders. That would be a total
> of 777,600 folders. I agree that is a lot. However, you'd need to
> subtract out the folders for the oceans, other large bodies of water
> and places like Antartica, where we aren't likely to do any mapping.
>
> I thought we would only create the directories as they were needed. If
> a new PD enthusiast joined the project and wanted to contribute in an
> area that wasn't covered by a folder, we'd create it in the repository
> so that if fit in the folder structure I defined. This wouldn't be
> hard at all.
>
> Joseph wrote: "Surely more than any human can
> manually manage."
>
> I think a simple software tool could assist with the downloading of
> data for a particular area of interest. I'm just looking for a logical
> folder structure. There would be some interesting problems, however,
> like a road segment that falls within two (2) cells. I'm still
> thinking about how to handle that situation. Perhaps we put the road
> semment in the cell that contains the majority of the road segment
> geometry?
>
> Joseph wrote: "Another way we could do it is make quad-trees out of
> directories. If
> you imagine the root directory contains the whole planet, then four
> subdirectories each with 1/4 of the world map... and they each contain
> 4 directories which quarter the regions again and so on to some
> particular detail level."
>
> That is an interesting concept. The only disadvantabe with this
> approach is it couldn't be built incrementally as we expand our
> coverage. (At least, I don't think it could.)
>
> I will inquire about the possibility of hosting a sample OSMPD
> repository like the one I described at the OSGeo, and will report back
> to the list.
>
> The Sunburned Surveyor
>
> On Thu, Nov 6, 2008 at 2:42 PM, Joseph Gentle <josephg at gmail.com> wrote:
> > Heh - not many people will comment for this reason:
> http://www.bikeshed.com/
> >
> > How many directories does that create in the repository? 1/4 of a
> > degree per makes it sound like a lot. Surely more than any human can
> > manually manage. I guess the trick is that people only need to check
> > out a few of the directories dependent on where they live (which is
> > kinda nice).
> >
> > Another way we could do it is make quad-trees out of directories. If
> > you imagine the root directory contains the whole planet, then four
> > subdirectories each with 1/4 of the world map... and they each contain
> > 4 directories which quarter the regions again and so on to some
> > particular detail level.
> >
> > I'm not sure if it buys you much, and again you'll have a lot of
> directories.
> >
> > -J
> >
> >
> > On Fri, Nov 7, 2008 at 2:33 AM, Sunburned Surveyor
> > <sunburned.surveyor at gmail.com> wrote:
> >> I've attached a zip file with a proposed OSMPD data repository folder
> >> structure. This is only an idea of what we could do. I know there will
> >> be lots of other ideas on how we could set something up, or
> >> modifications of my proposal. I just want to help move things forward
> >> with a productive discusssion. :]
> >>
> >> Here is a brief explanation of how the folder structure is designed:
> >>
> >> The top level of the repository is divided into folders that represent
> >> areas of the globe that cover 1/4 a degree of longitude. So, for
> >> example, the folder W-119-00 contains all of the data between the west
> >> longitude of 119-00-00 and 119-15-00. Each of these longitude folders
> >> contains subfolders that cover 1/4 a degree of latitude within the
> >> strip of longitude covered in the parent folder. In this way the globe
> >> is divided into a grid of 1/4 degree "cells".
> >>
> >> Each cell has the following subfolders:
> >>
> >> gpx_files
> >> history
> >> metadata
> >> osm_files
> >> supplemental_data
> >>   raster
> >>   vector
> >> photos
> >>
> >> The "gpx_files" folder will contain raw GPS data in GPX format.
> >> The "history" folder can contain versioning information and/or
> >> information on additions and changes to the data in the cell.
> >> The "metadata" folder can contain metadata files for the data in the
> cell.
> >> The "osm_files" folder will contain OSM files produced from the GPX
> files.
> >> The "supplemental_data" folder can contain other PD data that can
> >> assist in the mapping efforts. In the United States this would likely
> >> include USGS quad maps and TIGER shapefiles, but it might also contain
> >> public domain data from other sources. In different places of the
> >> world other types of public domain data might be stored here.
> >> The "photos" folder would contain digital pictures of the features
> >> that had been/were being mapped in the cell.
> >>
> >> I think only the "Gpx_files" and "osm_files" folders would be required
> >> to contain data. The other folders would be for the optional use of
> >> those working together on data within a "cell". Perhaps, over time,
> >> individual working within the same region could agree on a basic
> >> structure for these optional folders and there contents. The content
> >> and structure of these cells would vary by region. A naming convention
> >> for photos might also be helpful, and I can see having a standard
> >> format for a simple text file with photo descriptions.
> >>
> >> It would be possible to write simple tools to work with this type of
> >> folder structure. For example: I could write a plug-in for OpenJUMP
> >> that would do the following:
> >>
> >> - Allow the user to input latitude/longitude coordinates for an area
> >> of interest.
> >> - Merge the OSM or GPS files in the zones that cover this area into
> >> single files for the area of interest.
> >> - Load and display these files in the map view on OpenJUMP.
> >>
> >> I still think it would be a good idea to appoint volunteer "Cell
> >> Coordinators" that could over see the contents of each cell folder and
> >> help coordinate mapping efforts within the cell. I believe this would
> >> help improve the quality and organization of the OSMPD data
> >> repository.
> >>
> >> I am eager to get some type of folder structure determined and put
> >> online so I can have a place to put my data.
> >>
> >> OK. Now you can tear my proposal apart! :]
> >>
> >>
> >> The Sunburned Surveyor
> >>
> >> P.S. - To view the complete folder structure in the attached zip file
> >> you need to click on the W-121-00 longitude folder and then the
> >> N-37-00 latitude folder. This folder is the one that covers my home
> >> town.
> >>
> >> _______________________________________________
> >> Legal-general mailing list
> >> Legal-general at openstreetmap.org
> >> http://lists.openstreetmap.org/listinfo/legal-general
> >>
> >>
> >
>
> _______________________________________________
> Legal-general mailing list
> Legal-general at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/legal-general
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/legal-general/attachments/20081107/91b312ae/attachment.html>