[Imports] A few questions about importing U.S. NHD data for Subbasin 01080105 import (White River in VT)

[Alan Mintz]

At 2012-02-23 21:24, probiscus wrote:
>One more thing to note, my plan is to map all of the flowlines to 
>waterway=stream, then just manually change 7 rivers that have areas to 
>waterway=river before importing. This matches my local knowledge and I 
>think this covers this area pretty well, as it is a hilly area, with few 
>manmade water features. While there are some features that should ideally 
>be ditch or canal here and there, the source data has no indication of 
>this, so that will just need to be corrected from aerials or by local 
>mappers (me).


Sounds right. I'll mention that, in Kern County, CA, which is largely 
desert, there was a huge network of tiny streams imported from NHD - 
hundreds of thousands of nodes/ways. In reality, though, most of these 
streams are, at best, intermittent and movable. At any given time, most of 
them are non-existent. It's almost as if it was more of a relief map than a 
map of true water features. In some places, the feature density was so high 
as to make downloading more than a 0.05x0.05 degree sector impossible - in 
areas that might have only a hundred features otherwise. I hope your 
situation is not similar - it was quite a job to clean those up/out.



> >> You don't need to understand what every tag means, particularly
> >> these data carried from import sources.
> > This approach makes OSM uneditable. Imagine you don't know what the
> > tag abcd=1234 means. What is a user supposed to do with that tag when....


Just to be clear - I was referring to OSM tags, not tags from the import 
database, and I stand by my statement that it is too much to ask the casual 
mapper to understand every tag - particularly, those that are specifically 
grouped in their own namespace (like tiger:*) for exactly this reason.



>While I do understand what every tag means, I think I can see the reason 
>behind both sides here. Note again that ComId is one of two ID'ing fields 
>in the NHD data, the other being ReachCode, which I will most definitely 
>import. I know importing these IDs might accomodate some "future 
>potential", but I guess the important questions are:
>
>Do we have any examples of cases where imported source IDs have been used 
>to compare/update/modify either source or OSM data from the other? Has 
>this been done successfully using a tag like 
>"tiger:tlid=171531381:171531532:171531405:171531528" or possibly even 
>"tiger:tlid=woohoo"?


I attempted to do this to update existing TIGER data from TIGER09. 
Unfortunately, we discovered a bug in the original import logic that made 
this impossible. That doesn't mean the idea is flawed - just the 
implementation in this case. It would have been quite useful to create a 
diff of names in heavily edited areas to find corrections in OSM from TIGER 
(or vice-versa).



>What IS the user supposed to do with tags like tiger:tlid when splitting 
>or merging a way?


Nothing. When splitting a way, there are now two ways that carry the same 
TLID, having come from the same TIGER way. When combining ways, the TLIDs 
are concatenated automatically (as are any other multi-value fields) by any 
reasonable editor (the program, not the human). There is the problem of the 
~255-character limit on values (see below).



> > using some sort of pseudo-ComID for these pre-combined ways.
>Any suggestions as to how this might work? Are you just suggesting 
>"NHD:ComID=combined" for those ways which have been merged? Isn't this 
>just tiger:tlids with multiple values in disguise?


I was trying to get around the 255-char limit on values, since, in my 
earlier experience, there were so many small parts of a stream to be 
combined into one. I therefore suggested creating a table with a 
newly-assigned ComID and the original ComIDs of the features that are 
combined during import.

For example, you combine features with ComIDs 90012345, 90012346, and 
90012347 together since they are contiguous and otherwise identical. Your 
mapping table looks like:

NewComID  OldComID
--------- --------
A90012345 90012345
A90012345 90012346
A90012345 90012347

When the way is imported, tag it with NHD:ComID=A90012345. The table is 
stored somewhere publicly available, just like documenting any other 
transformation tables in the wiki.
--
Alan Mintz <Alan_Mintz+OSM at Earthlink.net>