[Imports] TIGER realignment import

Eric Fischer enf at pobox.com
Tue Aug 6 16:09:53 UTC 2013

On Tue, Aug 6, 2013 at 12:47 AM, Paul Norman <penorman at mac.com> wrote:

> > From: Eric Fischer [mailto:enf at pobox.com]
> > Sent: Monday, August 05, 2013 3:26 PM
> > To: imports at openstreetmap.org
> > Subject: [Imports] TIGER realignment import
> >
> > Hi everybody. It's probably time for me to stop just talking about
> > importing the TIGER realignment and actually do something.
> It's good to see some progress made
> > My questions are:
> >
> > 1. Is there still general agreement that it is a good idea to import
> > realignments for places that have been neglected in OSM?
> Looking at the file sizes, I'm struck by how different they are. The
> first two that me and Serge looked at were ~100K, but some are over 100MB.
> It would be useful to know which counties have so few changes it would be
> easier to manually review. Maybe a count of changed ways? The first one I
> reviewed had 14 ways in it.

Sure, I can make a list ordered from least to most edits, and manual review
should be easy for files with very few changes.

> > 2. Is it OK for me to go ahead and start submitting some of these?
> > (I'll make an import account.)
> It would be premature, since you proposed this on Monday and .osc files are
> very tricky to review when they involve geometry changes.

OK, I will hold off. And Josh Doe on imports-us showed a case where I was
letting one road cross over another, so I need to fix that before going any

> > 3. Is there a test server where I should try submitting them first?
> To test this properly what you need to do is get some sample data into a
> dev server, either master.apis.dev.openstreetmap.org, or one that you run.
> For the former you'd have to download some data, convert it into an
> uploadable file and upload it to the test server. Running one yourself
> might be easier.

Thanks. I'll try to learn how to run an API server then.

> > 5. Do the files look sensible now to people who have more experience
> > with .osc files?
> The files seem valid.
> Although not wrong in any way, I was surprised that modified nodes are
> in as <node ...>\r\n\t\t</node>. What XML library are you using to create
> the .osc files?

I'm just writing it out by hand, which I know you think is a bad idea. I
can run it through tidy or try to find an XML writer library if this is
important to you.

> Now, for some preliminary comments and questions
> 53061-0000.osc modifies way 40495680 which isn't a road at all. I'm not
> sure
> why.

It's doing the checks for anything that has a TIGER TLID, and in this case
it's an administrative boundary that came from TIGER. I can restrict it to
just roads (and railroads?) if you think that is better.

> I see a number of forestry roads in 53061-0000 being modified with no
> apparent changes in geometry. An example is way 6126562 (National Forest
> Development Road 020). It has node 50690471 at 48.117638, -121.766767 in
> it. The .osc file creates a new node at 48.117638, -121.766768 which is
> a move of about 10 mercator cm. If this was a once-off I'd not mention
> it, but most of the changes in 53061-0000 are like this.

That is strange. I would have guessed they added the node because some
invisible boundary for census tract, etc., crossed over the road, but it
looks like only that one TIGER edge uses that point. I think this is
actually legitimate, though, because the change moves node 50690471 to a
different point within the line, ~315 feet away.

What's going on here is that the TIGER reshaping of this road expands it
from 16 to 21 points, and I there's no information about which old points
correspond to which new ones except at the endpoints. I arbitrarily added
the new nodes at the end and kept the existing nodes toward the beginning,
but I could change it to add new nodes in the middle instead, or make it
try to guess which old points correspond to which new ones.

> It seems to of missed changes present in the TIGER overlay near
> 48.284822, -121.83297 in way 6118160 but it modified other parts of the
> way.
> Way 6118167 is self-intersecting. It also appears that the version in
> OSM hasn't been edited at all, so there's some algorithm failure. As it
> happens, it also doesn't correspond to anything at all on the ground.
> Way 6137727 becomes highly distorted, again with no apparent cause.

Thanks. I'll look into what happened with these and follow up.

> In 56025-0000.osc node 157668125 is modified in the .osc file but no
> changes are actually made to it. It is not moved at all. This seems to
> be fairly common and makes it hard to review the changes because there
> are so many "false positives" when looking for changes.
> In the same file node 157626325 is modified with no movement and neither
> of its parent ways are actually present in the file.

That is definitely a bug, because it should only output nodes that were
actually moved. I'll fix it.

You mentioned not including the parent ways—I am intentionally not
including the ways in the output when no nodes were added or removed, only
moved, since the way itself didn't really change. Do you think that is a
bad idea, and it should always include all affected ways?

> Overall I'd say this is actually pretty good for a first review. .oscs
> are hard to generate, and hard to do QA on.

Thanks for the detailed bug reports!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20130806/24e39c8a/attachment.html>

More information about the Imports mailing list