[Tilesathome] Recent bulk imports

[Jochen Frieling]

Hi,

> This may not necessarily be a t at h problem, but the bulk import in the
> 943,1499 area is importing what looks like thousands of nodes per tile.
> Even straight ways have hundreds of nodes.

That indeed is a problem of the data (and a little of t at h, as I'll explain
further down).
To understand why these ways have so many nodes, you have to know that
South Dakota has some streets running through the full extent of the
state. Some of the streets even continue into North Dakota. That results
in ways longer than 250 miles! All one street with the same name!
These ways have to be cut into smaller pieces by a script or by hand.

> The above tile in particular crashes xmlstarlet - I assume because of the
> excessive number of nodes in the tile.

There is a fix for xmlstarlet though. The abort happens when the limit
of nested template calls (xsltMaxDepth) is reached. An out-of-the-box
xmlstarlet installation uses a default value of 3000 for this parameter.
It is needed to catch endless recursions.
But by pushing this limit a bit, the problem at hand can be defeated.

Changing line 955 of tilesGen.pl (SVN Rev. 5202)
-    my $Cmd = sprintf("%s \"%s\" tr %s %s > \"%s\"",
+    my $Cmd = sprintf("%s \"%s\" tr --maxdepth 20000 %s %s > \"%s\"",

has done the trick.
This has successfully been tested with tile 638,1536 and the one you
stated, 943,1499. Both of which used to cause the maxdepth overflow
before.

I don't know if 20000 will always be enough. Maybe some tiles need even
more, or maybe 20000 already uses too much RAM / stack size on some
systems, so that xmlstarlet will crash muss less gracefully than it does
with a value of 3000.

Cheers,
Jochen



-- 
GMX FreeMail: 1 GB Postfach, 5 E-Mail-Adressen, 10 Free SMS.
Alle Infos und kostenlose Anmeldung: http://www.gmx.net/de/go/freemail