[OSM-dev] Planet OSM 'bad result during COPY'
Jon Burgess
jburgess777 at googlemail.com
Thu Apr 15 22:49:47 BST 2010
On Thu, 2010-04-15 at 14:24 -0700, Seth Voltz wrote:
> Jon and Frederik: good hunch, the Austrian import failed with:
>
>
> --------------------------------
> #> osm2pgsql --database world --username gis --password --verbose
> austria.osm.bz2
> ... (cut) ...
> Reading in file: austria.osm.bz2
> Processing: Node(6760k) Way(0k) Relation(0k)planet_osm_point - bad
> result during COPY, data 430058975 \N \N \N \N
> \N \N \N \N \N \N \N \N \N
> \
> N \N \N \N \N \N \N \N
> bus_stop \N \N \N \N \N \N \N
> \N \N \N \N Altöttinger Straße/Aldi \N \N
> \
> N \N \N \N \N \N \N \N \N
> \N \N \N \N \N \N \N \N \N
> \N SRID=900913;POINT(1398018.41536173 6112460.23312236)
> ... (cut) ...
> --------------------------------
>
>
> I double checked the database: everything, including my GIS templates
> and the working database are UTF8:
>
>
> --------------------------------
> #> psql -U gis -l
> ... (cut) ...
> world | gis | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
> --------------------------------
>
>
> Also, I just noticed that when it fails it also crashes the whole
> PostgreSQL server requiring a `service postgresql restart` to get
> going again. This started when I began using the --verbose switch.
> Failed imports with and without --slim and without --verbose just fail
> but leave the database alone.
Anything which crashes the database backend sounds more like an issue in
Postgresql rather than osm2pgsql.
One thing you might want to try is to recompile osm2pgsql against the
PostgreSQL-8.4 client library. There is a good chance that a binary
built in the normal way will be linking against the standard system
library (probably /usr/lib/libpq.so.4) not the postgresql-8.4 (probably
libpq.5 somewhere).
To find out the library it is using try:
$ ldd ~/bin/osm2pgsql | grep libpq
libpq.so.5 => /usr/lib64/libpq.so.5 (0x00007f6311499000)
The above output is from a F-12 system using PostgreSQL-8.4.3
You will need to install the pgdg postgresql-devel package and maybe
tweak the include path in the Makefile to pick up the right version when
you recompile.
>
> The log file in /var/log/pgsql is empty (0 bytes).
>
>
> Matt & Peter: good catch, it was a typo. I have 8.4.3 from the pgdg84
> repository which I setup with:
>
>
> --------------------------------
> #> rpm -ivh
> http://yum.pgsqlrpms.org/reporpms/8.4/pgdg-centos-8.4-2.noarch.rpm
> --------------------------------
>
>
>
> — Seth
>
>
> Seth Voltz | CTO | Shift Inc.
> seth.voltz at shiftresearch.com | www.shiftresearch.com
>
> voice: (774) 262-9092
> _________________________________________________
> This email and any attachments thereto may contain private,
> confidential, and privileged
> material for the sole use of the intended recipient. Any review,
> copying, or distribution of
> this email (or any attachments thereto) by others is strictly
> prohibited. If you are not the
> intended recipient, please contact the sender immediately and
> permanently delete the original
> and any copies of this email and any attachments thereto.
>
>
>
>
>
> On Apr 15, 2010, at 1:49 PM, Jon Burgess wrote:
>
> > On Thu, 2010-04-15 at 12:01 -0700, Seth Voltz wrote:
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA256
> > >
> > > I've searched high and low for a solution to this problem and have
> > > found nothing that works. I have successfully imported a CloudMade
> > > extract of California into my PostGIS database and rendered tiles
> > > from it using the Mapnik scripts. Able to confirm basic
> > > functionality, I decided to move up to a full planet import. I
> > > have tried 3 planet files (100317, 100401 and 100407) with the
> > > same error at different points for each file and sometimes
> > > different places for the same file on different runs. No import is
> > > able to get past the nodes import. From searching on the forums it
> > > at first seemed the error was being caused by duplicate data in
> > > the Planet file and running without --slim would bypass the
> > > problem. Here is the latest run on the 100401 planet export
> > > without slim mode:
> > >
> > > - ---------------------------------------------------
> > > Reading in file: planet-100401.osm.bz2
> > > Processing: Node(61870k) Way(0k) Relation(0k)planet_osm_point -
> > > bad result during COPY, data 86767681 \N \N \N
> > > \N \N \N \N \N \N \N \N
> > > \N \N \N \N \N \N \N \N
> > > \N \N \N \N \N \N \N \N
> > > \N \N \N \N \N \N
> > > Unterdürnbach \N \N \N \N \N \N
> > > village \N \N \N \N \N \N \N
> > > \N \N \N \N \N \N \N
> > > SRID=900913;POINT(1767106.17982627 6203496.15919218)
> > >
> > > Error occurred, cleaning up
> > > #>
> > > - ---------------------------------------------------
> > >
> > > Running with slim mode on yields the same errors earlier in the
> > > document (around 32000k if memory serves)
> >
> > Odd, I don't think I have ever seen that error before. The presence
> > of
> > that "ü" in the output makes me wonder if this is an encoding issue.
> > Your database using should be UTF-8:
> >
> > $ psql -l
> > List of databases
> > Name | Owner | Encoding | Collation | Ctype |
> > Access
> > privileges
> > -----------+----------+----------+-------------+-------------+-----------------------
> > gis | jburgess | UTF8 | en_GB.UTF-8 | en_GB.UTF-8 |
> >
> > If it is not UTF-8 then you will need to create it again, e.g.
> >
> > $ createdb -E UTF8 -O <username> gis
> >
> > If that does not work then perhaps you should look in the postgresql
> > server log to see if that includes a more verbose error message.
> >
> > Jon
> >
> >
> > > My installation is CentOS 5.4 with the latest SVN (20912)
> > > osm2pgsql. PostgreSQL 5.4.3 and postgis 1.3.6-1 were installed via
> > > Yum from the PGDG84 repository.
> > >
> > > Thanks in advance for any help.
> > > — Seth
> >
> >
> >
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
More information about the dev
mailing list