[OSM-dev] ROMA servers down - osmosis large way problem

[Brett Henderson]

Seems like you're all on top of the problem now.

I updated the schema creation script on 3rd November to use an int 
instead of smallint.  Internally osmosis doesn't care, it uses a 32 bit 
integer (more accurately, it uses a list index which is 32 bit).  It is 
just the schema itself which was using a smallint to minimise disk 
usage.  It should be possible to change the column type without losing 
data.  If there's anything I can do let me know.

Note that the 0.6 version of the schema still uses a smallint, I 
possibly should change that in case people start testing it with data 
migrated from a 0.5 planet.

D Tucny wrote:
> 2008/12/29 Jeremy Adams <milenko at king-nerd.com 
> <mailto:milenko at king-nerd.com>>
>
>     Hey all,
>
>     There's an issue with the minute change file
>     200812290912-200812290913.osc.gz.
>     It contains a way which has over 40k nodes.  It's an
>     administrative boundary
>     which appears to enclose all of Quebec.  It's way id is 29309772.
>      When osmosis
>     attempts to process this file it fails.  The relevant parts of the
>     error appear
>     to be "Unable to insert new way node" and a pgsql error "ERROR:
>     smallint out of
>     range."
>
>     I'm sure I can find the way and split it into smaller chunks, but
>     what's the
>     process for getting the ROMA servers running again?  Can we just
>     remove the
>     reference to the way in the changeset and continue updating?  The
>     way should be
>     added back in once it's split and reuploaded, correct?
>
>     -Jeremy
>
>
> The same problem happened on 31st October...
>
> Subject: HEADS UP osmosis pgsql schema users Was: psql osmosis simple 
> shema / smallint out of range
>
> On Fri, Oct 31, 2008 at 11:17:16AM +0100, Florian Lohoff wrote:
>
> > Hi,
> > i just discovered that osmosis was not able to apply the hourly osc file
> > starting 2008-10-29T20:00:00Z - It failed with:
> >
> > 2008-10-31 11:09:52 CET ERROR:  smallint out of range
> > 2008-10-31 11:09:52 CET STATEMENT:  INSERT INTO way_nodes (way_id, 
> node_id, sequence_id) VALUES ($1, $2, $3)
> >
> > Is there a way with >2^16 aka 65536 nodes?
> >
> > Or did someone manage to enter a completely broken sequence number?
> ... [show rest of quote 
> <http://www.nabble.com/psql-osmosis-simple-shema----smallint-out-of-range-td20263104.html#>]
> ... [rest of quote removed]
> Its 2^15 because it signed - and yes - somebody managed to get abovE:
>
> osm=> select max(sequence_id) from way_nodes;
>   max  
>   -------
>    39767
>   (1 row)
>
> osm=> select * from way_nodes where sequence_id = 39767;
>   way_id  |  node_id  | sequence_id
>   ----------+-----------+-------------
>    28098452 | 308532457 |       39767
>
> I converted the smallint to int ...
>
> Flo 
>
> So, perhaps following Flo's lead and changing the smallint to int 
> would be the best approach for dealing with getting the ROMA servers 
> back up and running...
>
> d
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>