[osmosis-dev] Osmosis error

[Brett Henderson]

Hi Laura,

On 19 October 2012 23:56, Laura Passigni <laurapassigni at yahoo.it> wrote:

> Hi Brett,
> thank you for your answer.
> Probably the problem is related to the huge size of the database.
>
> In fact the total database is about 850GB. Currently there is about 1
> billion of records in current_nodes and about the same size in
> current_way_nodes tables.
>

I've done some testing with the FileBasedSort class with large numbers of
sort items.  It turns out there are some issues with it that *might* have
caused your issue.  FileBasedSort should only use up to about 512 file
handles in a worst case, but in fact was using considerably more than that
on large sorts.  I would be surprised if it used anywhere near 32758
though.  On the other hand there might be several sorts being processed at
the same time which might make the problem worse.

Anyway, I've addressed the issue and committed a fix.  It is included in
the latest development build.  Try it out and let me know how it goes.
http://wiki.openstreetmap.org/wiki/Osmosis#Development_Build

If that fails we may need to monitor the files being opened by Osmosis.
You can list files held open by a process using the "lsof -a -p
<process_id>" command.

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20121020/c0b577fd/attachment.html>