[OSM-dev] osm2pgsql, planet-110126.osm, doubts

[IgnacioZ]

Hi!, I have been in a similar situation recently.
I have been able to redirect the output using gdb as it is mentioned here:

http://stackoverflow.com/questions/593724/redirect-stderr-stdout-of-a-process-after-its-been-started-using-command-line

Hope it helps..

cheers!
Ignacio.

On Wed, Feb 9, 2011 at 8:13 AM, Juan Lucas Domínguez Rubio <
juan_lucas_dr at yahoo.com> wrote:

> Dear list,
>
> I'm in a rather weird situation (mostly my fault). These are the facts:
>
> - I have launched the import of the OSM planet like this:
>
> osm2pgsql -d osmdb -l --slim -U postgres -W -H 127.0.0.1 -v
> /mnt/data/planet-110126.osm &
>
> - The size of planet-110126.osm is exactly 216,176,647,801 bytes
>
> - I didn't redirect the output and I closed the session, so I cannot see
> the counters (I mean the ones that say "ways 200K", etc). The process is
> running since last Friday on a Ubuntu 10.10 machine. I suspect RAM size is
> the bottleneck. 'free -m says':
>
>             total       used       free     shared    buffers     cached
> Mem:          5983       5929         54          0         21       4529
> -/+ buffers/cache:       1378       4605
> Swap:         5883          0       5883
>
>
> 'top' says:
>
> =========================================
> top - 11:59:59 up 5 days, 19:32,  2 users,  load average: 1.01, 1.04, 1.06
> Tasks: 128 total,   1 running, 127 sleeping,   0 stopped,   0 zombie
> Cpu(s):  3.7%us,  0.9%sy,  0.0%ni, 77.9%id, 17.0%wa,  0.0%hi,  0.5%si,
>  0.0%st
> Mem:   6127440k total,  6083552k used,    43888k free,    21088k buffers
> Swap:  6025212k total,        0k used,  6025212k free,  4648332k cached
>
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  1844 postgres  20   0  208m 134m 131m S   15  2.2 911:41.53 postgres
>  1838 desa      20   0 1055m 910m 1292 S    6 15.2 641:52.10 osm2pgsql
>  1845 postgres  20   0  218m 142m 132m D    1  2.4  41:05.37 postgres
>   47 root      20   0     0    0    0 S    0  0.0   5:06.97 kswapd0
>  1841 postgres  20   0  253m 161m 132m S    0  2.7  36:31.77 postgres
>  1842 postgres  20   0  287m 188m 132m S    0  3.2   9:52.45 postgres
>  4669 postgres  20   0 19276 1324  968 R    0  0.0   0:00.02 top
>    1 root      20   0 23740  808  212 S    0  0.0   0:01.56 init
> ...
> =========================================
>
> I'm a bit worried about the %MEM = 15.2 (?).
>
> Is there a way to see again the output? I think 'fg' won't work as 'job'
> does not list it because I didn't use CTRL+Z and 'bg'.
>
> If I do cat /proc/1828/10 (1828 being the process ID), I get:
>
> rchar: 216047549356
> wchar: 42604
> syscr: 52746017
> syscw: 175423
> read_bytes: 216274485248
> write_bytes: 0
> cancelled_write_bytes: 0
>
> As you can see the number of bytes read is already above the size of the
> .osm file (maybe osm2pgsql has created and read some temporary files?)
> Anyway I think the app has already read more than 95% of the .osm file. What
> do you think?
>
> What worries me is that if I access the DB from pgAdmin, it says it has no
> tables apart from the default tables of every postgis DB.
>
> What do you think?
> Is it normal that osm2pgsql has read more than 200 GB and no table appears
> in the DB? Are there (invisible?) temporary tables somewhere?
>
> Which % of the job you think has been done so far?
> Maybe I should kill the process and restart the import again?
>
>
> Thanks,
> Juan Lucas
>
>
>
>
> ____________________________________________________________________________________
> Now that's room service!  Choose from over 150,000 hotels
> in 45,000 destinations on Yahoo! Travel to find your fit.
> http://farechase.yahoo.com/promo-generic-14795097
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20110209/307f54c4/attachment.html>