[Mailman] Issue with deploying OSM (world data) on AWS server

[Pradeep Nair]

Hi,

We are in the process of deploying Open street map (world content) on our
AWS server for integrating this with our mobile application. However, we
were faced with a lot of issues during the deployment, where this process
was just not completing. Once the process started, it took up a lot of
memory and then the process went into a hung state.

We started the process with 32 GB RAM and then as memory utilization grew,
we increased RAM to 64 GB, but even that was being consumed very soon and
once again it would go into a hung state. The process wouldn't complete.
All we could see at that point was the server memory utilization, but we
couldn't make out what exactly was going on with the OSM deployment process.

We went through multiple unsuccessful rounds of the deployment process and
then we tried with MMAP configuration for data reader.dataacess and were
finally able to complete one cycle of deployment this morning after the
process ran for around 20 hours.

This has raised a few questions for us about the OSM (world data)
deployment -

1. Why does the deployment process take such a long time to complete? 20
hours is a very long time, although we are deploying world content, which
is relatively larger.

2. Memory utilization is very high during the process - The recommended
server config given to us was 32 GB RAM and 1 TB HDD. We had to upgrade RAM
to 64 GB and that too didn't seem to suffice until the last time we tried
when it actually utilized around 62 GB at peak time. We felt that garbage
collection process was probably not functioning properly.

3. What is the recommended CPU configuration?

5. We found some information on the web that once the server is stopped and
then re-started, OSM has to be re-deployed on the server. Why is that the
case? If this deployment process actually takes 20 hours to complete on
average, this will be a huge problem for getting this much downtime on the
production server.

6. Once OSM planet file is deployed, what is the facility provided for
updating a newer version? How often are new updates released? Is there a
provision for only updating the incremental changes in the new file?

It would be of great help if you could please provide your answers to the
above and any other useful information at the earliest?

Thanks

Pradeep Nair
Neova Solutions
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/mailman/attachments/20170612/072f28d3/attachment.html>