[OSM-dev] Freemap renderer: memory problems - one for the SQL gurus?

[Nick Hill]

Hello Nick

If this were implemented under some form of scheduling system, you could 
reasonably increase the upper limit to 64Mb or even 128Mb.

Under a web server, the PHP limit should not exceed approximately:
(available memory for apache)/Max number of apache processes -4.5Mb.

Given you could have 20 concurrent apache processes, if you have 256Mb allocated 
for apache, each process would have an upper limit of 10Mb, -4.5Mb apache 
overhead, giving 8.3Mb safe limit.

The best solution is to have a scheduling system. An http request comes in, a 
lightweight process (associated with the http connection) slots it's request 
into a rendering queue. Another process, when ready,  pops the request off the 
queue then renders it. The lightweight process associated with the http 
connection wakes up with an increasing back-off (0.25s, 0.5s, 1s, 2s) to check 
whether the job has been rendered. Once rendered, serves it.

A number of rendering processes should equal the number of CPU cores. Each could 
be allocated 100Mb. Each rendering process will be guaranteed the resources it 
needs to render each tile. Therefore, each tile will be rendered completely. The 
rendering queue system could be scalable for any number of processes spread 
across any number of machines. Perhaps using a transactional mysql table and a 
NFS file system.





Nick Whitelegg wrote:
...

> Anyone got any ideas? An SQL issue or an inherent issue with PHP?
> Incidentally is 8MB a reasonable upper memory limit for an apache/php process 
> or could it be safely upped somewhat?
> 
> Thanks,
> Nick