[Tile-serving] [openstreetmap/osm2pgsql] Improved debug information about memory use of Lua (PR #1696)

[Jochen Topf]

We could do that. It would be a bit of work because the information is specific to the output, but the printing of the progress report happens somewhere else.

Here is an alternative. Add this Lua function to your script:
```
function create_memory_reporter(filename, frequency)
    local counter = 0
    local file = io.open(filename, 'w')
    file:write('timestamp,counter,mbyte\n')

    return function()
        if counter % frequency == 0 then
            local mem = collectgarbage('count')
            local ts = os.date('%Y-%m-%dT%H:%M:%S,')
            file:write(ts .. counter .. ',' .. math.ceil(mem / 1024) .. '\n')
            file:flush()
        end
        counter = counter + 1
    end
end
```

Then call it like this from your process functions:
```
local mr = create_memory_reporter('/tmp/osm2pgsql-lua-memlog.csv', 10000)

function osm2pgsql.process_node(object)
    mr()
    ...
end
```
You can have one memory reporter for nodes, ways, and relations together or have separate ones.

This is really something where we can use the power of Lua to have something really configurable. So maybe it doesn't make sense to add this to osm2pgsql itself. (We can add the function above to our helper library though.)

-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/pull/1696#issuecomment-1193266652
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/osm2pgsql/pull/1696/c1193266652 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20220724/933ccbed/attachment-0001.htm>