[OSM-dev] Problems with planet files and bz2

[Jeffrey Ollie]

On Tue, Mar 1, 2011 at 6:05 PM, Frederik Ramm <frederik at remote.org> wrote:
>
> Eric Wolf wrote:
>>
>> I'm going to have to look into another solution.
>
> Usual advice is to bzcat and then read from stdin; you could proably even do
> that with a popen() from python if need be (but that comes from a
> non-pythoneer).

Here's a snippet from one of my scripts:

source = file(planet_file_name, 'rb')
target = file(target_file_name, 'wb')

p1 = subprocess.Popen(['bzip2', '--decompress'],
                      stdin = source,
                      stdout = subprocess.PIPE)
p2 = subprocess.Popen([os.path.expanduser('~/osmosis-0.38/bin/osmosis'),
                       '-verbose',
                       '--read-xml', 'file=/dev/stdin', 'enableDateParsing=no',
                       '--bounding-box', 'top=50', 'bottom=23',
'right=-66', 'left=-125', 'idTrackerType=BitSet', 'completeWays=yes',
'completeRelations=yes',
                       '--write-pbf', 'file=/dev/stdout'],
                      stdin = p1.stdout,
                      stdout = subprocess.PIPE)
p3 = subprocess.Popen(['bzip2', '--compress'],
                      stdin = p2.stdout,
                      stdout = target)

pids = set([p1.pid, p2.pid, p3.pid])
while pids:
    pid, status = os.wait()
    print pid, status
    pids.remove(pid)
    if not os.WIFEXITED(status) or os.WEXITSTATUS(status) != 0:
        for pid in pids:
            try:
                os.kill(pid, signal.SIGTERM)
            except OSError, e:
                print e



-- 
Jeff Ollie