[OSM-dev] Osmosis woes, something not right - can someone pls. see why?

[Brett Henderson]

I don't have access to a full planet at the moment.  It's friday evening and
about to head out for drinks so may not get to it tonight ... but I'll try
it out in the morning and see if I can get it working.

2008/7/25 Fire Girl <firegirl at amorous.com>:

>  Correct-o, yes, I have a smaller than expected New York sub-slice, from
> segment_earth.osm.  The segment_earth.osm I initially created, is 2.4 GB....
> and from that, I tried extracting smaller 2 degree extraction, new_york.osm
> which ended up about 27MB!
>
> Sorry for my confusion, what I meant to say was I tried this extraction
> from a OSM and a .BZ2 version of the segment_earth file.... both attempts
> ended up with the same result.
>
> Now, I am running the same extraction from the planet.osm file, and the
> process is still creating the new_york.osm slice.... and it is almost 1GB so
> far.... so it looks like things are going well.  I am so confused, because
> my syntax appears correct, and I am getting -some- kind of data when I
> created the initial segment_earth.osm, which I illustrated how I am getting
> that stage.  I do think 2.4 GB is pretty small for that scope of
> coordinates,
>
> 2.4GB, segment_earth was left="-108" right="-36" top="90" bottom="0" ....
> don't you think?  I would suspect that should be way over 10GB at least.
>
> I hope my Windows based Osmosis tool is doing the right thing..... I am
> getting good data when I run against planet.osm.... it's just my
> huge-sub-sliced extraction is not fairing so well when I try pulling smaller
> size segements from it. :( ??
>
> i appreciate your time with this so far.... the only thing I can think of,
> since my syntax seems to work against the planet.osm file, is maybe such
> extraction is not feasable.... i hope someone can confirm my analysis if
> psble.
>
>
>
>
> ----- Original Message -----
> From: "Brett Henderson"
> To: "Fire Girl"
> Subject: Re: [OSM-dev] Osmosis woes, something not right - can someone pls.
> see why?
> Date: Fri, 25 Jul 2008 16:37:53 +1000
>
> I just noticed you said the 27MB file was truncated.  Do you mean it didn't
> get closed with an </osm> tag at the end?  Or do you mean it was smaller
> than expected?
>
> One other thing that might be worth mentioning.  The osmosis bzip2 support
> is very slow.  It might be worth recompressing files as gzip if you wish to
> improve performance.  It will take up some more disk space but will run
> *much* faster.
>
> On Fri, Jul 25, 2008 at 4:30 PM, Brett Henderson <brett at bretth.com> wrote:
>
>> Hmm dunno.  It certainly looks like you're doing the right thing ...
>> there's no reason why this shouldn't work.  Osmosis doesn't care much about
>> the size of the file.
>>
>> Are you sure you're running osmosis against the correct segment_earth.osm
>> file?  I notice you had both a compressed (bz2 file) and uncompressed
>> version of the command line at the end of your email.  Could you have run
>> the new york extraction against an old version?
>>
>> 2008/7/25 Fire Girl <firegirl at amorous.com>:
>>
>>>  Hi, I wanted to thank others who have help me understand OSM better!
>>> Marco, Andy, and others recently.
>>>
>>> Here is where I am having a gaffe.... Ok, when I run this command on
>>> Osmosis, against the Planet OSM file,
>>>
>>> java.exe -jar osmosis.jar" --rx "planet-latest.osm.bz2"
>>> enableDateParsing="YES" --lp interval="600" --bb left="-75" right="-73"
>>> top="42" bottom="40" --wx "new_york_area.osm"
>>>
>>> It generates a huge OSM for New York Area, which is complete.
>>>
>>> However... If I run this command against the Planet OSM file,
>>>
>>> java.exe -jar osmosis.jar" --rx "planet-latest.osm.bz2"
>>> enableDateParsing="YES" --lp interval="600" --bb left="-108" right="-36"
>>> top="90" bottom="0" --wx "segment_earth.osm"
>>>
>>> It creates a 2.4 GB file.  But, I don't think it is complete.  Because,
>>> if I thereafter, try to extract the New York Slice from this file, I get a
>>> truncated 27MB file with hardly anything in it.  The Lat/Lon coodinates
>>> match the above extraction right?  Am I doing something wrong.... or can I
>>> simply not extract such a huge sub-slice before slicing it down further?
>>>
>>> running,
>>>
>>> java.exe -jar osmosis.jar" --rx "segment_earth.osm"
>>> enableDateParsing="YES" --lp interval="600" --bb left="-75" right="-73"
>>> top="42" bottom="40" --wx "new_york_area.osm"
>>> java.exe -jar osmosis.jar" --rx "segment_earth.osm.bz2"
>>> enableDateParsing="YES" --lp interval="600" --bb left="-75" right="-73"
>>> top="42" bottom="40" --wx "new_york_area.osm"
>>>
>>> generates a 27MB file from this.
>>>
>>> What is up? :)
>>>
>>> -- Be Yourself @ mail.com!
>>> Choose From 200+ Email Addresses
>>> Get a *Free* Account at www.mail.com <http://www.mail.com/Product.aspx>!
>>>
>>> _______________________________________________
>>> dev mailing list
>>> dev at openstreetmap.org
>>> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
>>>
>>>
>>
>
> -- Be Yourself @ mail.com!
> Choose From 200+ Email Addresses
> Get a *Free* Account at www.mail.com <http://www.mail.com/Product.aspx>!
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20080725/9117c392/attachment.html>