[OSM-dev] Osmosis woes, something not right - can someone pls. see why?
Fire Girl
firegirl at amorous.com
Fri Jul 25 09:51:34 BST 2008
Hey, that is great -- and Cheers!
When you get back, it would be interesting to find out, what kinda of
results you get from a two stage extraction, using the same commands,
java.exe -jar osmosis.jar" --rx "planet-latest.osm.bz2"
enableDateParsing="YES" --lp interval="600" --bb left="-108" right="-36"
top="90" bottom="0" --wx "segment_earth.osm"
and then,
java.exe -jar osmosis.jar" --rx "segment_earth.osm"
enableDateParsing="YES" --lp interval="600" --bb left="-75" right="-73"
top="42" bottom="40" --wx "new_york_area.osm"
^^^ this is what is yielding a 27 MB file from the 2.5 GB
segment_earth.osm slice, when according to my test doing this above same
command & parameters against a planet.osm yielded almost a 1.3 GB file :|
I mean, it is pretty clear I got a very slimmed down segment_earth.osm
extraction.... left="-108" right="-36" top="90" bottom="0", is a pretty
big cross-section which yielded 2.5 GB. Which is totally confusing since
I got 1.3 GB from left="-75" right="-73" top="42" bottom="40", ... a tiny
2 degree cross segment! This is driving me nuts! haha
BTW, can you perform this test on a Windows Machine like I am? :))
----- Original Message -----
From: "Brett Henderson"
To: "Fire Girl"
Subject: Re: [OSM-dev] Osmosis woes, something not right - can
someone pls. see why?
Date: Fri, 25 Jul 2008 17:12:50 +1000
I don't have access to a full planet at the moment. It's friday
evening and about to head out for drinks so may not get to it tonight
... but I'll try it out in the morning and see if I can get it
working.
2008/7/25 Fire Girl <firegirl at amorous.com>:
Correct-o, yes, I have a smaller than expected New York
sub-slice, from segment_earth.osm. The segment_earth.osm I
initially created, is 2.4 GB.... and from that, I tried
extracting smaller 2 degree extraction, new_york.osm which ended
up about 27MB!
Sorry for my confusion, what I meant to say was I tried this
extraction from a OSM and a .BZ2 version of the segment_earth
file.... both attempts ended up with the same result.
Now, I am running the same extraction from the planet.osm file,
and the process is still creating the new_york.osm slice.... and
it is almost 1GB so far.... so it looks like things are going
well. I am so confused, because my syntax appears correct, and I
am getting -some- kind of data when I created the initial
segment_earth.osm, which I illustrated how I am getting that
stage. I do think 2.4 GB is pretty small for that scope of
coordinates,
2.4GB, segment_earth was left="-108" right="-36" top="90"
bottom="0" .... don't you think? I would suspect that should be
way over 10GB at least.
I hope my Windows based Osmosis tool is doing the right
thing..... I am getting good data when I run against
planet.osm.... it's just my huge-sub-sliced extraction is not
fairing so well when I try pulling smaller size segements from
it. :( ??
i appreciate your time with this so far.... the only thing I can
think of, since my syntax seems to work against the planet.osm
file, is maybe such extraction is not feasable.... i hope someone
can confirm my analysis if psble.
----- Original Message -----
From: "Brett Henderson"
To: "Fire Girl"
Subject: Re: [OSM-dev] Osmosis woes, something not right -
can someone pls. see why?
Date: Fri, 25 Jul 2008 16:37:53 +1000
I just noticed you said the 27MB file was truncated. Do you
mean it didn't get closed with an </osm> tag at the end? Or
do you mean it was smaller than expected?
One other thing that might be worth mentioning. The osmosis
bzip2 support is very slow. It might be worth recompressing
files as gzip if you wish to improve performance. It will
take up some more disk space but will run *much* faster.
On Fri, Jul 25, 2008 at 4:30 PM, Brett Henderson <brett at bretth.com>
wrote:
Hmm dunno. It certainly looks like you're doing the
right thing ... there's no reason why this shouldn't
work. Osmosis doesn't care much about the size of the
file.
Are you sure you're running osmosis against the correct
segment_earth.osm file? I notice you had both a
compressed (bz2 file) and uncompressed version of the
command line at the end of your email. Could you have
run the new york extraction against an old version?
2008/7/25 Fire Girl <firegirl at amorous.com>:
Hi, I wanted to thank others who have help me
understand OSM better! Marco, Andy, and others
recently.
Here is where I am having a gaffe.... Ok, when I run
this command on Osmosis, against the Planet OSM file,
java.exe -jar osmosis.jar" --rx
"planet-latest.osm.bz2" enableDateParsing="YES" --lp
interval="600" --bb left="-75" right="-73" top="42"
bottom="40" --wx "new_york_area.osm"
It generates a huge OSM for New York Area, which is
complete.
However... If I run this command against the Planet
OSM file,
java.exe -jar osmosis.jar" --rx
"planet-latest.osm.bz2" enableDateParsing="YES" --lp
interval="600" --bb left="-108" right="-36" top="90"
bottom="0" --wx "segment_earth.osm"
It creates a 2.4 GB file. But, I don't think it is
complete. Because, if I thereafter, try to extract
the New York Slice from this file, I get a truncated
27MB file with hardly anything in it. The Lat/Lon
coodinates match the above extraction right? Am I
doing something wrong.... or can I simply not extract
such a huge sub-slice before slicing it down
further?
running,
java.exe -jar osmosis.jar" --rx "segment_earth.osm"
enableDateParsing="YES" --lp interval="600" --bb
left="-75" right="-73" top="42" bottom="40" --wx
"new_york_area.osm"
java.exe -jar osmosis.jar" --rx
"segment_earth.osm.bz2" enableDateParsing="YES" --lp
interval="600" --bb left="-75" right="-73" top="42"
bottom="40" --wx "new_york_area.osm"
generates a 27MB file from this.
What is up? :)
-- Be Yourself @ mail.com!
Choose From 200+ Email Addresses
Get a Free Account at www.mail.com!
_______________________________________________
dev mailing list
dev at openstreetmap.org
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
-- Be Yourself @ mail.com!
Choose From 200+ Email Addresses
Get a Free Account at www.mail.com!
_______________________________________________
dev mailing list
dev at openstreetmap.org
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
--
Be Yourself @ mail.com!
Choose From 200+ Email Addresses
Get a Free Account at www.mail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20080725/4c64d361/attachment.html>
More information about the dev
mailing list