On Fri, Feb 29, 2008 at 9:23 AM, Dave Stubbs <<a href="mailto:osm.list@randomjunk.co.uk">osm.list@randomjunk.co.uk</a>> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div></div><div class="Wj3C7c">On Thu, Feb 28, 2008 at 10:47 PM, Jason Reid<br>
<<a href="mailto:osm@bowvalleytechnologies.com">osm@bowvalleytechnologies.com</a>> wrote:<br>
><br>
> David Earl wrote:<br>
> > How feasible would it be to put a set of attributes either on the top<br>
> > level element or an element created for the purpose which tells me how<br>
> > many nodes, ways and relations there are in the file. If you have the<br>
> > counts to hand at the beginning, great, but if not if you wrote '...<br>
> > nodecount="000000000000" waycount="000000000000"<br>
> > relationcount="000000000000"' at the beginning, and then when you've<br>
> > output the elements and counted them up as you do it, at the end seek<br>
> > back and replace the zeros with the counts.<br>
> ><br>
> > This would enable me and others to do progress reporting on making a<br>
> > pass through the file. (I can't do it by file size and read position<br>
> > because the filesize function won't go bigger than 2Gb in PHP, and I<br>
> > can't count the elements before I start without completely decompressing<br>
> > the file first, which I no longer have enough free disk to do).<br>
> ><br>
> > David<br>
> ><br>
> > _______________________________________________<br>
> > dev mailing list<br>
> > <a href="mailto:dev@openstreetmap.org">dev@openstreetmap.org</a><br>
> > <a href="http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev" target="_blank">http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev</a><br>
> ><br>
> There is the planet statistics script that I wrote a while back (in<br>
> python) that I need to get around to popping into SVN, it doesn't count<br>
> nodes or relations currently, only ways, but it wouldn't be hard to add<br>
> (plus it would give it something to do since 92.5% of the objects in the<br>
> dump are nodes and it currently scans over them silently). It could be<br>
> modified to sit in between the output of the planet script and gzip and<br>
> calculate as the file is being compressed (the script uses a stream<br>
> consuming parser to read stdin, in my uses piping from bzcat currently,<br>
> and could pass the stream back out stdout unmodified)<br>
><br>
<br>
<br>
</div></div>I think if we wanted counting it would be simpler to just add it to<br>
the C code rather than pipe through another application which actually<br>
has the same limitations (no knowledge of counts at the start, and no<br>
seek).<br>
<br>
The other possibility would be to write to a whole sequence of files,<br>
all compressed, and just tar the results with a stats meta file to<br>
make a single file for download... most processors could be modified<br>
to read tarballs quite easily, and if not you could untar them first -<br>
it would basically be an OSM Jar but with choice of compression. Just<br>
a random thought... I'm sure you can think of many holes.<br>
<br>
Don't forget there's also<br>
<a href="http://www.openstreetmap.org/stats/data_stats.html" target="_blank">http://www.openstreetmap.org/stats/data_stats.html</a> -- if you just want<br>
a rough guess at the number of nodes/ways and you are dealing with a<br>
recent planet, then you could just scrape that to get the numbers.<br>
<font color="#888888"></font></blockquote><div><br>There's also <a href="http://osmxapi.hypercube.telascience.org/total.xml">http://osmxapi.hypercube.telascience.org/total.xml</a>. This is xml so it may be easier to handle than data_stats.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font color="#888888"><br>
Dave<br>
</font><div><div></div><div class="Wj3C7c"><br>
_______________________________________________<br>
dev mailing list<br>
<a href="mailto:dev@openstreetmap.org">dev@openstreetmap.org</a><br>
<a href="http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev" target="_blank">http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev</a><br>
</div></div></blockquote></div><br>