[Talk-us] TopOSM Colorado

[William Stearns]

Good day, all,

On Fri, 4 Sep 2009, Ian Dees wrote:

> On Fri, Sep 4, 2009 at 12:44 PM, Lars Ahlzen <lars at ahlzen.com> wrote:
>       There's also the size of the data sets. The entire hi-res NHD and NED
>       would probably be many, many TB. Unpacking, preparing and indexing that
>       data requires plenty of additional disk space as well. One would
>       probably have to re-work the scripts to work on smaller areas at a time,
>       as well as invest in some sizable drives. The tiles themselves for CO
>       are about 12 GB and 20 GB for MA, so I'd need a web host that's willing
>       to host a TB or so of images. :)
> 
> 
> The hi-res NHD data set fits on 4.5 DVDs. They sent me a set of NHD data last year around
> this time for free (they're very nice folks!). I was considering asking nicely for another
> set, but I was going to try and build a faster computer to handle it, first.
> 
> I don't know about the NED data, but I imagine it would be a similar size for the US
> continental area.
>  
>       Finally, Colorado alone took the better part of a week on my quad-core
>       rendering server. And this excludes all importing and preprocessing of
>       the data. Extrapolate for 48 more states..."
> 
> 
> This sounds like an interesting experiment worth running on Amazon's EC2 cloud. Their
> x-large machines run at $1/hour, but sure would chomp on this data quickly!

 	I've had good luck with www.rackspacecloud.com.  I'm using a 4 
core/1GB ram/40GB disk virtual machine running tilesathome and it's easy 
and relatively inexpensive (6 cents/hour).  Their high end vm (16GB 
ram/620GB disk) is also about a dollar/hour:

http://www.rackspacecloud.com/images/cloudservers/serverSizes.png

> I would be willing to try (and pay for) it if Lars went through a did an updated setup doc.

 	I'd be willing to as well.

> Because the layers are rendered separately, the less-frequently-changing tiles (like NED
> and for a while the NHD) could be done once for the entire country to save on rendering
> costs.

 	Cheers,
 	- Bill

---------------------------------------------------------------------------
 	"She worked with a subdued intensity... She once told me that the
only way to know when you have done something truly great is when your
spine tingles."
 	- on Alice Kober, cryptanalist, in The Code Book, Simon Singh.
--------------------------------------------------------------------------
William Stearns (wstearns at pobox.com, tools and papers: www.stearns.org)
Top-notch computer security training at www.sans.org , www.giac.net
--------------------------------------------------------------------------