[OSM-dev] Mass imports (TIGER and AND)

[Tom Hughes]

In message <41B65A2A-ADFA-4044-88A4-9816D20F1D13 at asklater.com> you wrote:

> Not in context with this thread much but can't you do
> 
> 
> begin
> 	Node.transaction do
> 		all_those_nodes do |node|
> 			 # create nodes
> 		end
> 	end
> rescue Error => ex
> 	#something fucked up, rails should abort the transaction
> 	render :text => 'argggggggghhhhhh'
> 	return :status => 500
> end
> 
> render :text  => 'everything woz fine'

Well the transaction's a bit pointless, but that's the sort of
thing - you need to make sure you call the right validation between
creating the node and saving it as we don't currently have everything
in the right place - too much is done manually by the controller rather
than being built into the model.

> > I think the problem with my scheme is going to be keeping the amount
> > of history required to map the negative IDs in the change file to the
> > allocated positive IDs as things are added. That will use up a lot of
> > memory in ruby.
> 
> Only what would be in a large map call anyway, which is what, 50,000
> nodes?

Well the Dutch guys are talking more like 8 million objects...

Tom

-- 
Tom Hughes (tom at compton.nu)
http://www.compton.nu/