[OSM-talk] osmarender tile server broken?

[matthew-osm at newtoncomputing.co.uk]

Hi!

On Sat, Feb 24, 2007 at 04:13:14PM +0000, OJW wrote:
> On Saturday 24 February 2007 16:00, matthew-osm at newtoncomputing.co.uk wrote:
> > Hi - have just come up with a new plan on IRC (well, last night actually),
> > which is to store files in a direct access archive and pull them out with a
> > C program to serve them.
> 
> I thought the plan was to move _away_ from a database?

OK - move away from a general purpose relational database to something that
specific to the problem.

> Putting a load of tiles in one file and having an application to access them 
> (with file-locking to make sure you can't read one tile while you write 
> another one) doesn't sound so different to MySQL, except for the non-standard 
> interface and choice of partitioning strategy.

File locking is not an issue - client renders new set of tiles from z12-z17,
uploads one archive with these tiles in. Server puts it at
/tiles/z12X/z12Y.archive.new.$$ and moves it to z12Y.archive

 * No SQL for the server to have to process.
 * No indexes to update.
 * No locks to handle (erm, does MySQL do these anyway, except for locking the
   whole table?)

> I'm worried here about things like 
> * apache not being able to serve tiles directly

This isn't going to happen unless there is a filesystem that can cope with 15M
(and growing) inodes in lots of small files. ReiserFS comes to mind for the
small files, but I dunno about the inodes. That seems to be out of favour these
days, though, anyway. I think a small CGI that can pull a file directly out of
an archive (not a compressed archive) and send it would not be as much overhead
as MySQL having to process a select statement?

> * upload script can't use the "move" command to insert a tile

Upload script can use move - they just move the entire new tileset archive on
top of the old one. That's an incredible speed improvement as far as I can see.

> * viewer script can't use the "file_exists" command to check for a tile

If the archive for z12 exists, the tile at z13-z17 must exist.

> * huge amount of extra complexity

Maybe. Doesn't seem that complex to me, and looks a great deal faster. But maybe
some MySQL expert will prove to me that removing it from the equation will slow
things down.

Cheers,

-- 
Matthew