[OSM-dev] using propper tools; building Packages; was:planetosm-to-db.pl

[Joerg Ostertag (OSM Munich/Germany)]

On Tuesday 07 November 2006 18:33, Richard Fairhurst wrote:
> Just trying to install a local copy of the OSM db on a new machine.
>
> Unfortunately planetosm-to-db.pl now requires
> "two bazillion king Perl modules" 

Well the modules are neither King Size nor written by a bazillion King ;-)
The reason I seperated them from the normal code into smaller modules was to 
try to make the code more readable and reusable. And you can see that in the 
meantime most of the perl Programs in osm can be improved by using them. It's 
all these little things you need over and over again like: 'selecting an 
area', 'finding the newest planet.osm', 'UTF8 sanitizing the 
planet.osm', 'mirroring the newest planet.osm' ... 
Our problem at OSM currently is, that we only have a hand full of active 
developers. So we have to focus on NOT writing the same stuff over and over 
again. Since only then we have a chance of getting a reasonable program 
collection others can work with in a reasonable timespan. 

> (ackn. Chris Lightfoot) rather than being a 
> straight-out-the-box script. 

So the interesting question would be which platform did you try to run it and 
what can we do to make the installation of a complete OSM package much 
easier. I'm thinking of building a complete OSM package which would hold ALL 
necessary programs and base Data for working with OSM on a userclient. So one 
important thing would be to find people who know how to build packages for 
the different platforms.

> To get these you have to (I think) either 
> install Subversion and get them that way, or download them
> file-by-file from the SVN web archive.

I don't see any problem with installing svn. Even more; if you try to do 
serious development you definitely should use a Program for version-control. 
For example if you want to make some local changes to any of the programs; 
you don't want to have to change this over and over again every time a small 
bug in the programs in the repository is fixed and you did a re-download. 
These are things which should even convince a 0815-user who is tweaking 
around a little bit inside the config files or programs.
But If you really want to refuse to install svn you still can do a 
  wget -r --mirror 'http:://svnopenstreetmap.org/'
Sorry to sound harsh; but I really don't understand this kind of whining 
around about using proper tools. And SVN is a proper tool for handling 
Versioned Software.

> I can understand the need for Bit::Vector 

Well this is the part I understand the Idea behind, but from my test I do 
remember that this part did improve the memory footprint too much. But this 
is not the primary question here. And i think the Vector Stuff is working 
well the way it currently is.

> and the DBI stuff,

Well, if you really want a lightweight program you should write a sed 
script ;-)
Something like 
  sed 's/<node /INSERT INTO 'nodes'/; s/\/>//' | mysql 
If you are writing this kind of hack i would really call it lightweight ;-)

> but the rest seems to be making it harder to use the OSM data.

Well it doesn't have to be too hard. What about really building packages for 
the standard user to install the complete OSM suite?

> Could I suggest either that:
>
> a) we resurrect an older version, perhaps as planet-osm-to-db-simple.pl, or

Feel free to do so, but I think it is only of use for a very limited group of 
people. As already stated; I think a way forward might be a little bit more 
work, but it it of much much more use for the whole osm-project.

> b) the Geo::OSM stuff  are submitted to CPAN, for ease of installation?

I'd love to. There was already a start where people were thinking of 
building/packaging a complete Geo::* Module. I would be the first one to be 
happy to commit stuff to this Module as soon as someone tells me where to put 
it and how the existing interfaces for these modules are. I would even try to 
rewrite/rearrange/rename/... my modules to fit into a given Geo::scheme. 
These Modules are only a start to show that we don't have to write the same 
stuff over and over again.


> (and the weirdly named Utils::Debug)

Well If you have a look at the Code and tell me a better or even more 
appropriate name, I'm happy to rename it. But this was the name which just 
came into my mind when I needed a name for those 4 routines.

> I don't mean to knock the effort 

I hope so ;-) I even would like to see you help me improve the 
structure/use-ability/install-ability of these modules.

> (mainly Joerg's, I think) 

At least some of them where added to svn by me. 

> but planetosm-to-db.pl used to be the easiest way to get and use OSM data,
> and I'm sad to see that lost.

What about if we try together to get it back to the easiest solution to create 
and populate a mysql-database with the current data?
We can do a start if you tell me where exactly you had Problems using it, 
which platform you tried it on, ...

If you want to discuss any improvements to the module-set/install-packages via 
voice you're also welcome to contact me via TS/Phone or even Skype(This is 
new ;-)

-- 
Jörg (Germany, Munich)

http://www.ostertag.name/
TeamSpeak2: ts2.ostertag.name, user: tweety, Channel: "GPS Drive"
irc://irc.oftc.net/#osm
Tel.: +49 89 420950304
Skype: joergostertag