[OSM-dev] Update of TIGER ruby import code

Brandon Martin-Anderson badhill at gmail.com
Sat Jul 7 01:05:34 BST 2007


On 7/6/07, Dave Hansen <dave at sr71.net> wrote:
>
> On Fri, 2007-07-06 at 16:44 -0700, Brandon Martin-Anderson wrote:
> > Agree'd: Ruby is best for small prototypes; it does not do heavy
> > lifting well. The best thing you (or I) could do in the short term is
> > run a profiler on a small county and see what's eating up all the
> > resources. It's likely that it can be optimized by a factor of two by
> > optimizing the code within Ruby.
>
> I've actually been using ruby-prof pretty heavily.  I realized that
> ruby's function calls are horrific.  My habits from coding in the Linux
> kernel taught me to write quick, small functions.  They were awful in
> ruby.
>
> > If that doesn't seem possible, it is relatively straightforward to
> > implement a Ruby function in C. A lot of the faster "Ruby" libraries
> > are actually written in C with liberal use of the " ruby.h" header to
> > give access to Ruby types.
>
> Perhaps I'll give this a try.  I'm going to post a new version in a bit.
> We'll see what people think of it.
>
> How hard would it be for you to re-write the tiger parser in C?  I
> could do the OSM bits pretty easily.


Oomph. I'll take a look at the latest code and see what I can do with it.
The efficiency of the implementation of various function calls varies
_wildly_. For example chunking a _huge_ string into an array using a regex
is very, very efficient. I might be able to speed things up a bit. If all
hope is lost, we'll see about C.

-B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070706/5c7973f2/attachment.html>


More information about the dev mailing list