[OSM-dev] The future of Potlatch

Tom Hughes tom at compton.nu
Fri May 2 08:35:06 BST 2008

In message <20080502002003.GA7709 at metacarta.com>
        Christopher Schmidt <crschmidt at metacarta.com> wrote:

2> On Fri, May 02, 2008 at 12:24:35AM +0100, Tom Hughes wrote:
>> In message <16e8cf860805011406t29e3d382ke4b18ecf9fe92a4a at mail.gmail.com>
>>           "Tom Carden" <tom at tom-carden.co.uk> wrote:
>> > I think the fact that it has its own API is a much bigger concern than
>> > it being written in AS 1.0 is.  If Potlatch was using the main API,
>> > development of API-backed features in Potlatch could be shared by
>> > other editors too.  Any tests written for the API would help Potlatch.
>> >  Any changes to the schema would only have to happen once. etc. etc.
>> I think most of us (with the possible exception of Richard) would
>> agree here.
>> Well actually I don't mind the existence of the AMF API as such so
>> long as it is just concerned with decoding the RPC calls and encoding
>> the results, and it uses the rails object model to do all the work
>> so that important code is shared with the XML API.
> There's an important difference between this and "Potlatch using the
> main API".

Well yes, the two are not very well aligned at the moment and
that also needs fixing.

> There are, I believe, some things that Potlatch can do that no
> other client can do, because the API to do it is not available.
> Specifically:
>  * whichways_deleted gets deleted ways in a bbox, which the main API
>    doesn't provide
>  * getway_old has a "last deleted version", which seems different
> More problematic than the specific methods which do or don't exist,
> however, is the fact that the Potlatch way of interacting with them is
> *entirely* different -- so when a bug is fixed in the main API, the fix
> has to be duplicted in the Potlatch/amf_controller.

Which is why we're trying to make as much code as possible common
to both - there's a long way to go, but it's a good target.

The point I would like to reach is one where the only difference is
how arguments are decoded and how results are encoded and everything
in between is the same with the same set of API calls exposed via
both the AMF and XML interfaces.

> This affects development of the server side APIs. You, at one point, put
> forward a significant amount of effort in improving the Rails code in
> amf_controller: that work benefited only Potlatch. If Potlatch was using
> the same backend rails calls (rather than just objects/models), then
> that development time could have benefited other clients as well.

Steve did most of the work on railsifying the AMF code. All I did
was one little performance tweak I think.

> Potlatch has a very different way of interacting with the API. This way
> of interacting with the API has some benefits, and could allow for
> other clients similar to Potlatch to be developed without being tied
> explicitly to amf_controller, which is (at least at the moment)
> essentially Potlatch specific. (Things like 'masterscale' and
> 'potlatch_lon' seem to be indicate this anyway: maybe I'm wrong here.)

Yes, those are ugly glitches, and things that I would prefer to see
done on the client side. I suspect that they are mostly on the server
side for historical reasons - bear in mind that Potlatch was being
written before we moved to rails so the server side component was
originally written for the old pure ruby server environment.

>> We have in fact started moving towards that goal with some work
>> that Steve did on some of the AMF calls.
> This is a different goal, that of replacing SQL with rails objects. A
> valuable goal, but not the same as abstracting the data selection calls
> to a single set of code, and then using that code to serve both the main
> API and amf_controller.

You make it sound like the data selection code is horribly complicated
but really all it amounts to is a few find calls on rails classes.

That said, it would be better to common up the data selection code
as well - what we really need is a volunteer to work on this stuff.

>> The problem is that it turned out that, even after I had optimised
>> the code, it is significantly slower to go through the rails object
>> model that to make direct SQL queries.
> This is interesting, but (imho) less relevant: The value of the above is
> actually doubled by this statement, since if Potlatch and the API were
> using the same function, one could optimize this, and since the code is
> used by *all* clients (not just one) you would see any problems with SQL
> calls more generally, and perhaps find it easier to debug them. (I think
> this is compounded by the fact that AMF is a format which is hard to
> replicate outside of a flash client, which makes server side developers
> less likely to have the technical chops to debug them.) A common library
> for API functionality between amf_controller and the various
> api_controller methods would help resolve this to some extent.

Well the fundamental reason for the performance change is rails
and I don't think any amount of optimisation is going to help with
that - it's basically because going through rails means doing work
that could otherwise be avoided.

>> I think even Richard wouldn't mind too much making the AMF API use
>> the rails object model if it wasn't for the performance issues.
> Using the Rails Object Model is solving an *internal* API issue, but
> there still continue to exist two external APIs that use different
> backend data manipulation to achieve their goals. This seperation is
> what I see as more problematic -- and I think that's what Tom was
> addressing as well, though I could be wrong. 

The point is that once both externals APIs are written in terms of
the same internal API it will be much easier to merge them - if both
APIs use Node.find_by_area() to find the nodes in a bbox it is much
easier to see what is common than if one does that one does a giant
SQL statement.

The approach we have been following also allows the changes to be
made incrementally.

To summarise I think we both want the same thing, but you perhaps
think somebody should just sit down and bang an AMF version of the
current XML API and I'm happy with trying to incrementally move
towards that position?


Tom Hughes (tom at compton.nu)

More information about the dev mailing list