[OSM-dev] XML Schema

[Hugh Barnes]

[replying to both of Frederik's messages]

On Sunday 09 November 2008, 01:02:13, Frederik Ramm did write:
> Hi,
> 
> Florian Lohoff wrote:
> > I dont think anyone expects every reponse of the server to be validated.
> 
> I was under the impression that Hugh did.
> 

Well, it would be ideal on the server, but I hadn't expected it after what you 
said. I appreciate it's not free (in cycles), but I suspect it may save humans 
some time cleaning up.

If there's a schema available via a URL, it can happen in clients and/or the 
server with very minimal effort. Any XML library will support validation 
against the variety of schema languages, and these are certainly available for 
Java, C (probably ++), and Ruby. So when you've imported your library, it's 
usually just a matter of calling

validate($XML,$schema)

and maybe doing things to feed back the output.

It's an important mutual trust mechanism in XML on the wire. What would you 
think of a database server that didn't check transactions against the schema? 
(that would be Excel :~) )

Actually, doesn't validation happen already against the GPX DTD when uploading 
traces?

> > I have no problems if the DTD is a moving target and
> > people get told something changed. But currently there is nothing to
> > check against
> 
> Does
> 
> http://wiki.openstreetmap.org/index.php/OSM_Protocol_Version_0.5/DTD
> 
> not work?
> 

http://dev.openstreetmap.org/~kleptog/osm-0.5.dtd will work to the extent DTDs 
can (see below), as long as it's maintained. Would need to use versioning like 
the API calls do.

> You're right about the change files though, I don't know if there's 
> anything for that. Then again, while it is very easy to validate your 
> application's output against a DTD, it is quite difficult to validate 
> the input side - whether or not your application will really be able to 
> process any document allowed by the DTD!
> 

Yep, DTDs are pretty weak, but you can lock it down better using more 
expressive schema languages. I am pretty sure that would be enough for any 
app.

… and on Saturday 08 November 2008, 23:38:23, Frederik Ramm did write:
> Hugh Barnes wrote:
> >  What kind 
> > of validation exists?
> 
> None that I am aware of. Frankly, I'm not even sad about this, I believe 
>   any kind of server-side validation would be
> 
> * a lot of work to implement

no

> * slowing down API requests

yes, but arguably worth it

> * hard to maintain

No, just maintain the schema. That's a good thing to do in data exchange.

> while any kind of client-side validation would just lead to "hey, the 
> server issues invalid XML!" - "then fix your definition of 'invalid'".
> 

The schema would be the canonical definition. That's why deprecating legacy 
features in it would be necessary along with versioning. I think what you're 
saying is that we might have unusable clients if we're not careful, through no 
fault of the user. Data might not get contributed. Very bad, I agree, but 
isn't that possible already? It's a communication thing.

> But you're welcome to support patches to the rails API and the clients 
> and promise to keep them up to date ;-)
> 

Yep - I'm still getting my head around Rails. All I (or another) would really 
have to keep up to date would be the schema. Everything else is static and 
standard.

Exactly how the schema would be communicated might be interesting. It would, 
of course, have to pass the same tests for documents you currently maintain 
for API testing.

Anyway, this conversation is getting very involved for a pleasant Sunday and, 
in any case, I don't want to keep you from real API 0.6 work.

I am contemplating thinking a bit more, outlining plans on the wiki, and then 
drawing attention to it here.

Cheers