[OSM-talk] OSM tagging validation lib

Sun Dec 24 20:26:21 UTC 2017

Colin, I think we should look at JOSM's validations as a good example of
what is needed. For some validations, it extended MapCSS with its own
custom quirks, and even added its own unit testing. For other types of
validations (e.g. geometries), it seems to fall back to full code.  Having
a single language to express any types of validation would simplify
maintenance and participation.

I don't think people should translate the JS reference implementation at
all. That recreates the problem we have now. Instead, all tools should use
one common library directly. The only problem is integration - Java,
Python, and C++ can all call JS functions, we just need to ensure it is
*reasonably* fast and easy to deploy.

Lastly, I am sure we will want to introduce "slow validations" - when a
call to an external service, e.g. OSM db or taginfo is required. Those may
be used by some of the tools, e.g. editors right before saving.

Thx!

On Sun, Dec 24, 2017 at 3:03 PM, Colin Smale <colin.smale at xs4all.nl> wrote:

> Hi Yuri,
>
> We have to decide what is most important - fidelity to the infinite number
> of tagging styles out there, or the ability to get a basic set of tagging
> grammar accepted in as many tools as possible.
>
> Any rules grammar will always have limits of course. If the rules are too
> complex to represent in a declarative way, that is in itself an indication
> of the mess we have got ourselves into. If the "unwritten rules" of OSM
> tagging are too complex to write down, then they need sorting out first!
> Having a simple base to work from might be a good first step. Automated
> chaos is still chaos.
>
> I agree that the ultimate rules engine may well end up using e.g. JS as a
> medium to express some of the subtleties of the rules. Once a JS
> implementation has been published, then people can translate the JS
> reference implementation into whatever language they need. But separating
> the basic rules from the execution engine is nothing more than
> architectural best practice, and there is no reason that the basic
> rules should not be portable across runtime environments.
>
> Can you think of any complex patterns which cannot (easily) be expressed
> in a declarative way?
>
> The real challenge here is not for the coders, but a perennial challenge
> for the OSM community. How do we get to such a consensus about tagging
> patterns, that we can actually say "this is correct" and "this is wrong
> enough to warrant correction" without upsetting a large number of people?
> As soon as a discussion is about right vs wrong, it degenerates into
> mudslinging.
>
>
> //colin
>
> On 2017-12-24 20:37, Yuri Astrakhan wrote:
>
> Declarative rules are usually not very good. Every tool must understand
> every type of rule, and must be updated when new rule types are introduced.
> Plus declarative grammar is either too limiting, or eventually starts
> looking like a scripting language itself, and we end up building an
> execution environment in every tool.
>
>
> I think a better path is executing scripts inside other languages, e.g.
>
> * a JavaScript library ran by Java, Python, C++, ...
> * a lib that gets compiled into a webassembly for browser, or connected to
> other languages via native bindings (less tried path)
>
> The lib would need an API to
> * access local data state
> * access master OSM DB for additional data
> * access other tools like taginfo
>
>
> Integrating scripting environment may be difficult, but offers far greater
> benefits of rule consistency and flexibility.
>
>
> On Sun, Dec 24, 2017 at 7:30 AM, Colin Smale <colin.smale at xs4all.nl>
> wrote:
>
>> The technical differences between java and JS do not preclude generic
>> thinking. Consider tzdata[1] for example, which does something analogous
>> for time zone data.
>>
>> The "rules database" can be made portable, in the form of XML or JSON for
>> example. The logic for using these rules can be described in a portable
>> way. Then you add a set of compliance tests, and publish a reference
>> implementation to demonstrate that is is possible to implement it. After
>> that, the logic can be implemented in any language you like, checked
>> against the compliance tests and the bindings published.
>>
>> Externalising the rules database enables updates and customisations for
>> particular reasons. Depending on the specific use case and the associated
>> non-functionals, validation could possibly be offered as a cloud service
>> (not necessarily by OSM).
>>
>>
>> //colin
>>
>> [1] https://en.wikipedia.org/wiki/Tz_database
>>
>> On 2017-12-24 12:18, James wrote:
>>
>> ID is javascript, JOSM is java. So right there I already see a
>> intercompatibility issue
>>
>> On Dec 24, 2017 6:12 AM, "François Lacombe" <fl.infosreseaux at gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> Here is an idea I got regarding tagging validation in editors (iD, JOSM,
>>> others).
>>> Subsequently to wiki proposal voting and cleanups, it's currently
>>> necessarily to open issues in each editor repository to ask for new tagging
>>> validation rules.
>>>
>>> It can sometimes be time consuming to develop those new rules and such a
>>> work is done independently by each project maintainer. While each project
>>> have its own specific components, background logic is the same.
>>>
>>> Would a new lib called like osmtagvalidator or so in charge of doing
>>> conform validation to wiki be useful?
>>> It may be shared by any project involved in osm editing and preserve its
>>> resources for other valuable developments.
>>>
>>> For me, validation doesn't prevent users to use tags they want, but only
>>> warn them about possible mistakes.
>>>
>>> How would devs and users feel about this?
>>>
>>> All the best
>>>
>>> François
>>>
>>> _______________________________________________
>>> talk mailing list
>>> talk at openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/talk
>>
>>
>> _______________________________________________
>> talk mailing list
>> talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>>
>> _______________________________________________
>> talk mailing list
>> talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk/attachments/20171224/caf918c3/attachment-0001.html>