[Imports-us] OSM US Import Document Draft

Randal Hale rjhale at northrivergeographic.com
Mon Nov 4 13:36:12 UTC 2013

Serge - the reason's Carol's comments can be viewed as more commentary 
vs suggestions - well - we're trying to be...well - nice.  We haven't 
yet asked what the imports groups rules are - you guys have rules for us 
- what are your internal rules for dealing with an import/importers?

Our whole issue has never been with the process for importing data. 
We're interested in data that is helpful for the map. You guys really 
need to make sure that what happened to us during an import doesn't 
happen to anyone else. Overall everyone is still over thinking this - Do 
you want new (potentially GIS) people importing data and if so do you 
want to make it "easier" or "harder" or (my vote) more straightforward 
and more well defined. We don't argue the rules - sweet <diety in the 
sky> - we live and die by our internal rules. I applaud at making this 
less like the wild wild west in import land (and I completely understand 
why what happened happened - Paul - you did the right thing) - but 
you're going to scare off new people. If that is what you want than just 
say it "We make this cryptic and scary because we don't want you doing 
an import" .

Just for kicks - I read the automated code of conduct  ( 
http://wiki.openstreetmap.org/wiki/Automated_Edits/Code_of_Conduct) - 
ours wasn't exceptionally automated. It was pretty manual. Is there one 
for importing in general?

There are days I get up and go "I wish we had known" and we wouldn't be 
having this discussion....BUT - I still think it's a good thing we are 
having it.


Randal Hale, GISP
North River Geographic Systems, Inc
423.653.3611 rjhale at northrivergeographic.com
<mailto:rjhale at northrivergeographic.com>

On 11/03/2013 02:41 PM, Serge Wroclawski wrote:
> All,
> At the last meeting, we discussed creating a page on the OSM US
> website about imports.
If you don't tie this back into the OSM Wiki it's a wasted effort. I'm 
not going to two places to learn about imports. The import that Carol 
was doing (and I was helping with) was neither Scripted import and 
automated. Hence the only rule we broke (rule and not a suggestion) was 
contacting the imports list. This needs to be addressed. Change the 
banner on pages dealing with scripts/imports. The tools are everywhere - 
I can sneak and do imports whenever I want as long as they aren't large 
imports. No one will notice. BUT YOU WANT THE USER TO KNOW WHY THIS IS 

> The message for this page is that we welcome
> imports, but that before an import is done, the importer must do a lot
> of homework and be prepared for a long process.
Homework being?.....I know what I would do - is that what the import 
committee feels like needs to be done. For the ACC import we checked the 
data, stripped the attributes, tagged accordingly, uploaded, and fixed 
every error directly related to our import (and we still had two goofs 
we had to fix). I think we did well....

You guys need a sandbox to test imports. Testing against the "live" 
database is not a good thing.
> Martijn and I separately decided to take a crack at writing this. Here
> is my draft, and feel free to make comments.
> Are you interested in making an import in OpenStreetMap?
> ================================================
> OpenStreetMap contains mostly surveyed data but also contains some
> data that's been sourced from external sources. Unlike many other GIS
> databases, though, imported data cannot simply be dropped alongside
> existing OSM data. Instead, an import in OSM must be carefully
> cultivated and is generally a long term commitment.
I take issue that OSM data is different than GIS data - except for file 
format and tagging there is no difference. If you do your homework you 
can "drop it alongside" existing data. To me imports have boiled down to 
an "us vs them". All OSM data is is a format/attribution change. In 
order to better cultivate you should have a checklist for the importer. 
The importer needs to knwo what they are about to do. While I said this 
import was hurried - we broke the area down into small defined AOIs to 
make sure we didn't screw up too much.
> If you're interested in starting an import, you should consider
> joining the [OSM US Import
> Committee](https://lists.openstreetmap.org/listinfo/imports-us), a
> working group of the OpenStreetMap US chapter dedicated to import
> issues. This group has a mailing list and an informal meeting once
> every two weeks. For a more general discussion about imports, join the
> imports mailing list.
> Before you consider doing an import, here are some important questions
> to ask yourself:
> * __How experienced are you with OpenStreetMap?__ We generally don't
> recommend people import into OSM unless they have at least a year's
> worth of experience editing OSM.
1 year is subjective. Are you going to revert is a person has 9 months? 
What evidence is there for 1 year? How do you check to make sure the 
person isn't brain dead? I'm not arguing so much as asking. Why a year? 
I know people with 5 years I wouldn't want doing an import - I know 
people with less than a year I wouldn't think twice about. Do you ever 
find an importer that couldn't chew gum and walk at the same time - did 
you stop the import?
> * __Is this a dataset that belongs in the OSM database?__ There are
> many datasets which are interesting or useful, but do not belong in
> the OSM main database. Instead, you may want to consider a mixin or
> other way of using OSM data alongside your dataset.
List the data. I have sidewalks? Can I import? What data does OSM want 
to see imported? List it instead of leaving it a clouded issue. We have 
a whole pile of data from ACC. Buildings are the only thing we think has 
a place currently. What about the guy wanting to do landcover or 
parcels? we even have addresses - but we haven't verified them so we 
won't import them. They look good but that's about it.

> * __Do you know license for the data you want to import?__ The OSM
> project is very strict with its data and licensing requirements, and
> any potential imported data needs to adhere to those strict standards.
No one is checking the licensing so I can list whatever I want. I 
understand why the licensing is important - but there is no self 
check/audit. It's just semantics. Not complaining - just pointing it 
out. We will most likely need to come up with one for ACC - ODBL? Do you 
have a suggested license? Give a list - let people know some choices.

> * __How will you handle data updates?__ As the data from the external
> source changes, how will you plan on keeping OSM up to date?
If it's your local area then you will want to keep it updated as a 
matter of pride in the work. To me this is just another question sent to 
frustrate the new user. You mention a plan - is there a suggested plan 
for updating the data? I hope this is where "blood and souls" (I 
listened Frederik) comes in. My old neighborhood - 
http://www.openstreetmap.org/#map=16/35.0560/-85.3079 .
The only import was hydro (and I did that a while back).
> * __How will you handle conflation issues?__ Your data cannot simply
> be placed on top of existing data, but instead, will need to be
> carefully conflated. Have you considered how that process would take
> place?
Define conflation. In our world it's a whole different term. When we 
started receiving emails during the import they were filled with OSM 
speak. Simple <fill in the blank> language is a good thing. I know it's 
part of being in the community but after the hazing is finished you can 
say conflation all you want.
> * __How much time and energy are you willing to dedicate to this
> process?__ Most imports require months, if not longer to complete.
> This time includes planning, discussion, revision and the import
> processes itself. If you take on an import project, you will be
> responsible for overseeing its execution from start to finish.
If it's a community project I hope the community chimes in and helps. 
Making it sound like it's you vs the world is once again in place to 
frustrate more than help. If my workload changes it may take me a year 
or longer. I hope the community of importers would be willing to help.
> Lastly, read over the [OpenStreetMap Import
> Guidelines](http://wiki.openstreetmap.org/wiki/Import/Guidelines).
> These govern the import process in OpenStreetMap, and not following
> them can result in your data being reverted, or your account being
> locked.
This needs to be at the top. Let people know there are rules and what's 
going to happen. Hence the uninformative banner you see at every page 
involving an import/script. MAKE IT INFORMATIVE. Let people know crap 
will hit the fan if you don't follow the rules.

> _______________________________________________
> Imports-us mailing list
> Imports-us at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports-us

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports-us/attachments/20131104/861d5b96/attachment-0001.html>

More information about the Imports-us mailing list