<p>For a long time the OSMF has used a <a href="https://github.com/openstreetmap/gpx-import">high performance GPX importer</a> to process uploaded traces. This runs as a separate daemon on the servers, unrelated to this code.</p>
<p>This has a number of drawbacks:</p>
<ul>
<li>There is a lot of ruby code in this repo for importing traces that is [unused and bitrotting].(<a class="issue-link js-issue-link" data-error-text="Failed to load issue title" data-id="14297362" data-permission-text="Issue title is private" data-url="https://github.com/openstreetmap/openstreetmap-website/issues/281" href="https://github.com/openstreetmap/openstreetmap-website/issues/281">#281</a>)</li>
<li>The emails sent by the GPX importer <a href="https://github.com/openstreetmap/gpx-import/tree/master/templates">are not translated</a>.</li>
<li>There is no queue management for pending traces, so only one import process (on one server) can run at any time.</li>
<li>The daemon needs its own installation and configuration, its own connections to the database, and its own ability to send mail.</li>
</ul>
<p>Having a daemon running background tasks is fairly standard in the modern rails world. So I'd like to propose some changes:</p>
<ul>
<li>We use the <a href="http://guides.rubyonrails.org/active_job_basics.html" rel="nofollow">ActiveJob</a> framework for managing the queue of traces that need importing.</li>
<li>We rework the C code as a ruby gem, so that we can declare a dependency in the Gemfile and it gets installed and compiled for 'free' with bundler.</li>
<li>We refactor so that only the parsing of the trace files is handled by the C code, and the email notifications and any other scaffolding comes from this rails app. This will allow us to support translations.</li>
<li>We ensure that as many worker processes as are necessary can be run in parallel, to improve import speeds.</li>
</ul>
<p>I'm not familiar at all with the C code, nor the operations of it in production, so any advice on this proposal is welcome!</p>

<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/openstreetmap/openstreetmap-website/issues/1852">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/ABWnLcvGhNibf7ZImAUG0GtZWoLYHyVdks5twrJwgaJpZM4T3-LP">mute the thread</a>.<img src="https://github.com/notifications/beacon/ABWnLf2qn9_YLgiHBjjoWzLlNsgAwK0iks5twrJwgaJpZM4T3-LP.gif" height="1" width="1" alt="" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
  <link itemprop="url" href="https://github.com/openstreetmap/openstreetmap-website/issues/1852"></link>
  <meta itemprop="name" content="View Issue"></meta>
</div>
<meta itemprop="description" content="View this Issue on GitHub"></meta>
</div>

<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/openstreetmap/openstreetmap-website","title":"openstreetmap/openstreetmap-website","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/openstreetmap/openstreetmap-website"}},"updates":{"snippets":[{"icon":"DESCRIPTION","message":"Integrate the high-performance GPX importer (#1852)"}],"action":{"name":"View Issue","url":"https://github.com/openstreetmap/openstreetmap-website/issues/1852"}}}</script>
<script type="application/ld+json">{"@type":"MessageCard","@context":"http://schema.org/extensions","hideOriginalBody":"false","originator":"37567f93-e2a7-4e2a-ad37-a9160fc62647","title":"Integrate the high-performance GPX importer (#1852)","sections":[{"text":"","activityTitle":"**Andy Allan**","activityImage":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","activitySubtitle":"@gravitystorm","facts":[{"name":"Repository: ","value":"openstreetmap/openstreetmap-website"},{"name":"Issue #: ","value":1852}]}],"potentialAction":[{"name":"Add a comment","@type":"ActionCard","inputs":[{"isMultiLine":true,"@type":"TextInput","id":"IssueComment","isRequired":false}],"actions":[{"name":"Comment","@type":"HttpPOST","target":"https://api.github.com","body":"{\"commandName\":\"IssueComment\",\"repositoryFullName\":\"openstreetmap/openstreetmap-website\",\"issueId\":1852,\"IssueComment\":\"{{IssueComment.value}}\"}"}]},{"name":"Close issue","@type":"HttpPOST","target":"https://api.github.com","body":"{\"commandName\":\"IssueClose\",\"repositoryFullName\":\"openstreetmap/openstreetmap-website\",\"issueId\":1852}"},{"targets":[{"os":"default","uri":"https://github.com/openstreetmap/openstreetmap-website/issues/1852"}],"@type":"OpenUri","name":"View on GitHub"},{"name":"Unsubscribe","@type":"HttpPOST","target":"https://api.github.com","body":"{\"commandName\":\"MuteNotification\",\"threadId\":333439695}"}],"themeColor":"26292E"}</script>