<p></p>
<h3 dir="auto">Progress on Implementing Search for Diaries</h3>
<p dir="auto">I've explored several approaches to add search functionality to the diaries page. Here’s a summary of what has been done so far, along with some considerations and questions moving forward:</p>
<ol dir="auto">
<li>
<p dir="auto"><strong>PostgreSQL <code class="notranslate">LIKE</code> Text Searching:</strong></p>
<ul dir="auto">
<li><strong>Method:</strong> Implemented a basic text search using PostgreSQL’s <code class="notranslate">LIKE</code> operator.</li>
<li><strong>Results:</strong> This method is fast and responsive, but the relevance of the results may not be ideal since <code class="notranslate">LIKE</code> searches do not inherently provide any ranking mechanism.</li>
<li><strong>Code Example:</strong>
<div class="highlight highlight-source-ruby" dir="auto"><pre class="notranslate"><span class="pl-s1">entries</span> <span class="pl-c1">=</span> <span class="pl-s1">entries</span><span class="pl-kos">.</span><span class="pl-en">where</span><span class="pl-kos">(</span><span class="pl-s">"title LIKE :query OR body LIKE :query"</span><span class="pl-kos">,</span> <span class="pl-pds">query</span>: <span class="pl-en">search_query</span><span class="pl-kos">)</span></pre></div>
</li>
</ul>
</li>
<li>
<p dir="auto"><strong>Using the <code class="notranslate">pg_search</code> Gem:</strong></p>
<ul dir="auto">
<li><strong>Method:</strong> Integrated the <code class="notranslate">pg_search</code> gem to enable full-text search capabilities.</li>
<li><strong>Results:</strong> This allows for more sophisticated search capabilities, including relevance ranking and partial word matching. However, with around 600,000 entries in the database, this approach has shown to be quite slow.</li>
<li><strong>Code Example:</strong>
<div class="highlight highlight-source-ruby" dir="auto"><pre class="notranslate"><span class="pl-en">include</span> <span class="pl-v">PgSearch</span>::<span class="pl-v">Model</span>
<span class="pl-en">pg_search_scope</span> <span class="pl-pds">:search_by_title_and_body</span><span class="pl-kos">,</span>
<span class="pl-pds">against</span>: <span class="pl-kos">[</span><span class="pl-pds">:title</span><span class="pl-kos">,</span> <span class="pl-pds">:body</span><span class="pl-kos">]</span><span class="pl-kos">,</span>
<span class="pl-pds">using</span>: <span class="pl-kos">{</span>
<span class="pl-pds">tsearch</span>: <span class="pl-kos">{</span> <span class="pl-pds">prefix</span>: <span class="pl-c1">true</span> <span class="pl-kos">}</span> <span class="pl-c"># Enables partial word matching</span>
<span class="pl-kos">}</span>
<span class="pl-s1">entries</span> <span class="pl-c1">=</span> <span class="pl-s1">entries</span><span class="pl-kos">.</span><span class="pl-en">search_by_title_and_body</span><span class="pl-kos">(</span><span class="pl-en">params</span><span class="pl-kos">[</span><span class="pl-pds">:search</span><span class="pl-kos">]</span><span class="pl-kos">)</span></pre></div>
</li>
</ul>
</li>
<li>
<p dir="auto"><strong>Exploring Trigram Indexes in PostgreSQL:</strong></p>
<ul dir="auto">
<li><strong>Next Steps:</strong> We’re considering the use of trigram indexes (<code class="notranslate">pg_trgm</code> extension) to accelerate text searches while still offering some level of relevance ranking. This approach could provide a balanced solution between performance and relevance. However, it would require creating additional database migrations to index both the <code class="notranslate">title</code> and <code class="notranslate">body</code> fields, which would add a layer of complexity to the project. The question here is whether we are comfortable with introducing this complexity. Are we okay with the trade-offs involved?</li>
</ul>
</li>
<li>
<p dir="auto"><strong>External Search Engines (DuckDuckGo, etc.):</strong></p>
<ul dir="auto">
<li><strong>Consideration:</strong> Another approach could involve leveraging external search engines like DuckDuckGo by parsing user input and redirecting to search results. This would expose users to privacy-focused search engines and could potentially offload the search functionality. However, this approach has significant downsides, including the delay in indexing by these engines and a lack of control over when and how content is indexed.</li>
</ul>
</li>
</ol>
<h3 dir="auto">Main Questions</h3>
<ul dir="auto">
<li><strong>Time Investment:</strong> How much time are we willing to invest in this feature? The implementation options vary greatly in complexity and development time.</li>
<li><strong>Relevance vs. Performance:</strong> How important is the relevance of search results compared to the speed of search? Should we prioritize one over the other?</li>
</ul>
<p dir="auto">Your feedback on these considerations would be valuable in deciding the best path forward. Thank you.</p>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />Reply to this email directly, <a href="https://github.com/openstreetmap/openstreetmap-website/issues/3289#issuecomment-2312543049">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AAK2OLPEJX5QWMWGRVDYWK3ZTR4F5AVCNFSM6AAAAABNGD7XIWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJSGU2DGMBUHE">unsubscribe</a>.<br />You are receiving this because you are subscribed to this thread.<img src="https://github.com/notifications/beacon/AAK2OLN7QOP7ANFR3IC4ENTZTR4F5A5CNFSM6AAAAABNGD7XIWWGG33NNVSW45C7OR4XAZNMJFZXG5LFINXW23LFNZ2KUY3PNVWWK3TUL5UWJTUJ22NUS.gif" height="1" width="1" alt="" /><span style="color: transparent; font-size: 0; display: none; visibility: hidden; overflow: hidden; opacity: 0; width: 0; height: 0; max-width: 0; max-height: 0; mso-hide: all">Message ID: <span><openstreetmap/openstreetmap-website/issues/3289/2312543049</span><span>@</span><span>github</span><span>.</span><span>com></span></span></p>
<script type="application/ld+json">[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/openstreetmap/openstreetmap-website/issues/3289#issuecomment-2312543049",
"url": "https://github.com/openstreetmap/openstreetmap-website/issues/3289#issuecomment-2312543049",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]</script>