[openstreetmap/openstreetmap-website] Move spammy phrases to DB (PR #6549)

[Paul Norman]

@pnorman commented on this pull request.



> +      link_size = 0
+
+      doc = Nokogiri::HTML(text.to_html)
+
+      if doc.content.empty?
+        link_proportion = 0
+      else
+        doc.xpath("//a").each do |link|
+          link_count += 1
+          link_size += link.content.length
+        end
+
+        link_proportion = link_size.to_f / doc.content.length
+      end
+
+      downcased_content = doc.content.downcase

1. We should apply a Unicode NFKC normalization before lowercase
2. Downcasing should probably case folding, not just case mapping
3. The same transformations need to be applied to the phrases retrieved from the DB

-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/pull/6549#pullrequestreview-3497751549
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/openstreetmap-website/pull/6549/review/3497751549 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20251123/dd3cab26/attachment.htm>