[Photon] Komoot API data - modernization ideas?

[Sarah Hoffmann]

Hi,

On Tue, Aug 22, 2023 at 07:42:29AM +0000, Cristian Schuszter wrote:
> I was fiddling around with the API code for a bit and realized that it's using a super old version of Elasticsearch, 5.6.16.
> 
> 
> We wanted to serve the data from one of our IT clusters, but they only provide newer installations based on OpenSearch. It seems that it's not trivial to migrate the data, since you need to reindex from ES 5 -> 6 then somehow migrate it to OpenSearch.
> 
> 
> Are there any plans to support any newer versions of ES, including having the data dump directly in the required format? What about OpenSearch?

The main reason why we are still stuck with ES 5.6 is that ElasticSearch
dropped the embedded mode in ES 6+. One of the big selling points of
Photon has always been that it is super-easy to set up: download two
files, install Java and you have a geocoder. I know that there is docker
these days, but that's not really the same.

I've looked into alternatives and currently my favourite one is
switching to Solr. Not only because it offers embedded mode but also
because it has a more low-level interface that would make it easier to
adapt to the specifics of a geocoder. But that solution is still a bit
away.

In the meantime, I'm quite open to take a pull request that adds support
for ES 6 next to the standard embedded mode using ES 5.6. This mainly
means switching to the HTTP client. I did some preliminary work on this
in https://github.com/lonvia/photon/tree/switch-to-http-client but then
got stuck with the problem of how to extend the tests to HTT client mode
see https://github.com/komoot/photon/issues/325#issuecomment-812920427
Maybe you want to finish the work?

I can't really promise that we would provide dumps for ES6.

Sarah