<!DOCTYPE html>
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<link rel="stylesheet"
href="moz-extension://cadb51de-a1e3-40e4-ac10-d5240ef25c10/vendor/textcomplete.css">
</head>
<body text="#000000" bgcolor="#ffffff">
<font size="2" face="Barlow">Hi,<br>
i was wondering what the expected throughput for the global
database is when running on a t2.2xlarge (8cpus, 32GB Ram, 500GB
gp3 SSD).<br>
<br>
We're currently getting ~2.38 it/s with 6 workers sending requests
in parallel.<br>
<br>
Logs from a test run on 1k addresses:<br>
</font><font size="2" face="Barlow">----------------------------------------------------------------------------------</font><br>
<font size="2" face="Barlow">2024-12-28 11:49:45 [INFO] __main__ -
Script started<br>
2024-12-28 11:49:45 [INFO] __main__ - Reading input data from
<a class="moz-txt-link-freetext" href="s3://***/source/unique_addresses_for_geocoding_1k_sample.csv">s3://***/source/unique_addresses_for_geocoding_1k_sample.csv</a><br>
2024-12-28 11:49:45 [INFO] botocore.credentials - Found
credentials from IAM Role: photon-geocoder-role<br>
2024-12-28 11:49:45 [INFO] __main__ - Loaded 1000 records to
geocode<br>
2024-12-28 11:49:45 [INFO] __main__ - Starting parallel geocoding
of 1000 addresses with max_workers=6<br>
Geocoding:
1%|█▎
| 7/1000 [00:00<01:32, 10.75it/s]2024-12-28 11:49:45 [ERROR]
__main__ - Network/Request error for address='SPT OILFIELD
EQUIPMENT & VESSELS MANUFACTURERS BUILDING RAS AL KHAIMAH
United Arab Emirates': 400 Client Error: Bad Request for url:
<a class="moz-txt-link-freetext" href="http://localhost:2322/api?q=SPT%20OILFIELD%20EQUIPMENT%20&%20VESSELS%20MANUFACTURERS%20BUILDING%20%20RAS%20AL%20KHAIMAH%20%20United%20Arab%20Emirates&limit=1">http://localhost:2322/api?q=SPT%20OILFIELD%20EQUIPMENT%20&%20VESSELS%20MANUFACTURERS%20BUILDING%20%20RAS%20AL%20KHAIMAH%20%20United%20Arab%20Emirates&limit=1</a><br>
Geocoding:
5%|█████████▏
| 49/1000 [00:10<04:48, 3.30it/s]2024-12-28 11:49:55 [ERROR]
__main__ - Network/Request error for address='JEWELLERY &
GEMPLEX DUBAI United Arab Emirates': 400 Client Error: Bad
Request for url:
<a class="moz-txt-link-freetext" href="http://localhost:2322/api?q=JEWELLERY%20&%20GEMPLEX%20%20DUBAI%20%20United%20Arab%20Emirates&limit=1">http://localhost:2322/api?q=JEWELLERY%20&%20GEMPLEX%20%20DUBAI%20%20United%20Arab%20Emirates&limit=1</a><br>
Geocoding:
9%|█████████████████
| 91/1000 [00:41<22:12, 1.47s/it]2024-12-28 11:50:27 [ERROR]
__main__ - Network/Request error for address='MADERO EDUARDO AV.
900 PISO:28 1106 BUENOS AIRES Argentina':
HTTPConnectionPool(host='localhost', port=2322): Read timed out.
(read timeout=10)<br>
Geocoding:
9%|█████████████████▌
| 94/1000 [00:42<15:46, 1.04s/it]2024-12-28 11:50:29 [ERROR]
__main__ - Network/Request error for address='BVRD CASTRO BARROS
1527 5000 CORDOBA Argentina':
HTTPConnectionPool(host='localhost', port=2322): Read timed out.
(read timeout=10)<br>
Geocoding:
10%|█████████████████▉
| 96/1000 [00:44<15:08, 1.01s/it]2024-12-28 11:50:36 [ERROR]
__main__ - Network/Request error for address='BERMEJO 1175 7600
MAR DEL PLATA Argentina': HTTPConnectionPool(host='localhost',
port=2322): Read timed out. (read timeout=10)<br>
2024-12-28 11:50:36 [ERROR] __main__ - Network/Request error for
address='AV CORDOBA 2428 1120 CAPITAL FEDERAL Argentina':
HTTPConnectionPool(host='localhost', port=2322): Read timed out.
(read timeout=10)<br>
Geocoding:
10%|██████████████████▏
| 97/1000 [00:51<32:54, 2.19s/it]2024-12-28 11:50:36 [ERROR]
__main__ - Network/Request error for address='25 DE MAYO 509 3300
POSADAS Argentina': HTTPConnectionPool(host='localhost',
port=2322): Read timed out. (read timeout=10)<br>
2024-12-28 11:50:37 [ERROR] __main__ - Network/Request error for
address=' Argentina': HTTPConnectionPool(host='localhost',
port=2322): Read timed out. (read timeout=10)<br>
Geocoding:
10%|██████████████████▌
| 100/1000 [00:52<20:04, 1.34s/it]2024-12-28 11:50:39 [ERROR]
__main__ - Network/Request error for address='AV MENDOZA D PEDRO D
3899 1294 BUENOS AIRES Argentina':
HTTPConnectionPool(host='localhost', port=2322): Read timed out.
(read timeout=10)<br>
Geocoding:
10%|██████████████████▊
| 101/1000 [00:54<21:43, 1.45s/it]2024-12-28 11:50:39 [ERROR]
__main__ - Network/Request error for address='RAWSON 3150 1618
RICARDO ROJAS Argentina': HTTPConnectionPool(host='localhost',
port=2322): Read timed out. (read timeout=10)<br>
Geocoding:
12%|█████████████████████▊
| 117/1000 [01:14<19:49, 1.35s/it]2024-12-28 11:51:01 [ERROR]
__main__ - Network/Request error for address='REP DE HONDURAS 5663
PB 1414 CAPITAL FEDERAL Argentina':
HTTPConnectionPool(host='localhost', port=2322): Read timed out.
(read timeout=10)<br>
Geocoding:
12%|██████████████████████▏
| 119/1000 [01:16<17:27, 1.19s/it]2024-12-28 11:51:02 [ERROR]
__main__ - Network/Request error for address='AV 11 DE SEPTIEMBRE
KM 85 0 5925 FERREYRA Argentina':
HTTPConnectionPool(host='localhost', port=2322): Read timed out.
(read timeout=10)<br>
Geocoding:
24%|█████████████████████████████████████████████▍
| 244/1000 [03:10<15:37, 1.24s/it]<br>
2024-12-28 11:52:56 [ERROR] __main__ - Network/Request error for
address='AVENIDA A SIN NRO 7600 MAR DEL PLATA Argentina':
HTTPConnectionPool(host='localhost', port=2322): Read timed out.
(read timeout=10)<br>
Geocoding:
25%|█████████████████████████████████████████████▊
| 246/1000 [03:11<12:23, 1.01it/s]<br>
2024-12-28 11:52:59 [ERROR] __main__ - Network/Request error for
address='RUTA 34 KM 272 0 PISO:0 DPTO:0 S:0 T:0 M: 0 2324 TACURAL
Argentina': HTTPConnectionPool(host='localhost', port=2322): Read
timed out. (read timeout=10)<br>
Geocoding:
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████|
1000/1000 [07:00<00:00, 2.38it/s]<br>
2024-12-28 11:56:45 [INFO] __main__ - Finished parallel geocoding
of 1000 addresses in 420.58 seconds<br>
2024-12-28 11:56:45 [INFO] botocore.credentials - Found
credentials from IAM Role: photon-geocoder-role<br>
2024-12-28 11:56:46 [INFO] __main__ - Results saved to
<a class="moz-txt-link-freetext" href="s3://istariai-photon-geocoding/source/unique_addresses_for_geocoding_1k_sample.gz">s3://istariai-photon-geocoding/source/unique_addresses_for_geocoding_1k_sample.gz</a><br>
2024-12-28 11:56:46 [INFO] __main__ - Script finished<br>
----------------------------------------------------------------------------------<br>
<br>
Best, <br>
David<br>
<br>
</font>
<div class="moz-signature">-- <br>
<table
style="font-family: Arial, sans-serif; font-size: 10pt; color: #444444; border-top: 1px solid #cccccc; width: 320px;"
cellspacing="0" cellpadding="0">
<tbody>
<tr>
<td style="padding: 10px 0 5px 0;"> <strong
style="font-size: 12pt;">Dr. David Lenz</strong><br>
Co-Founder, istari.ai GmbH </td>
</tr>
<tr>
<td style="padding: 0 0 5px 0;"> <strong>e:</strong> <a
href="mailto:david.lenz@istari.ai"
style="color: #0077aa; text-decoration: none;"
class="moz-txt-link-freetext">david.lenz@istari.ai</a><br>
<strong>w:</strong> <a href="http://www.istari.ai"
style="color: #0077aa; text-decoration: none;">www.istari.ai</a>
</td>
</tr>
<tr>
<td style="padding: 0 0 10px 0; font-size: 9pt;">
Julius-Hatry-Straße 1, 68163 Mannheim </td>
</tr>
</tbody>
</table>
</div>
<ul class="dropdown-menu textcomplete-dropdown"
style="display: none; position: absolute; z-index: 1000;"
contenteditable="false">
</ul>
</body>
</html>