Dockerizált osm postgres DB telepítési jegyzet
Kovacs Szilveszter
hunsly at gmail.com
2018. Nov. 18., V, 00:27:49 UTC
Sziasztok!
Egy rövid leírást csináltam egy Dockerizált osm postgres adatbázis
klónozásról. Azaz csak a magyar területről. Részben magamnak is csináltam
emlékeztetőnek.
Ha tetszik, akkor később beemelhetem a wikibe is.
Docker fájlok letöltése GIT-ről
sly at sly: ~/Documents/HOME/mypgisdb/osm2pgsql$* git clone
https://github.com/OsmHackTW/osm2pgsql-docker.git*
Cloning into 'osm2pgsql-docker'...
remote: Enumerating objects: 116, done.
remote: Total 116 (delta 0), reused 0 (delta 0), pack-reused 116
Receiving objects: 100% (116/116), 16.41 KiB | 190.00 KiB/s, done.
Resolving deltas: 100% (51/51), done.
sly at sly:~/Documents/HOME/mypgisdb/osm2pgsql$ *cd osm2pgsql-docker/*
Futtathatóvá tétel
sly at sly:~/Documents/HOME/mypgisdb/osm2pgsql/osm2pgsql-docker$ *chmod +x
run.sh*
sly at sly:~/Documents/HOME/mypgisdb/osm2pgsql/osm2pgsql-docker$ *chmod +x *
*postgis.sh*
Postgis DB letöltése és indítása
sly at sly:~/Documents/HOME/mypgisdb/osm2pgsql/osm2pgsql-docker$ *sudo
./postgis.sh*
[sudo] password for sly:
Unable to find image 'osmtw/postgis:latest' locally
latest: Pulling from osmtw/postgis
f2aa67a397c4: Pull complete
*[...]*
d5d0857dfa92: Pull complete
Digest:
sha256:7b5096f988ca99e7ea904e968dacf48f0c4dd4c9f3cfe11f025d2ed6e2153d1e
Status: Downloaded newer image for osmtw/postgis:latest
d12745342012af7bb5f9ae643864d9821eb172e61afc8c59dc212440aa49d386
Postgis DB frissítő letöltése és indítása
sly at sly:~/Documents/HOME/mypgisdb/osm2pgsql/osm2pgsql-docker$ *sudo
./run.sh*
Unable to find image 'osmtw/osm2pgsql:0.96.0' locally
docker: Error response from daemon: manifest for osmtw/osm2pgsql:0.96.0 not
found.
See 'docker run --help'.
Valamiért nem működik a v96-al ezért átállítom a run.sh-ban verziót. Ha már
erre járok az országot is:
#!/bin/bash
POSTGIS_INSTANCE=${1:-"osmdb"}
REGION=${2:-"europe/hungary"}
DATADIR=/osm
LOOP=600
VERSION=${3:-"0.90.1"}
docker run -t -i --rm \
--link ${POSTGIS_INSTANCE}:pg \
-e REGION=$REGION \
-e DATADIR=$DATADIR \
-e LOOP=$LOOP \
-v ${POSTGIS_INSTANCE}-volume:$DATADIR \
--name osm2pgsql \
osmtw/osm2pgsql:${VERSION}
Ezekután újra próbálkozás
sly at sly:~/Documents/HOME/mypgisdb/osm2pgsql/osm2pgsql-docker$ *sudo
./run.sh*
Unable to find image 'osmtw/osm2pgsql:0.90.1' locally
0.90.1: Pulling from osmtw/osm2pgsql
297061f60c36: Already exists
[...]
eafdf1df5b82: Pull complete
Digest:
sha256:5b4a511fb29cf30b12275e9cf84b21e497211045d002aeb7c905fc14ef95daaa
Status: Downloaded newer image for osmtw/osm2pgsql:0.90.1
REGION=europe/hungary
COUNTRY=hungary
DATADIR=/osm
PBF=/osm/hungary-latest.osm.pbf
LOOP=600
?column?
----------
1
(1 row)
--2018-11-17 23:34:35--
http://download.geofabrik.de/europe/hungary-latest.osm.pbf
Resolving download.geofabrik.de (download.geofabrik.de)... 88.99.105.98,
88.99.24.113
Connecting to download.geofabrik.de
(download.geofabrik.de)|88.99.105.98|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 152091283 (145M) [application/octet-stream]
Saving to: '/osm/sy8.pbf'
/osm/sy8.pbf
100%[=====================================================================================================================>]
145.04M 2.64MB/s in 64s
2018-11-17 23:35:39 (2.26 MB/s) - '/osm/sy8.pbf' saved [152091283/152091283]
osm2pgsql SVN version 0.90.1-dev (64 bit id space)
Using built-in tag processing pipeline
Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=2000MB, maxblocks=32000*65536, allocation method=11
Mid: pgsql, scale=100 cache=2000
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Reading in file: /osm/sy8.pbf
Using PBF parser.
Processing: Node(16565k 233.3k/s) Way(2264k 73.06k/s) Relation(81750
286.84/s) parse time: 387s
Node stats: total(16565415), max(6067328962) in 71s
Way stats: total(2264858), max(645435423) in 31s
Relation stats: total(81751), max(8971464) in 285s
Committing transaction for planet_osm_point
[...]
Setting up table: planet_osm_rels
Using built-in tag processing pipeline
Going over pending ways...
1326922 ways are pending
Using 8 helper-processes
Finished processing 1326922 ways in 58 s
1326922 Pending ways took 58s at a rate of 22877.97/s
Committing transaction for planet_osm_point
[...]
Stopped table: planet_osm_rels in 1s
node cache: stored: 16565415(100.00%), storage efficiency: 50.17% (dense
blocks: 64, sparse nodes: 16247848), hit rate: 100.20%
Osm2pgsql took 596s overall
'/osm/sy8.pbf' -> '/osm/hungary-latest.osm.pbf'
?column?
----------
1
(1 row)
osmupdate Parameter: --base-url=download.geofabrik.de/europe/hungary-updates
osmupdate Parameter: /osm/hungary-latest.osm.pbf
osmupdate Parameter: /osm/kNe.pbf
osmupdate: timestamp of /osm/hungary-latest.osm.pbf: 2018-11-16T21:14:03Z
osmupdate: newest sporadic timestamp: 2018-11-16T21:14:03Z
osmupdate: Found status information in base URL root.
osmupdate: Ignoring subdirectories "minute", "hour", "day".
osmupdate: Creating output file.
osmupdate: Your OSM file is already up-to-date.
Ekkor lefutott az import is.
*Elérhetőség ellenőrzése.*
Információk lekérése:
*sudo docker exec osmdb env*
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/lib/postgresql/10/bin
HOSTNAME=d12745342012
*POSTGRES_PASSWORD=lQVddc4d3nj*
*POSTGRES_USER=postgres*
*POSTGRES_DB=postgres*
GOSU_VERSION=1.10
LANG=en_US.utf8
PG_MAJOR=10
PG_VERSION=10.4-2.pgdg90+1
PGDATA=/var/lib/postgresql/data
POSTGIS_MAJOR=2.4
POSTGIS_VERSION=2.4.4+dfsg-1.pgdg90+1
HOME=/root
IP:
*sudo docker exec osmdb bash -c 'hostname -I'*
172.17.0.2
Kapcsolódás az adatbázishoz:
*psql -h 172.17.0.2 -U postgres -p 5432 -l*
Password for user postgres:
List of databases
Name | Owner | Encoding | Collate | Ctype |
Access privileges
------------------+----------+----------+------------+------------+-----------------------
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
=c/postgres +
| | | | |
postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
=c/postgres +
| | | | |
postgres=CTc/postgres
template_postgis | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
(4 rows)
* sudo docker ps*
[sudo] password for sly:
CONTAINER ID IMAGE COMMAND
CREATED STATUS PORTS NAMES
847f6527107c osmtw/osm2pgsql:0.90.1 "/bin/bash -i /usr/l…" 8
minutes ago Up 7 minutes osm2pgsql
d12745342012 osmtw/postgis "docker-entrypoint.s…" 22
minutes ago Up 21 minutes 5432/tcp osmdb
*Megjegyzés:*
Ha jól sejtem 600 másodpercenként frissül az adatbázis, amit az osm2pgsql
docker csinál.
osmdb maga az adatbázis.
--------- következő rész ---------
Egy csatolt HTML állomány át lett konvertálva...
URL: <http://lists.openstreetmap.org/pipermail/talk-hu/attachments/20181117/f257764c/attachment.htm>
További információk a(z) Talk-hu levelezőlistáról