<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1251">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
> <span style="font-family: -webkit-standard;">When you need relations-ways-nodes read order, blocks will save you from unnecessary read-through the whole file (yes, you can skip decompression for nodes/ways but still you must read the whole file).</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<span style="font-family: -webkit-standard;"><br>
</span></div>
<div>Or use mmap. Or directly Cap’n Proto or FlatBuffers, which support this out of the box.</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<a href="https://capnproto.org/cxx.html#using-mmap">https://capnproto.org/cxx.html#using-mmap</a></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<span style="font-family: -webkit-standard;"><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<span style="font-family: -webkit-standard;">> </span><span style="font-family: -webkit-standard;">Second example: find something by id, if you have blocks it's easy to map whole block into memory and do a binary search for logN block reads instead of seeing
through a file all the time. </span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<span style="font-family: -webkit-standard;"><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
I believe it will make implementation unnecessary more complicated. In case of Cap’n Proto you could set the initial position of your binary search to the array index, under the hood it would use mmap to seek the location. Also you can improve on lookup: for
example start a binary search for way with id 999 at ways[999] instead of ways[waysCount/2] </div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<span style="font-family: -webkit-standard;"><br>
</span></div>
<span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">Von: </span>Äìèòðèé Êèñåëåâ <<a href="mailto:dmitry.v.kiselev@gmail.com">dmitry.v.kiselev@gmail.com</a>><br>
<span style="font-weight:bold">Datum: </span>Sonntag, 7. Februar 2016 um 20:10<br>
<span style="font-weight:bold">An: </span>Andrew Byrd <<a href="mailto:andrew@fastmail.net">andrew@fastmail.net</a>><br>
<span style="font-weight:bold">Cc: </span>Benjamin Stadin <<a href="mailto:benjamin.stadin@heidelberg-mobil.com">benjamin.stadin@heidelberg-mobil.com</a>>, "<a href="mailto:dev@openstreetmap.org">dev@openstreetmap.org</a>" <<a href="mailto:dev@openstreetmap.org">dev@openstreetmap.org</a>><br>
<span style="font-weight:bold">Betreff: </span>Re: [OSM-dev] Simpler binary OSM formats<br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="color: rgb(0, 0, 0); font-family: -webkit-standard; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">
<div>As for fixed sized blocks in vex, I did consider that option but couldn’t come up with a compelling reason for it. I can see the case for a maximum block size (so we know what the maximum size of allocation will be), but can you give a concrete example
of how fixed-size blocks would be advantageous in practice? I would be very hesitant to split any entity across multiple blocks.</div>
<div class="">
<div id=":ow" class="" tabindex="0"><img class="" src="https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif"></div>
</div>
</blockquote>
<div style="color: rgb(0, 0, 0); font-family: -webkit-standard; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: -webkit-standard; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
When you need relations-ways-nodes read order, blocks will save you from unnecessary read-through the whole file (yes, you can skip decompression for nodes/ways but still you must read the whole file).<br>
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: -webkit-standard; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Second example: find something by id, if you have blocks it's easy to map whole block into memory and do a binary search for logN block reads instead of seeing through a file all the time.</div>
</span>
</body>
</html>