<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"Préformaté HTML Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Texte de bulles Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
color:black;
mso-fareast-language:EN-US;}
span.PrformatHTMLCar
{mso-style-name:"Préformaté HTML Car";
mso-style-priority:99;
mso-style-link:"Préformaté HTML";
font-family:Consolas;
color:black;
mso-fareast-language:EN-US;}
span.TextedebullesCar
{mso-style-name:"Texte de bulles Car";
mso-style-priority:99;
mso-style-link:"Texte de bulles";
font-family:"Tahoma","sans-serif";}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.EmailStyle22
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="FR-BE" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">(error sending previous mail)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">Hi Peter,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">Thanks for your quick reply.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">The data are travel time histograms.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">For each edge I have two histograms (backward and forward direction).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">Each histograms splits the day into 48 slots of 30 minutes. For each of this slot I have an integer value representing the travel time (in second) on the edge at the time slot.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">Knowing that I have 854 541 edges and 7 days in a week the estimated size could be 854541 *2 * 48 * 7 = 574 251 552 integers => 2 297 006 208 Bytes<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">Now that I have computed it, I think I’ll have to reduce it maybe encoding the time as a byte instead of an int, but it would only reduce by 4.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">The stack trace extract :<o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">java.lang.OutOfMemoryError: Java heap space - problem when allocating new memory. Old capacity: 178257920,
new bytes:1048576, segmentSizeIntsPower:20, new segments:1, existing:170</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.storage.RAMDataAccess.ensureCapacity(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">RAMDataAccess.java:135</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.storage.GraphStorage.incCapacity(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">GraphStorage.java:262</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.storage.traveltime.WeekTTFGraphStorage.ensureTTFIndex(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">WeekTTFGraphStorage.java:318</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.storage.traveltime.WeekTTFGraphStorage.newTravelTimeFunction(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">WeekTTFGraphStorage.java:301</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.reader.MultiNetReaderHelper.linkTTFToEdge(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">MultiNetReaderHelper.java:250</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.reader.MultiNetReader.writeSpeedProfiles(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">MultiNetReader.java:258</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.reader.MultiNetReader.writeMn2Graph(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">MultiNetReader.java:142</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.reader.MultiNetReader.mn2Graph(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">MultiNetReader.java:273</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.reader.MultiNetReader.importGraph(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">MultiNetReader.java:369</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.GraphHopper.importGraph(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">GraphHopper.java:571</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.GraphHopper.importGraph(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">GraphHopper.java:514</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:windowtext;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"> at com.graphhopper.GraphHopper.importOrLoad(</span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:navy;mso-fareast-language:FR-BE">GraphHopper.java:475</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE">)</span><span lang="EN-US" style="color:#1F497D;mso-fareast-language:FR-BE">
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D;mso-fareast-language:FR-BE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D;mso-fareast-language:FR-BE">Which is strange is that I also have the problem using the MMapDataAccess.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D;mso-fareast-language:FR-BE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D;mso-fareast-language:FR-BE">I keep investigating<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"></span><span lang="EN-US" style="font-size:10.0pt;font-family:"Courier New";color:red;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1F497D;mso-fareast-language:FR-BE">Nicolas GILLET<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D;mso-fareast-language:FR-BE"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="color:#1F497D;mso-fareast-language:FR-BE"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><b><span style="color:#1F497D;mso-fareast-language:FR-BE">Market-IP –</span></b><i><span style="color:#1F497D;mso-fareast-language:FR-BE">
</span></i><i><span style="color:#404040;mso-fareast-language:FR-BE">Creating Mobile Intelligence</span></i><span style="color:#1F497D;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D;mso-fareast-language:FR-BE">Phone : +32 81 33 11 11
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D;mso-fareast-language:FR-BE">Fax : +32 81 33 11 10<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D;mso-fareast-language:FR-BE"><a href="http://www.market-ip.com/">www.market-ip.com</a> –
<a href="http://www.telefleet.com/">www.telefleet.com</a> – <a href="http://www.geoplanning.net/">
www.geoplanning.net</a> – <a href="http://www.drivexpert.net/">www.drivexpert.net</a>
<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="FR" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:FR-BE">De :</span></b><span lang="FR" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:FR-BE">
Peter K [<a href="mailto:peathal@yahoo.de">mailto:peathal@yahoo.de</a>] <br>
<b>Envoyé :</b> mardi 2 juillet 2013 11:18<br>
<b>À :</b> GraphHopper Java routing engine<br>
<b>Objet :</b> Re: [GraphHopper] Out of memory at graph creation<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Nicolas,<br>
<br>
creating world wide graph needs lots of RAM (normal graph needs 8GB, CH needs 13GB, creating the latter one needs ~22GB). Although for only belgium territority this should be sufficient. Not sure how your additional data source looks like and what the exact
reason of the OOM might cause. What is the exact stacktrace?<br>
<br>
Did you increase the default -Xmx1000m settings either by editing graphhopper.sh or by providing your own JAVA_OPTS? (export JAVA_OPTS="-Xmx2000m ..."; ./graphhopper.sh ...)<br>
<br>
<span lang="EN-US">> I’d like to know if there could be a way to flush the DataAccess on a regular basis to release memory.</span><br>
<br>
You could use the MMapDataAccess but you'll have to make this thread-safe (e.g. with a simple DataAccess wrapper).
<span lang="EN-US">But if not enough RAM is available => everything will be slower (import + queries)<br>
<br>
Regards,<br>
Peter. <o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:12.0pt;font-family:"Times New Roman","serif";mso-fareast-language:FR-BE"><o:p> </o:p></span></p>
</div>
</body>
</html>