<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Nicolas,<br>
<br>
that kind of information should be probably stored in a
MMapDataAccess as you will often request the current time (I
guess) and so the OS does not need to put this data away on disc.<br>
(+do not forget the sync wrapper)<br>
<br>
Or you could manage this on your own in a new DataAccess class and
additionally try to compress that data and maintain a kind of a
cache. see some compression code in CompressedArray<br>
<br>
<span style="color:#1F497D;mso-fareast-language:FR-BE"
lang="EN-US"><o:p></o:p></span>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE" lang="EN-US">>
Which is strange is that I also have the problem using the
MMapDataAccess.</span></p>
<br>
Hmmh, are you on a 32bit machine? And are you making sure the JVM
gets enough RAM (via -Xmx)?<br>
<br>
Regards,<br>
Peter.<br>
<br>
</div>
<blockquote
cite="mid:aef105014d664a7b927388a83d14fab5@AMXPR05MB056.eurprd05.prod.outlook.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
color:black;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"Préformaté HTML Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Texte de bulles Car";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
color:black;
mso-fareast-language:EN-US;}
span.TextedebullesCar
{mso-style-name:"Texte de bulles Car";
mso-style-priority:99;
mso-style-link:"Texte de bulles";
font-family:"Tahoma","sans-serif";}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.PrformatHTMLCar
{mso-style-name:"Préformaté HTML Car";
mso-style-priority:99;
mso-style-link:"Préformaté HTML";
font-family:"Consolas","serif";
color:black;
mso-fareast-language:EN-US;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#1F497D">Hi Peter,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Thanks
for your quick reply.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">The
data are travel time histograms.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">For
each edge I have two histograms (backward and forward
direction).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Each
histograms splits the day into 48 slots of 30 minutes. For
each of this slot I have an integer value representing the
travel time (in second) on the edge at the time slot.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Knowing
that I have 854 541 edges and 7 days in a week the estimated
size could be 854541 *2 * 48 * 7 = 574 251 552 integers
=> 2 297 006 208 Bytes<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">Now
that I have computed it, I think I’ll have to reduce it
maybe encoding the time as a byte instead of an int, but it
would only reduce by 4.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US">The
stack trace extract :<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D" lang="EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span
style="font-size:10.0pt;color:#1F497D;mso-fareast-language:FR-BE">Nicolas
GILLET<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span
style="color:#1F497D;mso-fareast-language:FR-BE"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><b><span
style="color:#1F497D;mso-fareast-language:FR-BE">Market-IP
–</span></b><i><span
style="color:#1F497D;mso-fareast-language:FR-BE">
</span></i><i><span
style="color:#404040;mso-fareast-language:FR-BE">Creating
Mobile Intelligence</span></i><span
style="color:#1F497D;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE">Phone :
+32 81 33 11 11
<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE">Fax : +32
81 33 11 10<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE"><a
moz-do-not-send="true" href="http://www.market-ip.com/">www.market-ip.com</a>
–
<a moz-do-not-send="true" href="http://www.telefleet.com/">www.telefleet.com</a>
– <a moz-do-not-send="true"
href="http://www.geoplanning.net/">
www.geoplanning.net</a> – <a moz-do-not-send="true"
href="http://www.drivexpert.net/">www.drivexpert.net</a>
<o:p></o:p></span></p>
<p class="MsoNormal"><a moz-do-not-send="true"
href="http://www.linkedin.com/groups/MarketIP-4289716?gid=4289716&trk=hb_side_g"><span
style="color:#1F497D;mso-fareast-language:FR-BE;text-decoration:none"><img
id="Image_x0020_1"
src="cid:part5.00030701.09070503@yahoo.de"
alt="Description : cid:image003.png@01CD5521.849F3340"
border="0" height="25" width="30"></span></a><span
style="color:#1F497D;mso-fareast-language:FR-BE"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:FR-BE"
lang="FR">De :</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;mso-fareast-language:FR-BE"
lang="FR"> Peter K [<a class="moz-txt-link-freetext" href="mailto:peathal@yahoo.de">mailto:peathal@yahoo.de</a>] <br>
<b>Envoyé :</b> mardi 2 juillet 2013 11:18<br>
<b>À :</b> GraphHopper Java routing engine<br>
<b>Objet :</b> Re: [GraphHopper] Out of memory at graph
creation<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Nicolas,<br>
<br>
creating world wide graph needs lots of RAM (normal graph
needs 8GB, CH needs 13GB, creating the latter one needs
~22GB). Although for only belgium territority this should be
sufficient. Not sure how your additional data source looks
like and what the exact reason of the OOM might cause. What
is the exact stacktrace?<br>
<br>
Did you increase the default -Xmx1000m settings either by
editing graphhopper.sh or by providing your own JAVA_OPTS?
(export JAVA_OPTS="-Xmx2000m ..."; ./graphhopper.sh ...)<br>
<br>
<span lang="EN-US">> I’d like to know if there could be a
way to flush the DataAccess on a regular basis to release
memory.</span><br>
<br>
You could use the MMapDataAccess but you'll have to make
this thread-safe (e.g. with a simple DataAccess wrapper).
But if not enough RAM is available => everything will be
slower (import + queries)<br>
<br>
Regards,<br>
Peter.<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><span lang="EN-US">Hello,</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">I’m facing an
OutOfMemoryError (heap space) when creating a bigger
graph.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Actually, I am
creating a graph out of our own data source including
speed profile definitions on the Belgian territory.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Creating a simple
graph works fine but when it comes to create the speed
profiles, my process quickly runs out of memory.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">To store these speed
profiles, I have created 7 more DataAccess, one for each
day of week and each edge has one speed profile stored in
there.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">I’d like to know if
there could be a way to flush the DataAccess on a regular
basis to release memory.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">When I see that you
managed to create a worldwide graph, I guess there might
be such mechanism but I couldn’t find it.</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US">Thank you</span><o:p></o:p></p>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:10.0pt;color:#1F497D;mso-fareast-language:FR-BE">Nicolas
GILLET</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE"> </span><o:p></o:p></p>
<p class="MsoNormal"><b><span
style="color:#1F497D;mso-fareast-language:FR-BE"> </span></b><o:p></o:p></p>
<p class="MsoNormal"><b><span
style="color:#1F497D;mso-fareast-language:FR-BE">Market-IP
–</span></b><i><span
style="color:#1F497D;mso-fareast-language:FR-BE">
</span></i><i><span
style="color:#404040;mso-fareast-language:FR-BE">Creating
Mobile Intelligence</span></i><o:p></o:p></p>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE">Phone :
+32 81 33 11 11
</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE">Fax : +32
81 33 11 10</span><o:p></o:p></p>
<p class="MsoNormal"><span
style="color:#1F497D;mso-fareast-language:FR-BE"><a
moz-do-not-send="true" href="http://www.market-ip.com/">www.market-ip.com</a>
–
<a moz-do-not-send="true" href="http://www.telefleet.com/">www.telefleet.com</a>
– <a moz-do-not-send="true"
href="http://www.geoplanning.net/">
www.geoplanning.net</a> – <a moz-do-not-send="true"
href="http://www.drivexpert.net/">www.drivexpert.net</a>
</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-language:FR-BE"><br>
<br>
<br>
<o:p></o:p></span></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>GraphHopper mailing list<o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="mailto:GraphHopper@openstreetmap.org">GraphHopper@openstreetmap.org</a><o:p></o:p></pre>
<pre><a moz-do-not-send="true" href="http://lists.openstreetmap.org/listinfo/graphhopper">http://lists.openstreetmap.org/listinfo/graphhopper</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman","serif";mso-fareast-language:FR-BE"><o:p> </o:p></span></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
GraphHopper mailing list
<a class="moz-txt-link-abbreviated" href="mailto:GraphHopper@openstreetmap.org">GraphHopper@openstreetmap.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstreetmap.org/listinfo/graphhopper">http://lists.openstreetmap.org/listinfo/graphhopper</a>
</pre>
</blockquote>
<br>
</body>
</html>