docker - Why is Internal memory in java Native Memory Tracking increasing -


my application running in docker container, use scala , use "openjdk 64-bit server vm (build 25.102-b14, mixed mode)", xmx set 16gb , container memory limit 24gb, after running time container killed:

last state:         terminated   reason:           oomkilled   exit code:        137

however can't find "java.lang.outofmemoryerror: java heap space" errors in log, not once in last 2 weeks in 48 nodes. it's not normal heap oom.

dmesg output:

$ dmesg -l err,crit,alert,emerg stdin not terminal [1647254.978515] memory cgroup out of memory: kill process 10924 (java) score 1652 or sacrifice child [1647254.989138] killed process 10924 (java) total-vm:34187148kb, anon-rss:24853120kb, file-rss:23904kb [1655749.664871] memory cgroup out of memory: kill process 1969 (java) score 1652 or sacrifice child [1655749.675513] killed process 1969 (java) total-vm:35201940kb, anon-rss:24856624kb, file-rss:24120kb [1655749.987605] memory cgroup out of memory: kill process 2799 (java) score 1656 or sacrifice child

i run jcmd multiple times before killed again , data looks following: native memory tracking:

total: reserved=25505339kb, committed=25140947kb - java heap (reserved=16777216kb, committed=16777216kb) (mmap: reserved=16777216kb, committed=16777216kb)

  • class (reserved=247996kb, committed=93500kb) (classes #14539) (malloc=2236kb #29794) (mmap: reserved=245760kb, committed=91264kb)

  • thread (reserved=1013160kb, committed=1013160kb) (thread #1902) (stack: reserved=1003956kb, committed=1003956kb) (malloc=6240kb #9523) (arena=2964kb #3803)

  • code (reserved=263255kb, committed=86131kb) (malloc=13655kb #20964) (mmap: reserved=249600kb, committed=72476kb)

  • gc (reserved=776174kb, committed=776174kb) (malloc=120814kb #164310) (mmap: reserved=655360kb, committed=655360kb)

  • compiler (reserved=812kb, committed=812kb) (malloc=681kb #1823) (arena=131kb #3)

  • internal (reserved=6366260kb, committed=6366256kb) (malloc=6366256kb #178778) (mmap: reserved=4kb, committed=0kb)

  • symbol (reserved=18391kb, committed=18391kb) (malloc=16242kb #153138) (arena=2150kb #1)

  • native memory tracking (reserved=9002kb, committed=9002kb) (malloc=186kb #2000) (tracking overhead=8816kb)

  • arena chunk (reserved=273kb, committed=273kb) (malloc=273kb)

  • unknown (reserved=32800kb, committed=32kb) (mmap: reserved=32800kb, committed=32kb)

one thing noticed section: internal (reserved=6366260kb, committed=6366256kb)

it keeps growing , causing total memory usage exceed 24gb limit.

anyone has seen similar issue before? , knows internal memory here , reason keeps growing without releasing memory?

this not answer question, workaround.

i have observed same problem in docker containers running jruby on java version "1.8.0_45". solution explicitly invoke garbage collection. have absolutely no idea, why works, after gc internal java memory returned 8mb.


-

Comments

Post a Comment

Popular posts from this blog

javascript - Thinglink image not visible until browser resize -

Image
i trying repurpose pre-existing theme, includes drop down menu displays image. trying embed thinglink drop down menu's content section, not visible until resize browser window. here menu looks like: then when clicked, can see empty space image should show: then when browser resized, image appears: this code content of menu: <div class="ajax_accordion_content" style="display: none;"> <div class="report-detail"> <table border="0" cellpadding="0" cellspacing="0" width="100%" class="table_basic"> <tbody> <!-- thinglink image embedded --> <img style="max-width: 100%;" src="http://cdn.thinglink.me/api/image/832705713204625409/1024/10/scaletowidth#tl-832705713204625409;1043138249'" class="alwaysthinglink"/> <script async charset="utf-8" src="http://cdn.thinglink.me/jse/embed.j
Read more

firebird - Error "invalid transaction handle (expecting explicit transaction start)" executing script from Delphi -

i'm developing update app delphi 10. i'm running windows 7 64x, firebird 2.5.1.26351 32x. execution order: checks current version(select statement firebird database) downloads update(via ftp) apply it(.exe files, , execute .sql) error code -901 pops out when try execute .sql files. know there incompatibility issue w7 64x , fdb 32x, but, can everything: backup , restore(gbak), db commands(all). tried fdb 64x instead, , not working. code: //components fdwaitcursor: tfdguixwaitcursor; fddriverlink: tfdphysfbdriverlink; fdconnection:tfdconnection; fdscript: tfdscript; function tthr_script.createcomponents:boolean; begin try fddriverlink := tfdphysfbdriverlink.create(fddriverlink); fdwaitcursor := tfdguixwaitcursor.create(fdwaitcursor); fdconnection := tfdconnection.create(fdconnection); fdconnection begin drivername := 'fb'; params.username := thrbanco.bduser; params.password := thrbanco.bdpass; params.database :=
Read more

mongodb - How to keep track of users making Stripe Payments -

i'm trying figure out way mean stack application keep track of users have paid grant them access portion of webpage. i've considered several options: stripe customer id, mongodb record, , html attribute can update. my mean stack keeps track of users jwt, , appears stripe assigns them own customer id isn't ideal. can done jwt opposed forced cutomer id? mongodb record. i'm thinking might best option. when new user has been created, i'll give attribute of haspaid = no. update record of customer when payment submitted. guess run script set unpaid each day? html element/attribute. don't know if possible; cool create key carried during html session after payment received. if person closers browser session closed? i'm looking guidance on 3 options determine if they're best solution. also, if has suggestions alternatives, i'm ears! thanks in advance. speaking generally, common approach second one: use attribute in data model indicates
Read more