java - Elasticsearch increasing heap size -


we running elasticsearch inside docker container on amazons ecs. noticed heap increase on time. first time noticed when raised above 70% , started throw away requests (indices.breaker.total.limit).

the thing never seen decreasing heap, feels fishy!

so far have increased instance size, running instance 30g memory. heap set aprox half memory, es_heap_size=14g (xmx=xms=14g).

someone else have similar experience? bug in elasticsearch? or incorrect configurated?

elasticsearch ver: 1.5.1

> curl localhost:9200/_cat/fielddata?v  id                     host         ip         node            total position deal_status heading.sortorder org_relationlastmodified deal_value deal_probability 1_zipcodevisit_0   tag employer_tag 1_cityvisit_0 dateofregistration temperature uniqueid _type quick_ratio org_relation employer_relationlastmodified turnover turnover_per_employee deal_source employer_relation deal_statusdate 1_custom_1466 average_salary_per_employee deal_orderdate 0_numberofemployeesrange_20 1_legalform_0 1_custom_1816 0_worksitetype_100 0_lineofbusiness_2 equity_ratio profitmargin 0_lineofbusiness_1 0_countyvisit_40 0_numberofemployeesrange_22 0_municipalityvisit_61 0_legalform_110 dividends 1_custom_1744 0_municipalityvisit_60 responsiblecoworker result_before_tax xmtlkdnstokvmhqgapmbag 5dc819096905 172.17.0.2 hitman        729.8mb    8.1mb       1.1mb           261.5mb                    1.7mb    305.3kb          849.1kb           20.9mb 6.4mb        1.3mb        19.3mb             12.3mb          0b  283.7mb 9.6mb       5.1mb      810.5kb                       632.2kb   11.6mb                 4.1mb     150.8kb           566.4kb         568.6kb        34.1kb                       4.2mb        973.5kb                       5.7mb         4.6mb        37.4kb              4.9mb              8.1mb        4.7mb        4.2mb              9.2mb            3.3mb                       4.2mb                802.9kb           3.9mb     4.3mb        37.7kb                  7.5mb               2.4mb               5mb dhaowkhmqksnwab0krjrjw 8ffc068518f9 172.17.0.2 moira brandon 718.9mb    8.2mb       1.1mb           261.5mb                    1.3mb      124kb          793.3kb           19.6mb 6.4mb          1mb        19.1mb             10.2mb          0b  283.8mb 9.6mb       5.2mb      714.7kb                       791.3kb    8.8mb                 3.4mb          0b           422.6kb          83.9kb        16.8kb                       4.6mb        989.4kb                       5.6mb         4.5mb            0b              5.2mb              7.9mb        4.1mb        4.3mb                9mb            3.2mb                       4.3mb                     0b           3.8mb     4.3mb            0b                  7.1mb               2.5mb             4.4mb

[update 2016-10-24] have updated version 2.4.0 still experience same problem. if force gc, heap released 4%, that's same value fresh instance.

example instance 73% heap, jvm mem shows old 1 10g, not sure if that's normal

jvm mem heap percent 73% "young":    "used_in_bytes" : 199026920 "survivor": "used_in_bytes" : 2422528 "old":      "used_in_bytes" : 10754631392

what triggers gc? should let heap increase above 70%?

this maybe related kind-of known behavior in pre 2.x versions, affect kibana, guess elasticsearch.

see github issue : https://github.com/elastic/kibana/issues/5170

it may same in case, boils down node issue : https://github.com/nodejs/node/issues/2683

it may configuration in es not ideal. usual suspect in elasticsearch config:

bootstrap.mlockall: true

do have lot of shards / replicas ? use kibana ?


-

Comments

Post a Comment

Popular posts from this blog

javascript - Thinglink image not visible until browser resize -

Image
i trying repurpose pre-existing theme, includes drop down menu displays image. trying embed thinglink drop down menu's content section, not visible until resize browser window. here menu looks like: then when clicked, can see empty space image should show: then when browser resized, image appears: this code content of menu: <div class="ajax_accordion_content" style="display: none;"> <div class="report-detail"> <table border="0" cellpadding="0" cellspacing="0" width="100%" class="table_basic"> <tbody> <!-- thinglink image embedded --> <img style="max-width: 100%;" src="http://cdn.thinglink.me/api/image/832705713204625409/1024/10/scaletowidth#tl-832705713204625409;1043138249'" class="alwaysthinglink"/> <script async charset="utf-8" src="http://cdn.thinglink.me/jse/embed.j...
Read more

firebird - Error "invalid transaction handle (expecting explicit transaction start)" executing script from Delphi -

i'm developing update app delphi 10. i'm running windows 7 64x, firebird 2.5.1.26351 32x. execution order: checks current version(select statement firebird database) downloads update(via ftp) apply it(.exe files, , execute .sql) error code -901 pops out when try execute .sql files. know there incompatibility issue w7 64x , fdb 32x, but, can everything: backup , restore(gbak), db commands(all). tried fdb 64x instead, , not working. code: //components fdwaitcursor: tfdguixwaitcursor; fddriverlink: tfdphysfbdriverlink; fdconnection:tfdconnection; fdscript: tfdscript; function tthr_script.createcomponents:boolean; begin try fddriverlink := tfdphysfbdriverlink.create(fddriverlink); fdwaitcursor := tfdguixwaitcursor.create(fdwaitcursor); fdconnection := tfdconnection.create(fdconnection); fdconnection begin drivername := 'fb'; params.username := thrbanco.bduser; params.password := thrbanco.bdpass; params.database := ...
Read more

Sound is not coming out while implementing Text-to-speech in Android activity -

i using texttospeak in 1 of app. while adding speech string texttospeech object instance, doesn't produce sound. though not getting error in log still sound not coming out. want device speak value of textview gets set resultmsg.settext() method. public class resultactivity extends activity { @override protected void oncreate(bundle savedinstancestate){ ............ resultmsg.settext(resultmsg_str); texttospeech = new texttospeech(getapplicationcontext(),newtexttospeech.oninitlistener(){ @override public void oninit(int status) { system.out.println("txt speech status = "+status); system.out.println("txt speech error status = "+texttospeech.error); system.out.println("txt speech success status = "+texttospeech.success); if(status == texttospeech.success){ texttospeech.setlanguage(locale.english); ...
Read more