c# - OutOfMemoryException: using DbContext in Task -
this following setup i'm dealing with:
- i got sql-table ~1.000.000 entries need update
- the update takes place in seperate thread (started task)
- as memory limited in thread, process list in batches of 1000 entries per batch (when running in test-project in mainthread/without task, there no oom exception)
- the
updatelist()
function either updates fields of list or creates new record or other tables in dbcontext - in
process_failure()
function, have single context instance entire list - in
process_success()
function, movedwhile
-loop outside of context
private void process_success() { var totalprocessedcounter = 0; while( true ) { using( var context = new mydbcontext() ) { var list = context.myclass.orderby( x => x.id ) .skip( totalprocessedcounter ).take( 1000 ) .tolist(); if( !list.any() ) break; updatelist( list ); totalprocessedcounter += list.count; } } } private void process_failure() { var totalprocessedcounter = 0; using( var context = new mydbcontext() ) { while( true ) { var list = context.myclass.orderby( x => x.id ) .skip( totalprocessedcounter ).take( 1000 ) .tolist(); // throws outofmemoryexception if( !list.any() ) break; updatelist( list ); totalprocessedcounter += list.count; } } } private void updatelist( list<myclass> list ) { var dosavechanges = false; list = list.where( x => somefilter( x ) ).tolist(); using( var context = new mydbcontext() ) { foreach( var item in list ) { changeitem( item ); } if( dosavechanges ) context.savechanges(); } }
gets context
somehow polluted/filled when create instance in nested functioncall of updatelist()
?
the reason why exception because dbcontext
caches the data read db, therefore @ point blow memory if keep adding entities cache , outofmemoryexception
. entities not cleared gc because being referenced dbcontext
.
try using .asnotracking()
:
private void process_notracking() { var totalprocessedcounter = 0; using( var context = new mydbcontext() ) { while( true ) { var list = context .myclass .asnotracking() .orderby( x => x.id ) .skip( totalprocessedcounter ) .take( 1000 ) .tolist(); if( !list.any() ) break; updatelist( list ); totalprocessedcounter += list.count; } } }
but if not track entities, updating them is more difficult (read "attaching existing entity context"), because entities not belong context , not being tracked.
i not use ef somethng this, looks mission update/select sql statement.
Comments
Post a Comment