View Issue Details

Related ChangesetsJump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0026515CommunityOCCT:Modeling Algorithmspublic2015-08-04 17:332015-10-23 20:50
Reportersrlockley Assigned Tobugmaster  
PrioritynormalSeveritymajor 
Status closedResolutionfixed 
PlatformWindowsOSVC++ 2013 
Product Version6.9.0 
Target Version6.9.1Fixed in Version6.9.1 
Summary0026515: Exponential memory usage problems with BOPDS_IndexedMapOfPaveBlock and NCollection_IncAllocator
DescriptionIn BopAlgo_PaveFiller, MakeBlocks(), NCollection_IncAllocator is used to allocate memory for the collections. When largish sets of tools (>150 SHAPES) are used in a boolean operation the memory consumption of this class rises exponentially, over 2GB of memory is consumed when the collections have the following numbers
aLSE=3607
aLBV=0
aMVOnIn=12
aMF=3739
aMVStick=0
aMVEF=0
aMI=18
aMPBOnIn=15
aMPBAdd=9799
aLPB=1
aMSCPB=2974
aMVI=4832
aDMExEdges=222
aMVTol=0
aDMI=222
aDMBV=0

These are not really large numbers and definitely should not consume 2GB of RAM
Removing the use of the allocator from the collection constructors completely resolves the problem, in fact there is no significant memory use at all in Task manager or processes viewer. Indicating the issue is in the allocation class.
I have tried to find the cause, it appears to be BOPDS_IndexedMapOfPaveBlock and BOPCol_MapOfInteger that causes excessive memory allocation.
Strangely, removing the use of the allocator not only improves memory performance but also speed in my environment (.NET, mix mode)
Steps To ReproduceCut more than 150 shapes from any shape. In fact the main class that clauses the problem BOPDS_IndexedMapOfPaveBlock only has 15 members in my example so the problem hopefully is simple, but I am afraid i couldn't grasp it.
TagsNo tags attached.
Test case numberbugs modalg_6 bug26567

Activities

msv

2015-08-04 18:11

developer   ~0043821

The aim of using IncAllocator is to reduce the number of times of calling malloc/free. It never returns memory back to system till its destruction or till someone calls its Reset method. Therefore it should be used with care. If a container is constructed with IncAllocator then it is supposed one of two conditions to be true:
1. Life time of this container is small.
2. If life time is large then the container will accommodate new objects without removing existing objects, because allocator's Free method does not return memory to system.
Probably, for container-fields of BOP classes that live long it is needed to refuse passing IncAllocator to constructor.

msv

2015-08-25 12:38

developer   ~0044597

Dear srlockley,
Could you, please attach input data and provide the draw script in "steps to reproduce" field? We need it to reproduce your problem and insert it in non-regression test base.
Thanks in advance,
Mikhail

srlockley

2015-08-27 13:43

reporter   ~0044739

I will try it was a complex operation that didn't really set well in a brep file.

I will see if I can convert it to a draw script but it may take me a while.

PS I am pretty sure the function is only caused by the two functions I pinpointed BOPDS_IndexedMapOfPaveBlock and BOPCol_MapOfInteger is there something different about these from other maps?

msv

2015-08-27 14:48

developer   ~0044745

I did not understand the question, but the problem is here not with maps, but with overall algorithm behavior and mutual scope of containers and IncAllocator used for them. We have already another bug, which fix should help in your case. However, it would be fine to add to non-regression test base your case too, if possible.

msv

2015-08-27 14:50

developer   ~0044746

Dear Ilya, please take the patch from the branch to related issue #26567, and merge it in master.

git

2015-09-10 15:25

administrator   ~0045425

Branch CR26515 has been created by isn.

SHA-1: abd17b99dddbe30bdd76c4f27e82b984c5d6f5a0


Detailed log of new commits:

Author: pkv
Date: Tue Aug 18 10:06:11 2015 +0300

    0026567: Exception in Boolean intersection command
    
    Correcting the usage of NCollection_IncAllocator
    
    Conflicts:
        src/BOPAlgo/BOPAlgo_BOP.cxx
        src/BOPAlgo/BOPAlgo_Builder.cxx
        src/BOPAlgo/BOPAlgo_BuilderSolid.cxx
        src/BOPAlgo/BOPAlgo_Builder_2.cxx
        src/BOPAlgo/BOPAlgo_MakerVolume.cxx
        src/BOPAlgo/BOPAlgo_PaveFiller.cdl
        src/BOPAlgo/BOPAlgo_PaveFiller_1.cxx
        src/BOPAlgo/BOPAlgo_PaveFiller_10.cxx
        src/BOPAlgo/BOPAlgo_PaveFiller_3.cxx
        src/BOPAlgo/BOPAlgo_PaveFiller_4.cxx
        src/BOPAlgo/BOPAlgo_PaveFiller_5.cxx
        src/BOPAlgo/BOPAlgo_PaveFiller_6.cxx
        src/BOPAlgo/BOPAlgo_PaveFiller_7.cxx
        src/BOPAlgo/BOPAlgo_PaveFiller_8.cxx
        src/BOPAlgo/BOPAlgo_Section.cxx
        src/BOPDS/BOPDS_DS.cxx
        src/BOPDS/BOPDS_Iterator.cxx
        src/BOPDS/BOPDS_IteratorSI.cxx
        src/BOPDS/BOPDS_SubIterator.cxx
        src/BOPTest/BOPTest_BOPCommands.cxx
        src/BOPTest/BOPTest_PartitionCommands.cxx
        src/IntTools/IntTools_Context.cxx

isn

2015-09-10 15:27

developer   ~0045426

Last edited: 2015-09-10 15:28

 Branch from issue ##26567 have been merged with the current master

msv

2015-09-10 17:51

developer   ~0045447

Some changes were not transfered. Please check all changes manually.

git

2015-09-10 18:42

administrator   ~0045453

Branch CR26515 has been updated forcibly by isn.

SHA-1: a370d63830e66de0d944589caf1e0ed943b79426

msv

2015-09-11 11:15

developer   ~0045468

The changes in the files

src\BOPAlgo\BOPAlgo_PaveFiller_7.cxx
src\BOPAlgo\BOPAlgo_Section.cxx
src\BOPTest\BOPTest_PartitionCommands.cxx

were not transfered. Please make the changes in them also.

git

2015-09-11 11:34

administrator   ~0045470

Branch CR26515 has been updated by isn.

SHA-1: 7242be6c1e85bb7d55d83ccb388c7ce5a51b2d75


Detailed log of new commits:

Author: isn
Date: Fri Sep 11 11:29:57 2015 +0300

    additional corrections

msv

2015-09-11 16:17

developer   ~0045505

Reviewed.

git

2015-09-11 17:43

administrator   ~0045522

Branch CR26515 has been updated forcibly by apv.

SHA-1: dc9b9b99dee2e43209cab3523efe05dbf41b88f9

apv

2015-09-11 17:43

tester   ~0045524

Branch CR26515 has been rebased on the current master

apv

2015-09-14 14:25

tester   ~0045570

Dear BugMaster,

Branch CR26515 from occt git-repository (and master from products git-repository) was compiled on Linux and Windows platforms and tested.
SHA-1: dc9b9b99dee2e43209cab3523efe05dbf41b88f9

Number of compiler warnings:
occt component:
   Linux: 15 (15 on master)
   Windows: 0 (0 on master)
products component:
   Linux: 39 (39 on master)
   Windows: 0 (0 on master)

Regressions/Differences:
http://occt-tests/CR26515-master-occt-64/Debian70-64/summary.html
http://occt-tests/CR26515-master-occt-64/Windows-64-VC10/summary.html
bugs fclasses bug7287_1

Testing on Linux:
Total MEMORY difference: 90349201 / 92721808 [-2.56%]
Total CPU difference: 17980.68999999927 / 17686.84999999906 [+1.66%]

Testing on Windows:
Total MEMORY difference: 56152860 / 57753560 [-2.77%]
Total CPU difference: 16694.18181329908 / 16847.78039789915 [-0.91%]

msv

2015-09-14 14:57

developer   ~0045580

Dear Alexey, please remove TODO of the improved test case bugs fclasses bug7287_1.

git

2015-09-15 14:00

administrator   ~0045634

Branch CR26515 has been updated by abv.

SHA-1: b84c457f8aa899ebae3f3c99f0a03be86f64bf83


Detailed log of new commits:

Author: abv
Date: Tue Sep 15 13:54:13 2015 +0300

    Adaptation of test case bugs fclasses bug7287_1 (limit for detection of memory leak reduced due to leak reduction)

abv

2015-09-15 14:02

manager   ~0045635

I have pushed correction of test case bugs fclasses bug7287_1 to branch CR26515, please check it (only this test!).

Besides, while integrating this fix, note that title of its commit messages is that of #26567; this is correct.

git

2015-09-15 17:42

administrator   ~0045659

Branch CR26515 has been updated by apv.

SHA-1: 13471ac8a1aea34766494eb214db25b26f2f6dd8


Detailed log of new commits:

Author: apv
Date: Tue Sep 15 17:42:07 2015 +0300

    Adjusting test-case bugs fclasses bug7287_1

apv

2015-09-15 17:44

tester   ~0045660

Result of testing case bugs fclasses bug7287_1:
http://occt-tests/CR26515-master-occt-64/Debian70-64/bugs/fclasses/bug7287_1.html
http://occt-tests/CR26515-master-occt-64/Windows-64-VC10/bugs/fclasses/bug7287_1.html

git

2015-10-16 16:37

administrator   ~0046981

Branch CR26515 has been deleted by kgv.

SHA-1: 13471ac8a1aea34766494eb214db25b26f2f6dd8

Related Changesets

occt: master 488e5b9d

2015-09-17 10:26:46

isn


Committer: bugmaster Details Diff		 0026515: Exponential memory usage problems with BOPDS_IndexedMapOfPaveBlock and NCollection_IncAllocator

Correcting the usage of NCollection_IncAllocator

Conflicts:
src/BOPAlgo/BOPAlgo_BOP.cxx
src/BOPAlgo/BOPAlgo_Builder.cxx
src/BOPAlgo/BOPAlgo_BuilderSolid.cxx
src/BOPAlgo/BOPAlgo_Builder_2.cxx
src/BOPAlgo/BOPAlgo_MakerVolume.cxx
src/BOPAlgo/BOPAlgo_PaveFiller.cdl
src/BOPAlgo/BOPAlgo_PaveFiller_1.cxx
src/BOPAlgo/BOPAlgo_PaveFiller_10.cxx
src/BOPAlgo/BOPAlgo_PaveFiller_3.cxx
src/BOPAlgo/BOPAlgo_PaveFiller_4.cxx
src/BOPAlgo/BOPAlgo_PaveFiller_5.cxx
src/BOPAlgo/BOPAlgo_PaveFiller_6.cxx
src/BOPAlgo/BOPAlgo_PaveFiller_7.cxx
src/BOPAlgo/BOPAlgo_PaveFiller_8.cxx
src/BOPAlgo/BOPAlgo_Section.cxx
src/BOPDS/BOPDS_DS.cxx
src/BOPDS/BOPDS_Iterator.cxx
src/BOPDS/BOPDS_IteratorSI.cxx
src/BOPDS/BOPDS_SubIterator.cxx
src/BOPTest/BOPTest_BOPCommands.cxx
src/BOPTest/BOPTest_PartitionCommands.cxx
src/IntTools/IntTools_Context.cxx

additional corrections

Adaptation of test case bugs fclasses bug7287_1 (limit for detection of memory leak reduced due to leak reduction)

Adjusting test-case bugs fclasses bug7287_1		 Affected Issues
0026515
mod - src/BOPAlgo/BOPAlgo_BOP.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_Builder.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_BuilderSolid.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_Builder_2.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_Builder_3.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_MakerVolume.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller.hxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller_1.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller_10.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller_3.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller_4.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller_5.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller_6.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller_7.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_PaveFiller_8.cxx Diff File
mod - src/BOPAlgo/BOPAlgo_Section.cxx Diff File
mod - src/BOPDS/BOPDS_DS.cxx Diff File
mod - src/BOPDS/BOPDS_Iterator.cxx Diff File
mod - src/BOPDS/BOPDS_IteratorSI.cxx Diff File
mod - src/BOPDS/BOPDS_SubIterator.cxx Diff File
mod - src/BOPTest/BOPTest_BOPCommands.cxx Diff File
mod - src/BOPTest/BOPTest_Objects.cxx Diff File
mod - src/BOPTest/BOPTest_PartitionCommands.cxx Diff File
mod - src/IntTools/IntTools_Context.cxx Diff File
mod - tests/bugs/fclasses/bug7287_1 Diff File

Issue History

Date Modified Username Field Change
2015-08-04 17:33 srlockley New Issue
2015-08-04 17:33 srlockley Assigned To => msv
2015-08-04 18:11 msv Note Added: 0043821
2015-08-04 18:11 msv Assigned To msv => pkv
2015-08-04 18:11 msv Status new => assigned
2015-08-04 18:12 msv Summary Exponential memory usage problems with BOPDS_IndexedMapOfPaveBlockand NCollection_IncAllocator => Exponential memory usage problems with BOPDS_IndexedMapOfPaveBlock and NCollection_IncAllocator
2015-08-19 13:36 abv Target Version => 7.1.0
2015-08-25 12:38 msv Note Added: 0044597
2015-08-25 12:38 msv Assigned To pkv => srlockley
2015-08-25 12:38 msv Status assigned => feedback
2015-08-27 13:43 srlockley Note Added: 0044739
2015-08-27 14:48 msv Note Added: 0044745
2015-08-27 14:48 msv Assigned To srlockley => isn
2015-08-27 14:50 msv Note Added: 0044746
2015-08-27 14:50 msv Status feedback => assigned
2015-09-10 15:25 git Note Added: 0045425
2015-09-10 15:27 isn Note Added: 0045426
2015-09-10 15:27 isn Assigned To isn => msv
2015-09-10 15:27 isn Status assigned => resolved
2015-09-10 15:28 isn Note Edited: 0045426
2015-09-10 17:51 msv Note Added: 0045447
2015-09-10 17:51 msv Assigned To msv => isn
2015-09-10 17:51 msv Status resolved => assigned
2015-09-10 18:42 git Note Added: 0045453
2015-09-10 18:43 isn Assigned To isn => msv
2015-09-10 18:43 isn Status assigned => resolved
2015-09-11 11:15 msv Note Added: 0045468
2015-09-11 11:15 msv Assigned To msv => isn
2015-09-11 11:15 msv Status resolved => assigned
2015-09-11 11:34 git Note Added: 0045470
2015-09-11 11:35 isn Assigned To isn => msv
2015-09-11 11:35 isn Status assigned => resolved
2015-09-11 16:17 msv Note Added: 0045505
2015-09-11 16:17 msv Assigned To msv => bugmaster
2015-09-11 16:17 msv Status resolved => reviewed
2015-09-11 16:42 apv Assigned To bugmaster => apv
2015-09-11 17:43 git Note Added: 0045522
2015-09-11 17:43 apv Note Added: 0045524
2015-09-14 14:25 apv Note Added: 0045570
2015-09-14 14:25 apv Assigned To apv => isn
2015-09-14 14:25 apv Status reviewed => assigned
2015-09-14 14:57 msv Note Added: 0045580
2015-09-14 16:21 msv Assigned To isn => apv
2015-09-14 16:21 msv Status assigned => resolved
2015-09-14 16:21 msv Assigned To apv => bugmaster
2015-09-14 16:21 msv Status resolved => reviewed
2015-09-14 16:22 msv Assigned To bugmaster => apv
2015-09-14 16:22 msv Status reviewed => assigned
2015-09-15 14:00 git Note Added: 0045634
2015-09-15 14:02 abv Note Added: 0045635
2015-09-15 14:02 abv Status assigned => feedback
2015-09-15 14:02 abv Target Version 7.1.0 => 6.9.1
2015-09-15 17:10 apv Test case number => bugs modalg_6 bug26567
2015-09-15 17:42 git Note Added: 0045659
2015-09-15 17:44 apv Note Added: 0045660
2015-09-15 17:44 apv Assigned To apv => bugmaster
2015-09-15 17:44 apv Status feedback => tested
2015-09-18 13:31 bugmaster Changeset attached => occt master 488e5b9d
2015-09-18 13:31 bugmaster Status tested => verified
2015-09-18 13:31 bugmaster Resolution open => fixed
2015-10-16 14:55 aiv Status verified => closed
2015-10-16 16:37 git Note Added: 0046981
2015-10-23 20:50 aiv Fixed in Version => 6.9.1