View Issue Details

Related ChangesetsJump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0022627Open CASCADEOCCT:Modeling Algorithmspublic2011-06-24 11:372013-05-24 12:37
Reporterkgv Assigned Tobugmaster  
PriorityhighSeverityfeature 
Status closedResolutionfixed 
OSAll 
Target Version6.5.3Fixed in Version6.5.3 
Summary0022627: Change OCCT memory management defaults
DescriptionCurrently default settings are:
  MMGT_OPT = 1;
  MMGT_REENTRANT = 0;
This corresponds to using of advanced OCCT memory manager Standard_MMgrOpt in
thread-unsafe manner.

There several reasons why current defaults are bad:
  - most mordern applications are multi-threaded and using thread-unsafe
allocator will cause crashes forcing application to change defaults (however
developer should found these settings - root cause of problems - first time he
detect them and this is untrivial);
  - OS changed since Standard_MMgrOpt was firstly introduced; todays built-in
system memory allocators already use same techniques (probably configured more
optimally for mordern systems) and even more advanced (optimized for multi-
threading applications).

So there are no reasons to use limited advanced memory manager in OCCT and if
user really need it - he should choose it consciously.

Thus suggested changes:
  - change MMGT_OPT to 0 (will use Standard_MMgrRaw - wrapper over standard
memory allocators);
  - change MMGT_REENTRANT to 1 (to make Standard_MMgrOpt thread-safe by
default) or totally remove this parameter (always thread-safe).

These changes require performance tests on mordern systems and on some
existing single-threaded applications to make a decision.
TagsNo tags attached.
Test case numberno test case

Attached Files

  • chl_928_V8_mmgtopt0.txt (146,042 bytes)
  • LP2-TestMemAlloc.xls (23,040 bytes)

Relationships

related to 0022734 closeddbv Community Memory allocation error in OpenGl 
related to 0022819 closedbugmaster Open CASCADE Redesign of OpenGl driver 
related to 0023032 closedbugmaster Open CASCADE A crash of a draw-command voxeloctboolds on MMGT_OPT=0, MMGT_REENTRANT=1 
related to 0023988 closedszv Open CASCADE Force use of reentrant mode 

Activities

abv

2011-09-05 09:18

manager   ~0018070

See discussion on forum thread http://www.opencascade.org/org/forum/thread_21696 leading to the same conclusion

abv

2011-09-13 09:33

manager   ~0018106

Dear Kirill,

Please prepare patch (to be tested and integrated).

kgv

2011-09-13 12:48

developer   ~0018115

Branch http://svn/svn/occt/branches/OCC22627 was created for this BUG.

Current changes applied:
 - MMGT_OPT default value changed to 0 (to use Standard_MMgrRaw - wrapper over standard memory allocators);
 - MMGT_REENTRANT default value changed to 1 (to make Standard_MMgrOpt thread-safe by default).

abv

2011-09-13 13:06

manager   ~0018116

Some minor remarks (stylistic):

- Parameters bMMap, aCellSize, aNbPages, aThreshold are needed only if MMGT_OPT=1; hence they can be moved to relevant case of the switch

- It would be reasonable to modify Standard_Reentrant only in the case if MMGT_REENTRANT is actually defined; furthermore it is better to use conditional statement instead of direct assignment (now it is possible to set Reentrant to e.g. -5).

kgv

2011-09-13 13:30

developer   ~0018118

Remarks applied

abv

2011-09-13 13:39

manager   ~0018119

Reviewed without remarks.

Please test ASAP as we expect to have "regressions" due to possible presence of bugs previously obscured by latency of memory release by OCCT memory manager

mkv

2011-09-15 12:48

tester   ~0018160

Dear BugMaster,
Workbench KAS:dev:mkv-OCC22627-occt has been created from SVN branch http://svn/svn/occt/branches/OCC22627
(and mkv-OCC22627-products from trunk) and compiled on Linux platform.

There are folloving regressions in mkv-OCC22627-products regarding to KAS:dev:products-12092011-opt
chl 901 A8 - *** Abort *** an exception was raised, but no catch was found.
chl 927 V8 - *** Abort *** an exception was raised, but no catch was found.
chl 928 O5 - *** Abort *** an exception was raised, but no catch was found.
chl 928 R1 - memory leakage
chl 929 E2 - *** Abort *** an exception was raised, but no catch was found.
chl 937 C5 - *** Abort *** an exception was raised, but no catch was found.
chl 941 A3 - *** Abort *** an exception was raised, but no catch was found.

See results in /dn45/occttests/results/KAS/dev/mkv-OCC22627-products_13092011/lin
See reference results in /dn45/occttests/results/KAS/dev/products-20110909-opt_12092011/lin
See test cases in /dn45/occttests/tests/ED
N.B. In order to launch testing case you can make use the following instructions
http://doc/doku.php?id=occt:certification

kgv

2011-09-21 11:15

developer   ~0018236

2apl,

could you please look onto this issue?

I found that test case 'chl 901 A8' raise segmentation violation after #22650 patch (revision 8920 in TRUNK). Notice that this happens on DRAWEXE exiting (thus probably caused problems in static memory management).

To reproduce this you needn't compile branch for this bug - just change MMGT_OPT to 0 before launching DRAWEXE (set env(QA_DUMP) 1; set env(MMGT_OPT) 0; wokenv -s;).

apl

2011-09-22 18:15

developer   ~0018268

Hi Kirill,

I've corrected problem with exceptions in graphic3d_group. However, the following cases raises exception anyway (even without patch 22650):

927/V8
937/C5
941/A3

The exceptions (which was corrected) were raised on update of Graphic3d_StructureManager by Graphic3d_Group before destruction. The pointer used for update operation is stored in Graphic3d_Structure and it isn't a handle to avoid cross-reference on destruction.

The possible reason is that the V3d_Viewer keeps a handles to a graphical objects as a class fields, and a handle to structure manager. If the order of destruction is incorrect (structure manager destroyed first) then these objects calls invalid pointers on destruction. However, my newly integrated corrections to graphic3d_group fixes a mistake in graphic3d_structure::remove operation that due to this behavior leads to exception with MMGT_OPT=0.

kgv

2011-09-26 10:29

developer   ~0018275

Last edited: 2011-09-27 11:48

 927/V8 exception is a result of early memory corruption which cause ~Draw_Viewer() destructor for one global instance called twice. This happens either when MMGT_OPT=0 and MMGT_OPT=1 but in second case double destruction passes silently due to laziness of OCCT optimized memory manager.

The original corruption seems to be happens in TKOpenGl module and could be debugged using gdb on Linux platform (chl_928_V8_mmgtopt0.txt in attach is a valgrind log).

The bug 0022734 may be root cause of this.

kgv

2011-09-26 10:30

developer  

chl_928_V8_mmgtopt0.txt (146,042 bytes)

abv

2011-12-26 14:28

manager   ~0019002

I have merged the code with current trunk and integrated one more change: activation of low-fragmentation heap on Windows, which allows to get best performance of standard memory manager on Windows XP and 2000. Environment variable MMGT_LFH can be used to avoid this setting (if set to 0).

I suggest this fix is re-tested in its current state. Please review the change

kgv

2011-12-26 14:56

developer   ~0019003

2abv
> #if(defined(_WIN32) || defined(__WIN32__))
> ...
> HANDLE aCRTHeap = (HANDLE)_get_heap_handle();
> #endif
wrong macro used to determine _get_heap_handle() availability.
Please test for MSVC compiler instead (mingw headers doesn't declare this function despite it declare it crt library)
OR declare this function in our sources like "_CRTIMP intptr_t __cdecl _get_heap_handle(void);"

> if ( ! aVar || atoi (aVar) != 0 )
Please use (aVar == NULL) check to improve readability...

abv

2011-12-26 15:12

manager   ~0019004

Code corrected, please check

kgv

2011-12-26 15:18

developer   ~0019006

Patch is OK now.
Dear bugmaster, please perform tests on this branch.

abv

2011-12-26 20:18

manager  

LP2-TestMemAlloc.xls (23,040 bytes)

abv

2011-12-26 20:25

manager   ~0019009

Attached file LP2-TestMemAlloc.xls provides results of testing of performance of importing big STEP file (184 Mb) in the customer's application with different memory options. It demonstrates that activation of low-fragmentation heap on Windows XP makes standard-memory manager even more performant than those of OCCT and TBB; on Windows 7 it is only 10% slower than TBB. At the same time, standard memory manager is almost twice less memory hungry as TBB!

mkv

2011-12-28 20:08

tester   ~0019042

Dear BugMaster,
Workbench KAS:dev:mkv-OCC22627-occt has been created from SVN branch http://svn/svn/occt/branches/OCC22627
(and mkv-OCC22627-products from trunk) and compiled on Linux platform.

There are folloving regressions in mkv-OCC22627-products regarding to KAS:dev:products-20111216-opt
chl 927 V8
chl 928 O5
chl 928 R1
chl 937 C5
chl 941 A3

See results in /QADisk/occttests/results/KAS/dev/mkv-OCC22627-products_27122011/lin
See reference results in /QADisk/occttests/results/KAS/dev/products-20111216-opt_17122011/lin
See test cases in /QADisk/occttests/tests/ED
N.B. In order to launch testing case you can make use the following instructions
http://doc/doku.php?id=occt:certification

abv

2011-12-29 12:05

manager   ~0019044

Dear Sergey,
Please check reported regressions chl 928 R1 and O5

abv

2012-02-20 16:05

manager   ~0019670

Please re-test on current version: most of the regressions should have been fixed by recent integrations. The fix is on the branch with the same name (OCC22627), rebased on current trunk

aan

2012-02-23 18:21

developer   ~0019757

Last edited: 2012-02-23 18:24

 Dear BugMaster,
Workbench KAS:dev:aan-22627-occt from SVN branch http://svn/svn/occt/branches/OCC22627
(and aan-22627-products from trunk) was successfully compiled and tested on Linux.

Workbench KAS:dev:aan-22627-products was tested with MMGT_OPT=1 and MMGT_REENTRANT=0

There are not regressions in aan-22627-products regarding to KAS:dev:products-20120217-opt

See results in /QADisk/occttests/results/KAS/dev/aan-22627-products_21022012/lin
See reference results in /QADisk/occttests/results/KAS/dev/products-20120217-opt_17022012/lin
See test cases in /QADisk/occttests/tests/ED
N.B. In order to launch testing case you can make use the following instructions
http://doc/doku.php?id=occt:certification

aan

2012-02-23 18:26

developer   ~0019758

Dear BugMaster,
Workbench KAS:dev:aan-22627-1-occt from SVN branch http://svn/svn/occt/branches/OCC22627
(and aan-22627-1-products from trunk) was successfully compiled and tested on Linux.

Workbench KAS:dev:aan-22627-1-products was tested with MMGT_OPT=0 and MMGT_REENTRANT=1

There are following regressions in aan-22627-1-products regarding to KAS:dev:products-20120217-opt

chl 928 R1
chl 936 G9

vpr 007 C1 C2 C3 C4 C5 C6 D1 D2 D3 D4 D5 D6
vpr 008 B2 B3 B5 C1 C2 C3 C4 C5 C6 D1 D2 D3 D4 D5 D6
vpr 0015 B1 B2 B3 B4 B5 B6 C1 C2 D1 D2
vpr 016 B1 B4 C1 C2 D1


See results in /QADisk/occttests/results/KAS/dev/aan-22627-1-products_21022012/lin
See reference results in /QADisk/occttests/results/KAS/dev/products-20120217-opt_17022012/lin
See test cases in /QADisk/occttests/tests/ED
N.B. In order to launch testing case you can make use the following instructions
http://doc/doku.php?id=occt:certification [^]

kgv

2012-02-24 10:16

developer   ~0019770

chl 928 R1 - aka memory leak
chl 936 G9 - exception, voxeloctboolds

Increased memory usage:
vpr 007 C1 C2 C3 C4 C6 - 1220 KB -> 1684 KB (limit 1308)
vpr 007 C5 - 1220 KB -> 1356 KB (limit 1348)
vpr 007 D1 - 1216 KB -> 1804 KB (limit 1324)
vpr 007 D2 - 1216 KB -> 2072 KB (limit 1336)
vpr 007 D3 - 1216 KB -> 1784 KB (limit 1320)
vpr 007 D4 - 1216 KB -> 1396 KB (limit 1324)
vpr 007 D5 - fit in limit
vpr 007 D6 - 2008 KB -> 2232 KB (limit 2168)
vpr 008 B2 - 1216 KB -> 2080 KB (limit 1360)
vpr 008 B3 - 1216 KB -> 1700 KB (limit 1372)
vpr 008 B5 - 1216 KB -> 1456 KB (limit 1356)
vpr 008 C1 - 1220 KB -> 2076 KB (limit 1380)
vpr 008 C2 - 1220 KB -> 2156 KB (limit 1384)
vpr 008 C3 - 1220 KB -> 2168 KB (limit 1376)
vpr 008 C4 - 1220 KB -> 2124 KB (limit 1376)
vpr 008 C5 - 1220 KB -> 2120 KB (limit 1376)
vpr 008 C6 - 1220 KB -> 2116 KB (limit 1372)
vpr 008 D1 - 1216 KB -> 2096 KB (limit 1404)
vpr 008 D2 - 1216 KB -> 2184 KB (limit 1404)
vpr 008 D3 - 1216 KB -> 2172 KB (limit 1368)
vpr 008 D4 - 1216 KB -> 2124 KB (limit 1392)
vpr 008 D5 - 2008 KB -> 2896 KB (limit 2240)
vpr 008 D6 - 2008 KB -> 2884 KB (limit 2236)
vpr 015 B1 - 1216 KB -> 1608 KB (limit 1348)
vpr 015 B2 - 1216 KB -> 1608 KB (limit 1360)
vpr 015 B3 - 1216 KB -> 1608 KB (limit 1360)
vpr 015 B4 - 1216 KB -> 1608 KB (limit 1344)
vpr 015 B5 - 1216 KB -> 1608 KB (limit 1352)
vpr 015 B6 - 1216 KB -> 1608 KB (limit 1356)
vpr 015 C1 - 1216 KB -> 1608 KB (limit 1360)
vpr 015 C2 - 1224 KB -> 2108 KB (limit 1364)
vpr 015 D1 - 1216 KB -> 1608 KB (limit 1356)
vpr 015 D2 - 1216 KB -> 2072 KB (limit 1360)
vpr 016 B1 - 1216 KB -> 1608 KB (limit 1372)
vpr 016 B4 - 1216 KB -> 1608 KB (limit 1364)
vpr 016 C1 - 1216 KB -> 1608 KB (limit 1264)
vpr 016 C2 - 1220 KB -> 2176 KB (limit 1324)
vpr 016 D1 - 1216 KB -> 1608 KB (limit 1272)
vpr 016 D2 - 1216 KB -> 2072 KB (limit 1320)

dbv

2012-03-21 11:14

developer   ~0020050

Looks like there is no regressions in chl 928 R1 test and all listed vpr tests.
For MMGT_OPT=0 tests should be differ, due to different memory manager and different memory usage.

Dear VRO,
Could you please take care about regression chl 936 G9

vro

2012-03-21 12:29

developer   ~0020056

The regression chl 936 G9 is pure Voxel mistake in the method Voxel_OctBoolDS::OptimizeMemory(). The issue 22627 may ignore this crash, which will be fixed by another issue.

vro

2012-03-21 12:36

developer   ~0020058

The issue number for the fix of chl 936 G9 is 0023032

abv

2012-03-21 12:38

manager   ~0020059

Last edited: 2012-03-21 14:05

 Please integrate (update chl 928 R1 and vpr test scripts for increased memory with MMGT_OPT=0 as needed)

apn

2012-03-21 18:20

administrator   ~0020076

Dear BugMaster,
     SVN branch http://svn/svn/occt/branches/OCC22627 has been moved to GIT CR22627 one.

bugmaster

2012-03-22 11:09

administrator   ~0020087

Integrated into master

Related Changesets

occt: master af09dbff

2012-03-21 14:10:41

kgv


Committer: bugmaster Details Diff		 0022627: Change OCCT memory management defaults Affected Issues
0022627
mod - src/Standard/Standard.cxx Diff File

Issue History

Date Modified Username Field Change
2011-08-02 11:31 bugmaster Category OCCT:MOA => OCCT:Modeling Algorithms
2011-09-05 09:18 abv Test case number => no test case
2011-09-05 09:18 abv Note Added: 0018070
2011-09-05 09:18 abv Assigned To bugmaster => kgv
2011-09-05 09:18 abv Status new => assigned
2011-09-05 09:18 abv Fixed in Version EMPTY =>
2011-09-05 09:18 abv Target Version => 6.5.2
2011-09-05 09:18 abv Description Updated
2011-09-13 09:33 abv Note Added: 0018106
2011-09-13 09:33 abv Priority normal => high
2011-09-13 12:48 kgv Note Added: 0018115
2011-09-13 12:48 kgv Status assigned => resolved
2011-09-13 12:48 kgv Assigned To kgv => abv
2011-09-13 13:06 abv Note Added: 0018116
2011-09-13 13:30 kgv Note Added: 0018118
2011-09-13 13:39 abv Note Added: 0018119
2011-09-13 13:39 abv Status resolved => reviewed
2011-09-15 12:48 mkv Note Added: 0018160
2011-09-15 12:49 mkv Assigned To abv => kgv
2011-09-15 12:49 mkv Status reviewed => assigned
2011-09-21 09:41 kgv Assigned To kgv => mkv
2011-09-21 09:41 kgv Status assigned => reviewed
2011-09-21 09:44 kgv Assigned To mkv => kgv
2011-09-21 11:08 kgv Assigned To kgv => apl
2011-09-21 11:15 kgv Note Added: 0018236
2011-09-22 18:13 apl Assigned To apl => kgv
2011-09-22 18:15 apl Note Added: 0018268
2011-09-23 11:13 mkv Status reviewed => assigned
2011-09-26 10:29 kgv Note Added: 0018275
2011-09-26 10:30 kgv File Added: chl_928_V8_mmgtopt0.txt
2011-09-26 10:31 kgv Note Edited: 0018275
2011-09-27 11:47 kgv Relationship added related to 0022734
2011-09-27 11:48 kgv Note Edited: 0018275
2011-12-26 11:25 san Relationship added related to 0022819
2011-12-26 14:28 abv Note Added: 0019002
2011-12-26 14:28 abv Status assigned => resolved
2011-12-26 14:56 kgv Note Added: 0019003
2011-12-26 15:12 abv Note Added: 0019004
2011-12-26 15:18 kgv Note Added: 0019006
2011-12-26 15:18 kgv Status resolved => reviewed
2011-12-26 20:18 abv File Added: LP2-TestMemAlloc.xls
2011-12-26 20:25 abv Note Added: 0019009
2011-12-28 20:08 mkv Note Added: 0019042
2011-12-28 20:08 mkv Status reviewed => assigned
2011-12-29 12:05 abv Assigned To kgv => skv
2011-12-29 12:05 abv Note Added: 0019044
2012-01-23 12:19 abv Assigned To skv => kgv
2012-02-20 16:04 abv Status assigned => resolved
2012-02-20 16:05 abv Note Added: 0019670
2012-02-20 16:15 abv Status resolved => reviewed
2012-02-20 17:16 mkv Assigned To kgv => aan
2012-02-23 18:21 aan Note Added: 0019757
2012-02-23 18:22 aan Note Edited: 0019757
2012-02-23 18:22 aan Note Edited: 0019757
2012-02-23 18:24 aan Note Edited: 0019757
2012-02-23 18:26 aan Note Added: 0019758
2012-02-23 18:27 aan Assigned To aan => kgv
2012-02-23 18:27 aan Status reviewed => assigned
2012-02-24 10:16 kgv Note Added: 0019770
2012-02-24 14:11 kgv Assigned To kgv => dbv
2012-03-21 11:14 dbv Note Added: 0020050
2012-03-21 11:14 dbv Assigned To dbv => vro
2012-03-21 12:27 vro Assigned To vro => kgv
2012-03-21 12:29 vro Note Added: 0020056
2012-03-21 12:36 vro Note Added: 0020058
2012-03-21 12:37 abv Status assigned => resolved
2012-03-21 12:38 abv Note Added: 0020059
2012-03-21 12:38 abv Status resolved => reviewed
2012-03-21 12:38 abv Status reviewed => tested
2012-03-21 12:40 abv Relationship added related to 0023032
2012-03-21 12:41 abv Note Edited: 0020059
2012-03-21 14:05 abv Note Edited: 0020059
2012-03-21 18:20 apn Note Added: 0020076
2012-03-22 11:01 bugmaster Target Version 6.5.2 => 6.5.3
2012-03-22 11:09 bugmaster Note Added: 0020087
2012-03-22 11:09 bugmaster Status tested => verified
2012-03-22 11:09 bugmaster Resolution open => fixed
2012-03-29 17:26 bugmaster Changeset attached => occt master af09dbff
2013-05-24 12:37 kgv Relationship added related to 0023988