MantisBT
Mantis Bug Tracker Workflow

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0023979Open CASCADE[OCCT] OCCT:Data Exchangepublic2013-05-21 16:572018-06-09 18:19
Reportersdv 
Assigned Togka 
PrioritynormalSeverityminor 
StatusassignedResolutionopen 
PlatformOSOS Version
Product Version 
Target Version[OCCT] 7.4.0*Fixed in Version 
Summary0023979: Performance problem of STEP export for large files
DescriptionIn order to reproduce the bug you can use attached scripts. Scripts t1, t3, t40, t60, t200 load from BRep file 1000, 3000... 200000 prisms, convert them to STEP format and save the result.
Analyzing convertation with Intel VTune Amplifier I have not found significant hotspots (except hotspot described in the issue 23974). Also I have analyzed convertation with using CallCount. Comparing logs for 1000, 3000 and more shapes, does not detect unlinear dependencies.
If you run script and open Windows Task Manager, you will see reducing of the memory. When the memory will be completely allocated, all open applications become unresponsive.
Setting flag MMGT_OPT does not give visible improving of perfomance.
Steps To Reproducepload ALL
autodisplay 0

compound c

chrono h restart
set n 200
set m 500
for {set i 0} {$i < $n} {incr i} {
  for {set j 0} {$j < $m} {incr j} {
    set xbase [expr $i*10]
    set ybase [expr $j*10]
    set size 5
    vertex v1 $xbase $ybase 0
    vertex v2 [expr $xbase+$size] $ybase 0
    vertex v3 [expr $xbase+0.5*$size] [expr $ybase+0.866*$size] 0
    edge e1 v1 v2
    edge e2 v2 v3
    edge e3 v3 v1
    wire w e1 e2 e3
    mkplane f w
    prism p f 0 0 10
    add p c
  }
}
chrono h stop counter buildshape

newmodel
chrono h restart
stepwrite 0 c 100K.step
chrono h stop counter stepwrite

meminfo
TagsNo tags attached.
Test case number
Attached Files7z file icon T1000.7z (78,797 bytes) 2013-05-21 16:57
7z file icon T3000.7z (276,987 bytes) 2013-05-21 17:01
7z file icon T40000.7z (3,507,025 bytes) 2013-05-21 17:03
7z file icon T60000.7z (4,643,853 bytes) 2013-05-21 17:04
? file icon compound_of_prisms.tcl (475 bytes) 2013-05-22 19:58
txt file icon OCC23979.txt (1,740 bytes) 2014-11-19 11:08
? file icon bug23979 (311 bytes) 2014-11-19 11:08

- Relationships
related to 0023974verifiedbugmaster Optimization of convertation to STEP. 
related to 0029830verifiedbugmaster Data Exchange, STEPCAFControl_Reader poor performance - quadratic dependence 

-  Notes
(0024453)
sdv (developer)
2013-05-21 17:48

In order to minimize secondary information of profiling with Intel VTune Amplifier, I reccomend to wrap the target function by methods __itt_resume() and __itt_pause().
Also it is important to specify settings of Amplifier project.
Start new analysis -> Project properties -> Advanced
For long-tme target is necessary increase "Duration time estimate" and "Collection data limit", otherwise result of profiling may be broken.
(0024454)
sdv (developer)
2013-05-21 18:17

Also I have use "Intel Advisor" to find out critical loops.
In my opinion it would be useful to analyse possibility to make parallel loop
in the STEPControl_ActorWrite::TransferCompound

for ( i = 1; i <= nbs; i ++) {
    Handle(TransferBRep_ShapeMapper) subs = TransferBRep::ShapeMapper (FP,RepItemSeq->Value(i));
    Handle(StepGeom_Axis2Placement3d) AX1;
    
    Handle(Transfer_Binder) bnd = TransferSubShape(subs, SDR0, AX1, FP, NonManifoldGroup, isManifold);

    if (!AX1.IsNull()) ItemSeq->Append (AX1);
    // copy binders so as to have all roots in upper binder, but do not conflict
    while ( !bnd.IsNull() ) {
      Handle(Transfer_SimpleBinderOfTransient) bx =
        Handle(Transfer_SimpleBinderOfTransient)::DownCast(bnd);
      if ( !bx.IsNull() )
        // Single SDR is created for a non-manifold group (ssv: 12.11.2010)
        if (!isManifold && i > 1)
          break;
        else {
          Handle(Transfer_Binder) aResult = TransientResult( bx->Result() );
          mapOfResultToAdd.Add( aResult );
        }
      bnd = bnd->NextResult();
    }
  }
(0024455)
sdv (developer)
2013-05-21 18:22

Reminder sent to: vro

Dear vro,
Could you attached snippet of code for generation BRep files which was used for convertation into STEP.

Best regards,
Sergey
(0024468)
vro (developer)
2013-05-22 15:38

Private property, sorry.
(0024469)
vro (developer)
2013-05-22 16:00

The test-model represents a set of identical shapes: prisms extruded from triangles. The triangles are the same and differ only by the location. It looks like a chess-field filled by the prisms in. The number of prisms varies up to 200 000. Probably a draw-script might generate the test-model, what do you think?
(0024475)
san (developer)
2013-05-22 19:54

compound_of_prisms.tcl that generates the compound of 200K prisms is attached.
(0024508)
san (developer)
2013-05-24 14:42

DRAW reproducer script is inefficient, it takes more than a whole day for it to complete. Thus a dedicated DRAW command should be created that builds the compound of prisms as illustrated by compound_of_prisms.tcl but fully with C++.
(0027696)
bugmaster (administrator)
2014-01-29 17:29

Test case is required.
(0034502)
ski (developer)
2014-11-19 11:06

QA command OCC23979 and test case bug23979 were attached to this issue.
DRAW command build the compound of 200k prism, but it leads to out of memory (4GB RAM is not enough for execution of this command).
(0076698)
msv (developer)
2018-06-09 15:22
edited on: 2018-06-09 15:23

The script compound_of_prisms.tcl still can be used efficiently if we disable autodisplay feature in draw:
autodisplay 0

Then this script takes 67 s to create the shape.

(0076700)
msv (developer)
2018-06-09 16:05

With the patch for 0029830 writing to STEP of the shape got by this script takes 514 s. However the process ended with failure (malloc), and the result file has zero size. The error message is:

 Step File Name : 200K.step(55000020 ents)     ****    Interruption SendAll par Exception :   ****
Standard_MMgrRaw::Allocate(): malloc failed
    Abandon
Error: File 200K.step written with fail messages


During run, the application took about 28 Gb memory.
(0076702)
msv (developer)
2018-06-09 16:17

With the model reduced down to 100K prisms it completed OK. The time is 265 s. The pick occupied memory is 15 Gb. The produced STEP file has the size 1.5 Gb.
(0076709)
msv (developer)
2018-06-09 18:13

I have tried to perfom the same on current master. The process has taken 2 hours and it seems it did not reach yet even a half of the progress (it took only 2.6 Gb memory so far). Looking at its stack I have found that the hot spot is the method Transfer_Binder::AddResult(). This hot spot has been eliminated in the patch for 29830.
(0076710)
msv (developer)
2018-06-09 18:17

So, the remaining problem is probably too much high memory eating.
It is needed to investigate if this task can be done with less memory requirements.
(0076711)
msv (developer)
2018-06-09 18:18

I have put the improved script in steps to reproduce.

- Issue History
Date Modified Username Field Change
2013-05-21 16:57 sdv New Issue
2013-05-21 16:57 sdv Assigned To => gka
2013-05-21 16:57 sdv File Added: T1000.7z
2013-05-21 17:01 sdv File Added: T3000.7z
2013-05-21 17:03 sdv File Added: T40000.7z
2013-05-21 17:04 sdv File Added: T60000.7z
2013-05-21 17:48 sdv Note Added: 0024453
2013-05-21 17:53 sdv Relationship added related to 0023974
2013-05-21 18:17 sdv Note Added: 0024454
2013-05-21 18:22 sdv Note Added: 0024455
2013-05-22 15:38 vro Note Added: 0024468
2013-05-22 16:00 vro Note Added: 0024469
2013-05-22 19:54 san File Added: compound_of_prisms.tcl
2013-05-22 19:54 san Note Added: 0024475
2013-05-22 19:58 san File Deleted: compound_of_prisms.tcl
2013-05-22 19:58 san File Added: compound_of_prisms.tcl
2013-05-24 14:42 san Note Added: 0024508
2014-01-29 17:29 bugmaster Note Added: 0027696
2014-01-29 17:29 bugmaster Assigned To gka => apn
2014-01-29 17:29 bugmaster Status new => assigned
2014-11-19 11:06 ski Note Added: 0034502
2014-11-19 11:08 ski File Added: OCC23979.txt
2014-11-19 11:08 ski File Added: bug23979
2014-11-19 11:08 ski Assigned To apn => gka
2014-11-19 11:26 abv Target Version => 7.1.0
2016-10-26 12:18 gka Target Version 7.1.0 => 7.2.0
2017-07-27 09:33 abv Target Version 7.2.0 => 7.4.0*
2018-06-09 12:48 abv Assigned To gka => msv
2018-06-09 15:22 msv Note Added: 0076698
2018-06-09 15:23 msv Note Edited: 0076698 View Revisions
2018-06-09 16:05 msv Note Added: 0076700
2018-06-09 16:06 msv Relationship added related to 0029830
2018-06-09 16:17 msv Note Added: 0076702
2018-06-09 18:13 msv Note Added: 0076709
2018-06-09 18:17 msv Note Added: 0076710
2018-06-09 18:18 msv Note Added: 0076711
2018-06-09 18:18 msv Steps to Reproduce Updated View Revisions
2018-06-09 18:19 msv Steps to Reproduce Updated View Revisions
2018-06-09 18:19 msv Assigned To msv => gka


Copyright © 2000 - 2018 MantisBT Team
Powered by Mantis Bugtracker