View Issue Details

IDProjectCategoryView StatusLast Update
0032769CommunityOCCT:Data Exchangepublic2024-03-22 15:10
Reporterrobertlipman Assigned Todpasukhi  
PrioritynormalSeveritymajor 
Status newResolutionopen 
Product Version7.6.0 
Target VersionUnscheduled 
Summary0032769: Data Exchange - Slow processing of very large STEP file
DescriptionI am testing our NIST STEP to X3D translator that is based on OCCT 7.6.0 with very large STEP files. The largest STEP file that I have is about 1.5 GB. It takes about 30-40 minutes to process in CAD Assistant and our software. However, in two other free viewers (C3D Viewer, eDrawings Viewer) it only takes about 5 minutes. Is there something to be improved to reduce the processing time? I think the slow part is actually reading the STEP file.

20:24 ...    Step File Reading : 'C:/LEGO Star Wars - UCS Millennium Falcon (75192).stp'      ...    STEP File   Read    ...
            ... Step File loaded  ...
         34810634 records (entities,sub-lists,scopes), 108692999 parameters      ... Parameters prepared ...
            ...   Objects analysed  ...
        STEP Loading done : 19047906 Entities
20:58 Count   STEP Reader Failures for 'LEGO Star Wars - UCS Millennium Falcon (75192).stp'
       -----   -----------
          23   F:CONTEXT_DEPENDENT_OVER_RIDING_STYLED_ITEM: Parameter n0.1 (style_context) : Entity has illegal type
           2   F:CONTEXT_DEPENDENT_OVER_RIDING_STYLED_ITEM: Parameter n0.2 (style_context) : Entity has illegal type
        7112   F:ITEM_DEFINED_TRANSFORMATION: Parameter n0.1 (name) not a quoted String
        7112   F:(REPRESENTATION_RELATIONSHIP,REPRESENTATION_RELATIONSHIP_WITH_TRANSFORMATION,SHAPE_REPRESENTATION_RELATIONSHIP): Parameter n0.1 (name) not a quoted String
        7112   F:PRODUCT_DEFINITION_SHAPE: Parameter n0.1 (property_definition.name) not a quoted String
          Nb Total:21361  for 5 items
20:58 Count   STEP Translator Warnings for 'LEGO Star Wars - UCS Millennium Falcon (75192).stp'
       -----   -----------
       12893   W:EDGE_LOOP: Edges were intersecting, corrected
          26   W:EDGE_LOOP: Wire was split on several wires
          25   W:(BOUNDED_SURFACE,B_SPLINE_SURFACE,B_SPLINE_SURFACE_WITH_KNOTS,GEOMETRIC_REPRESENTATION_ITEM,RATIONAL_B_SPLINE_SURFACE,REPRESENTATION_ITEM,SURFACE): Surface forced to be periodic
         323   W:B_SPLINE_SURFACE_WITH_KNOTS: Surface forced to be periodic
          Nb Total:13267  for 4 items
20:58 File C:/LEGO Star Wars - UCS Millennium Falcon (75192).stp
      [loaded in 383.172 s]
      [prepared in 1929.96 s]
21:00 [brepmesh Deflection:  3.2828 mm (rel.: 0.001) Angle: 20° Parallel: 1 MinSize: 0.1 mm InternalVerticesMode: 1 ControlSurfaceDeflection: 1]
21:04 [presentation computed in 309.362 s]
21:05 Frame is redrawn in 3253.62 ms
      [5213 groups] [7491 parts] [2913131 faces]
      [65601082 triangles] [average per part: 3780] [average per group: 6236]
        Private memory:     9623 MiB
        Working Set:        6945 MiB (peak: 11708 MiB)
        Pagefile usage:     9623 MiB (peak: 12543 MiB)
        Virtual memory:     16494 MiB
        Heap memory:     5255 MiB
Steps To ReproduceThe very large STEP file is available here https://grabcad.com/library/lego-star-wars-ucs-millennium-falcon-75192-1 The author has many other Lego models, some that are also very large but not as large as this one.

TagsNo tags attached.
Test case number

Activities

robertlipman

2022-01-12 23:24

reporter   ~0106323

For the "STEP Reader Failures", they are all related to strings that are either two single quotes '' or a dollar sign $. In both cases they are meant to indicate that the attribute is a blank string. I would not consider them failures nor anything to be reported.

On my computer, both eDrawings Viewer and C3D Viewer displayed the STEP file in about 4 minutes. CAD Assistant took 39 minutes.

Any improvement in processing would be great.

git

2022-11-28 15:54

administrator   ~0112363

Branch CR32769 has been created by ona.

SHA-1: 56fb4e879ebdf8bea96f627e68884ef8dd20b941


Detailed log of new commits:

Author: ona
Date: Mon Nov 28 12:27:51 2022 +0300

    0032769: Data Exchange - Slow processing of very large STEP file
    
    - Set on optimized memory manager by default

ona

2022-11-28 16:16

developer   ~0112365

Last edited: 2022-11-29 12:46

Most of the time was spent on pcurves computing (ShapeFix_Edge::FixAddPCurve function), where in turn a lot of time is spent on memory allocation in Extrema_GenExtPS class.
Memory allocation works much faster if the optimized memory manager is used. In my machine it takes 12 min instead of 35 min with the standard raw memory manager.
If the optimized memory manager is set globally some regressions are observed:
- performance is decreased
- memory is not freed immediately
So, the optimized memory manager should be used in particular cases with parameters chosen specifically.
Refer https://dev.opencascade.org/doc/overview/html/occt_user_guides__foundation_classes.html#occt_fcug_2_3 for details.

dpasukhi

2023-08-08 11:45

administrator   ~0113922

Optimized Memory manager is deprecated.
I really do not recommed to use this manager.

robertlipman

2024-03-20 05:20

reporter   ~0115425

With the faster STEP file processing in OCCT 7.8.0 this is no longer an issue. The 1.4 GB STEP file is now processed in 10 minutes which is an acceptable amount of time for a file that large.

dpasukhi

2024-03-22 15:10

administrator   ~0115440

We are working on new optimizations of reading and writing STEP processes based on reorganization of architecture and optimising logic.
Additionally we are working on multithreading processing.
All of that features planned to 7.9.

In the next month I will prepare a annonce of ongoing features.
I hope the file will have more options to read in faster way.

Issue History

Date Modified Username Field Change
2022-01-11 20:10 robertlipman New Issue
2022-01-11 20:10 robertlipman Assigned To => gka
2022-01-11 21:33 kgv Description Updated
2022-01-11 21:33 kgv Steps to Reproduce Updated
2022-01-11 21:34 kgv Target Version => 7.7.0
2022-01-11 21:34 kgv Summary Slow processing of very large STEP file => Data Exchange - Slow processing of very large STEP file
2022-01-12 23:24 robertlipman Note Added: 0106323
2022-10-24 10:38 szy Target Version 7.7.0 => 7.8.0
2022-11-15 09:49 gka Assigned To gka => robertlipman
2022-11-15 09:49 gka Assigned To robertlipman => ona
2022-11-28 15:54 git Note Added: 0112363
2022-11-28 16:16 ona Note Added: 0112365
2022-11-29 12:46 ona Note Edited: 0112365
2022-11-29 13:52 ona Assigned To ona => gka
2023-08-01 15:06 dpasukhi Target Version 7.8.0 => Unscheduled
2023-08-08 11:38 dpasukhi Assigned To gka => dpasukhi
2023-08-08 11:45 dpasukhi Note Added: 0113922
2024-03-20 05:20 robertlipman Note Added: 0115425
2024-03-22 15:10 dpasukhi Note Added: 0115440