MantisBT
Mantis Bug Tracker Workflow

View Issue Details Jump to Notes ] Related Changesets ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0032095Open CASCADE[OCCT] OCCT:Testspublic2021-01-29 10:122021-02-21 11:44
Reporterkgv 
Assigned Tobugmaster 
PrioritynormalSeverityminor 
StatusverifiedResolutionfixed 
PlatformOSOS Version
Product Version[OCCT] 7.5.0 
Target Version[OCCT] 7.6.0*Fixed in Version 
Summary0032095: Tests - drop comparison in unstable perf/ncollection group
DescriptionTest case perf/ncollection/A1 regularly fails due to instabilities in test system.
By it's nature test case was first created to compare performance of similar STL and OCCT collections in common operations and then collected numbers have been put into test in attempt to detect possible performance degradation in OCCT collections.

Obviously, as there are many different STL implementations (at least one per compiler), these implementation evolve and debug/release/optimization flags dramatically affect performance, it is practically impossible to embed meaningful numbers into test case itself. Several attempts have been already tried to adjust numbers and extend comparison tolerance, but test grid remains unstable on Jenkins.

Therefore, it is proposed to not report test case failures and just report deviations as warnings.
Steps To Reproduceperf/ncollection/A1
TagsNo tags attached.
Test case numberNot required
Attached Files

- Relationships
child of 0023880closedbugmaster Integration of grid "ncl" into the new testing system 

-  Notes
(0098499)
git (administrator)
2021-01-29 10:23

Branch CR32095 has been created by kgv.

SHA-1: 96037b1bd51a5445a29a333e990c157a6efef483


Detailed log of new commits:

Author: kgv
Date: Fri Jan 29 10:24:36 2021 +0300

    0032095: Tests - drop comparison in unstable perf/ncollection group
    
    Mark performance deviations compared to STL collections as warnings instead of errors.
(0098502)
git (administrator)
2021-01-29 11:00

Branch CR32095 has been updated forcibly by kgv.

SHA-1: 8ce4f205c57cc265628a7578b90afcc23fe1cead
(0098503)
git (administrator)
2021-01-29 11:02

Branch CR32095 has been updated forcibly by kgv.

SHA-1: 3d098e54bd3622dae3e7a7d3bda0e50af46085c2
(0098504)
kgv (developer)
2021-01-29 11:03

Proposed patch is ready for review (putting these failures to IGNORE rule)
- OCCT branch: CR32095.
(0098780)
abv (manager)
2021-02-08 19:54

No remarks, please integrate
(0098859)
bugmaster (administrator)
2021-02-13 11:10

Combination -
OCCT branch : IR-2021-02-12
master SHA - fb68dbdbd6810ee27f9398c749c5740b7bea7818
a87b7ddc8cb44606b91e3f37113847c3f5f50fdc
Products branch : IR-2021-02-12 SHA - 94205f40e685a70c77e547147d90c09e0a21c461
was compiled on Linux, MacOS and Windows platforms and tested in optimize mode.

Number of compiler warnings:
No new/fixed warnings

Regressions/Differences/Improvements:
No regressions/differences

CPU differences:
Debian80-64:
OCCT
Total CPU difference: 17805.430000000237 / 17802.420000000013 [+0.02%]
Products
Total CPU difference: 11513.250000000124 / 11523.070000000105 [-0.09%]
Windows-64-VC14:
OCCT
Total CPU difference: 19375.296875 / 19372.265625 [+0.02%]
Products
Total CPU difference: 12857.265625 / 12882.921875 [-0.20%]


Image differences :
No differences that require special attention

Memory differences :
No differences that require special attention
(0099047)
git (administrator)
2021-02-21 11:44

Branch CR32095 has been deleted by inv.

SHA-1: 3d098e54bd3622dae3e7a7d3bda0e50af46085c2

- Related Changesets
occt: master 9779ff75
Timestamp: 2021-01-29 07:24:36
Author: kgv
Committer: bugmaster
Details ] Diff ]
0032095: Tests - drop comparison in unstable perf/ncollection group

Mark performance deviations compared to STL collections as warnings instead of errors.
mod - tests/perf/ncollection/A2 Diff ] File ]
mod - tests/perf/ncollection/A3 Diff ] File ]
mod - tests/perf/parse.rules Diff ] File ]

- Issue History
Date Modified Username Field Change
2021-01-29 10:12 kgv New Issue
2021-01-29 10:12 kgv Assigned To => kgv
2021-01-29 10:13 kgv Relationship added child of 0023880
2021-01-29 10:23 git Note Added: 0098499
2021-01-29 11:00 git Note Added: 0098502
2021-01-29 11:02 git Note Added: 0098503
2021-01-29 11:03 kgv Note Added: 0098504
2021-01-29 11:03 kgv Assigned To kgv => abv
2021-01-29 11:03 kgv Status new => resolved
2021-02-08 19:54 abv Note Added: 0098780
2021-02-08 19:54 abv Assigned To abv => bugmaster
2021-02-08 19:54 abv Status resolved => reviewed
2021-02-13 11:10 bugmaster Note Added: 0098859
2021-02-13 11:10 bugmaster Status reviewed => tested
2021-02-13 11:13 bugmaster Test case number => Not required
2021-02-13 11:29 bugmaster Changeset attached => occt master 9779ff75
2021-02-13 11:29 bugmaster Status tested => verified
2021-02-13 11:29 bugmaster Resolution open => fixed
2021-02-21 11:44 git Note Added: 0099047


Copyright © 2000 - 2021 MantisBT Team
Powered by Mantis Bugtracker