View Issue Details

Related ChangesetsJump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0027710Open CASCADEOCCT:Testspublic2016-07-25 17:282019-07-13 14:31
ReporternbvAssigned Tobugmaster  
PrioritynormalSeverityminor 
Status closedResolutionfixed 
Target Version7.4.0Fixed in Version7.4.0 
Summary0027710: Make Data-exchange test cases more machine independent
DescriptionIn the current version of DE-test scripts (including product test cases such as Parasolid, SAT...), comparison with reference data (especially floating points values, such as TOLERANCE) is made with constant tolerance (e.g. 1.0e-7).

However, complex OCCT-algorithms often cannot provide this precision and result is often different on LINUX and WINDOWS platforms. At that, delta can be greater than 1.0e-7.

More over, if current shape tolerance (value got from the file) is ~ 1.0e-4 then the fluctuations in range 2...3e-7 seems to be OK.


My offer is following:

1. Enter new variable (e.g. MaxFloatingError).
2. Define this variable in all "begin" files with default value.
3. Redefine this variable in several test scripts (where cross-platform differences are detected) directly.
4. Use this variable (instead of constant value 1.0e-7) in all "end" files.

P.S.

This solution is considered to be implemented in both OCCT DE-test scripts and OCCT-products. It allows reducing number of variables such as "LinuxDiff" and "LinuxFaulties" (see de\step_4\B3 test case). At that, this-variables-mechanism is used in OCCT only (it is absence in OCCT-products).
Steps To ReproduceN/A
TagsNo tags attached.
Test case numberNot required

Activities

git

2016-09-05 17:42

administrator   ~0057472

Branch CR27710 has been created by apn.

SHA-1: 70760da0ad61f8467fc235de7d968a333252ab07


Detailed log of new commits:

Author: apn
Date: Mon Sep 5 17:44:05 2016 +0300

    0027710: Make Data-exchange test cases more machine independent
    
    Enter variable MaxFloatingTol to use it instead of constant value in end file.

apn

2016-09-05 17:57

administrator   ~0057473

Dear nbv, could you please review OCCT branch CR27710 and PRODUCTS branch CR27710_1.

nbv

2016-09-05 18:35

developer   ~0057479

Dear Sergei,

Please check, if all necessary grids are changed in CR27710_1 branch (for OCCT-products). Are there any lost grids?

TIA.

msv

2016-09-06 09:53

developer   ~0057491

I see that the implementation of the fix is incomplete, because the point 3 of the proposed steps has not been done:

3. Redefine this variable in several test scripts (where cross-platform differences are detected) directly.

It is needed to identify all non-stable test cases and set necessary value of MaxFloatingError there.

nbv

2016-09-06 10:49

developer   ~0057493

Dear Colleagues,

This feature is entered in order to avoid regressions connected with numerical instability by the reason of different floating-point-number representations on Windows and Linux platforms. I.e. it does not mean, that we must insert new variable in all test cases unthinkingly (in automatic mode). The reason of the difference between numbers must be investigated individually in every test case.

Especially, if these differences will be very big we shall understand the reason more carefully (enter the tolerance in this test case is wrong solution).

Dear Andrei,

According to above said, please make following:

1. Check all TODOs in unstable test cases. If this TODO refers to the real bug and this bug describes real reason of differences then we shall fix it (not to hide by the tolerance).

2. Please write something about made changes in User Guides and in commit messages. Otherwise, nobody will know about new feature.

git

2016-09-08 12:24

administrator   ~0057591

Branch CR27710 has been updated forcibly by apn.

SHA-1: d03be33f0d8f5926dd8e3bdd8584c2f44da3b79c

git

2016-09-08 15:18

administrator   ~0057594

Branch CR27710 has been updated by apn.

SHA-1: a807bfecbe5a7aa39d83122f1ad796ad6ce1dcbe


Detailed log of new commits:

Author: apn
Date: Thu Sep 8 15:20:43 2016 +0300

    Added description of MaxFloatingTol in user guides.

apn

2016-09-08 15:24

administrator   ~0057595

Dear nbv,

There are no unstable test cases is de group in occt.
In Products:
cr standard E1 - unstable test case.

Commit message was corrected. Description of MaxFloatingTol was added in user guides.

nbv

2016-09-08 15:31

developer   ~0057596

Dear Artem,

Please check, if all necessary grids are changed in CR27710_1 branch (for OCCT-products). Are there any lost grids (see message 0027710:0057479)?

TIA.

anv

2016-09-08 15:49

developer   ~0057598

Canonical Recognition is not part of Data Exchange. Thus it's tests doesn't need to be updated. All other grids are fine.

apn

2016-09-08 16:29

administrator   ~0057601

Dear anv, remarks were applied (changes in cr were reverted).

msv

2016-09-08 17:12

developer   ~0057605

I wonder what was the reason of this bug creation, if eventually no one test case was updated.

git

2019-07-11 09:28

administrator   ~0085582

Branch CR27710_1 has been created by abv.

SHA-1: f283875beef39a582f023ccc6f3de25d09cc4851


Detailed log of new commits:

Author: apn
Date: Thu Sep 8 15:20:43 2016 +0300

    0027710: Make Data-exchange test cases more machine independent
    
    Added description of de test grid in user guide.

git

2019-07-11 09:30

administrator   ~0085584

Branch CR27710 has been deleted by abv.

SHA-1: a807bfecbe5a7aa39d83122f1ad796ad6ce1dcbe

abv

2019-07-11 09:32

manager   ~0085585

I propose integrating branch CR27710_1 containing only change in documentation (added description of test group "de"). Other changes made within this issue do not seem to have any effect. The problems in tests should be addressed in more specific way.

msv

2019-07-11 10:31

developer   ~0085589

OK.

git

2019-07-13 14:31

administrator   ~0085618

Branch CR27710_1 has been deleted by inv.

SHA-1: f283875beef39a582f023ccc6f3de25d09cc4851

Related Changesets

occt: master 943b4e5b

2016-09-08 12:20:43

apn


Committer: bugmaster Details Diff		 0027710: Make Data-exchange test cases more machine independent

Added description of de test grid in user guide.		 Affected Issues
0027710
mod - dox/dev_guides/tests/tests.md Diff File

Issue History

Date Modified Username Field Change
2016-07-25 17:28 nbv New Issue
2016-07-25 17:28 nbv Assigned To => apn
2016-07-25 17:29 nbv Description Updated
2016-09-05 17:42 git Note Added: 0057472
2016-09-05 17:57 apn Note Added: 0057473
2016-09-05 18:21 apn Assigned To apn => nbv
2016-09-05 18:21 apn Status new => resolved
2016-09-05 18:35 nbv Note Added: 0057479
2016-09-05 18:35 nbv Assigned To nbv => szv
2016-09-05 18:35 nbv Status resolved => feedback
2016-09-06 09:53 msv Note Added: 0057491
2016-09-06 09:53 msv Assigned To szv => apn
2016-09-06 09:53 msv Status feedback => assigned
2016-09-06 09:58 nbv Assigned To apn => nbv
2016-09-06 09:58 nbv Status assigned => feedback
2016-09-06 10:49 nbv Note Added: 0057493
2016-09-06 10:49 nbv Assigned To nbv => apn
2016-09-06 10:49 nbv Status feedback => assigned
2016-09-08 12:24 git Note Added: 0057591
2016-09-08 15:18 git Note Added: 0057594
2016-09-08 15:24 apn Note Added: 0057595
2016-09-08 15:25 apn Assigned To apn => nbv
2016-09-08 15:25 apn Status assigned => feedback
2016-09-08 15:31 nbv Note Added: 0057596
2016-09-08 15:34 nbv Assigned To nbv => anv
2016-09-08 15:47 anv Assigned To anv => apn
2016-09-08 15:49 anv Note Added: 0057598
2016-09-08 15:49 anv Status feedback => assigned
2016-09-08 16:29 apn Note Added: 0057601
2016-09-08 16:30 apn Assigned To apn => anv
2016-09-08 16:30 apn Status assigned => resolved
2016-09-08 16:37 nbv Assigned To anv => nbv
2016-09-08 17:12 msv Note Added: 0057605
2016-09-08 17:59 nbv Assigned To nbv => apn
2016-09-08 17:59 nbv Status resolved => assigned
2016-11-07 14:08 apn Target Version 7.1.0 => 7.2.0
2017-07-24 10:34 bugmaster Target Version 7.2.0 => 7.3.0
2018-02-25 21:09 abv Target Version 7.3.0 => 7.4.0
2019-07-11 09:28 git Note Added: 0085582
2019-07-11 09:30 git Note Added: 0085584
2019-07-11 09:32 abv Note Added: 0085585
2019-07-11 09:32 abv Assigned To apn => msv
2019-07-11 09:32 abv Status assigned => resolved
2019-07-11 09:32 abv Steps to Reproduce Updated
2019-07-11 10:31 msv Note Added: 0085589
2019-07-11 10:31 msv Assigned To msv => bugmaster
2019-07-11 10:31 msv Status resolved => reviewed
2019-07-11 11:00 bugmaster Test case number => Not required
2019-07-11 11:06 bugmaster Status reviewed => tested
2019-07-13 14:17 bugmaster Changeset attached => occt master 943b4e5b
2019-07-13 14:17 bugmaster Status tested => verified
2019-07-13 14:17 bugmaster Resolution open => fixed
2019-07-13 14:31 git Note Added: 0085618