MantisBT - Open CASCADE
View Issue Details
0031740Open CASCADE[OCCT] OCCT:Configurationpublic2020-08-29 07:452020-09-14 15:25
abv 
bugmaster 
normalminor 
verifiedfixed 
 
[OCCT] 7.5.0 
Not required
0031740: Configuration - recover support of Yacc and Lex generation
Support of Yacc and Lex generation in OCCT CMake scripts is incomplete:
- we have no ready-to-use package of Flex and Bison matching the generated files currently stored in master
- CMake script cannot find flex and bison automatically even when they are present in the system (on Linux) or in 3rd-party folder
- options used for calling Flex and Bison in CMake script differ from those used for generation of files currently stored in master

N/A
No tags attached.
related to 0027342verified bugmaster Open CASCADE Data Exchange, STEP - support C++ streams for import / export 
related to 0031481resolved abv Open CASCADE Data Exchange - provide parser of STEP EXPRESS schema for generation of new STEP entities 
Issue History
2020-08-29 07:45abvNew Issue
2020-08-29 07:45abvAssigned To => bugmaster
2020-08-29 07:46abvRelationship addedrelated to 0027342
2020-08-29 21:28abvNote Added: 0093722
2020-08-29 21:30gitNote Added: 0093723
2020-08-30 06:11gitNote Added: 0093724
2020-08-30 06:32abvNote Added: 0093725
2020-08-30 06:49abvNote Added: 0093726
2020-08-30 07:00abvRelationship addedrelated to 0031741
2020-08-30 18:58abvNote Added: 0093732
2020-08-30 18:58abvAssigned Tobugmaster => dpasukhi
2020-08-30 18:58abvStatusnew => resolved
2020-08-30 18:58abvSteps to Reproduce Updatedbug_revision_view_page.php?rev_id=23262#r23262
2020-08-30 18:59abvNote Edited: 0093732bug_revision_view_page.php?bugnote_id=93732#r23264
2020-08-30 21:57dpasukhiNote Added: 0093743
2020-08-30 21:57dpasukhiAssigned Todpasukhi => adv
2020-08-30 21:57dpasukhiStatusresolved => assigned
2020-08-30 21:58dpasukhiNote Edited: 0093743bug_revision_view_page.php?bugnote_id=93743#r23266
2020-08-30 22:00dpasukhiNote Edited: 0093743bug_revision_view_page.php?bugnote_id=93743#r23267
2020-08-30 22:01dpasukhiNote Edited: 0093743bug_revision_view_page.php?bugnote_id=93743#r23268
2020-08-31 09:43abvAssigned Toadv => abv
2020-08-31 10:16gitNote Added: 0093749
2020-08-31 10:33gitNote Added: 0093751
2020-08-31 10:35abvNote Added: 0093752
2020-08-31 10:35abvAssigned Toabv => bugmaster
2020-08-31 10:35abvStatusassigned => resolved
2020-08-31 10:35abvAssigned Tobugmaster => dpasukhi
2020-08-31 11:32dpasukhiNote Added: 0093756
2020-08-31 11:32dpasukhiStatusresolved => assigned
2020-08-31 12:11dpasukhiStatusassigned => resolved
2020-08-31 12:12dpasukhiNote Added: 0093757
2020-08-31 12:12dpasukhiAssigned Todpasukhi => abv
2020-08-31 12:12dpasukhiStatusresolved => reviewed
2020-08-31 16:00abvNote Added: 0093768
2020-08-31 16:10abvAssigned Toabv => bugmaster
2020-09-05 12:06bugmasterNote Added: 0094050
2020-09-05 12:06bugmasterStatusreviewed => tested
2020-09-05 12:16bugmasterTest case number => Not required
2020-09-05 12:25bugmasterChangeset attached => occt master fba34cf8
2020-09-05 12:25bugmasterStatustested => verified
2020-09-05 12:25bugmasterResolutionopen => fixed
2020-09-05 12:35gitNote Added: 0094093
2020-09-05 12:35gitNote Added: 0094094
2020-09-05 12:35gitNote Added: 0094096
2020-09-13 08:22abvRelationship addedrelated to 0031481
2020-09-14 15:24bugmasterStatusverified => closed
2020-09-14 15:24bugmasterAssigned Tobugmaster => isn
2020-09-14 15:24bugmasterStatusclosed => assigned
2020-09-14 15:25bugmasterAssigned Toisn => bugmaster
2020-09-14 15:25bugmasterStatusassigned => resolved
2020-09-14 15:25bugmasterStatusresolved => reviewed
2020-09-14 15:25bugmasterStatusreviewed => verified

Notes
(0093722)
abv   
2020-08-29 21:28   
The files generated by flex and bison in the current master (in StepFile and ExprIntrp) were generated by the version of the tools that were distributed with WOK 6.8.0:
• Flex 2.5.3
• Bison 2.7

Note that these versions do not match any available variant of win_flex_bison package: even version 1.0 of that distribution contained flex 2.5.35, i.e. newer than that used by OCCT.
Note that this contradicts with statement in Readme file located together with these binaries in WOK.
(0093723)
git   
2020-08-29 21:30   
Branch CR31740 has been created by abv.

SHA-1: 9aa82f7bcbe8c99fa7fba3f0308c4db63230b348


Detailed log of new commits:

Author: abv
Date: Fri Aug 28 18:09:54 2020 +0300

    0031740: Configuration - recover support of Yacc and Lex generation
    
    Scripts adm/cmake/bison.cmake and adm/cmake/flex.cmake are refactored to enable actual search for bison and flex.
    Apart of standard locations, also sub-folders of 3RDPARTY_DIR whose names contain "bison" and "flex", respectively, are added to search.
    Cache variables 3RDPARTY_BISON_EXECUTABLE and 3RDPARTY_FLEX_EXECUTABLE are removed to avoid confusion (they duplicated similar variables without "3RDPARTY_" prefix).
    
    Lex and Yacc files are corrected to match changes made manually in generated files during last years:
    - StepFile/step.yacc: correction missing from 0022972
    - StepFile/step.lex: corrected for compilation (broken by 0031060)
    - MSVC-specific code is synchronized between StepFile/step.lex and ExprIntrp/ExprIntrp.lex
    
    Commands for execution of Flex and Bison tools in CMake scripts are tweaked to avoid embedding line numbers (with local paths) in generated files.
    
    Scanners and parsers are regenerated from updated source files with modified options.
    Note that lex.ExprIntrp.c is regenerated with multiple differences because option -f (fast scanner) was used for generation of previous version (by WOK).
(0093724)
git   
2020-08-30 06:11   
Branch CR31740_win_flex_bison has been created by abv.

SHA-1: 74475399d21aa1dfc7225c6ee8d3f669fbf36259


Detailed log of new commits:

Author: abv
Date: Sun Aug 30 06:10:50 2020 +0300

    For testing only: build with latest win_flex_bison
    
    flex 2.6.4
    bison 3.7.1
(0093725)
abv   
2020-08-30 06:32   
On Ubuntu 16.04 I have

flex 2.6.0
bison 3.0.4
(0093726)
abv   
2020-08-30 06:49   
Upgrading to bison newer than 2.7 can be helpful at least in one sense: we can get rid of annoying inclusion of local path in header files produced by bison, like this:

#ifndef YY_EXPRINTRP_D_ABV_OCCT_OCCT_SRC_EXPRINTRP_EXPRINTRP_TAB_H_INCLUDED
# define YY_EXPRINTRP_D_ABV_OCCT_OCCT_SRC_EXPRINTRP_EXPRINTRP_TAB_H_INCLUDED

For that, option -M can be added in bison command in adm/cmake/occt_toolkit.cmake, as follows:

          BISON_TARGET (Parser_${CURRENT_BISON_FILE_NAME} ${CURRENT_BISON_FILE} ${CMAKE_SOURCE_DIR}/${RELATIVE_SOURCES_DIR}/${OCCT_PACKAGE}/${BISON_OUTPUT_FILE} COMPILE_FLAGS "-p ${CURRENT_BISON_FILE_NAME} -l -M ${CMAKE_SOURCE_DIR}/${RELATIVE_SOURCES_DIR}/=")
(0093732)
abv   
2020-08-30 18:58   
(edited on: 2020-08-30 18:59)
Fix pushed to branch CR31740, please review. Tests are OK with flex and bison from both WOK 6.8.0 and latest win_flex_bison 2.5.23, see Jenkins job CR31740-abv.

(0093743)
dpasukhi   
2020-08-30 21:57   
(edited on: 2020-08-30 22:01)
Dear abv, some files have a some misprint and inaccuracy
adm/cmake/bison.cmake:
adm/cmake/flex.cmake:
>> 5 # delete obsolete 3RDPARTY_BISON_EXECUTABLE cache variable ((not used any more)
 double '('
 anymore - has the meaning of ‘no longer’

src/ExprIntrp/ExprIntrp.tab.c:
src/ExprIntrp/ExprIntrp.tab.h:
src/StepFile/step.tab.c:
src/StepFile/step.tab.h:
>>
 Not have Open Cascade comments
maybe a solution, update comment in the .yacc file with directive
%code top {
 /* anythings */
 }

(0093749)
git   
2020-08-31 10:16   
Branch CR31740 has been updated by abv.

SHA-1: c3d3eb478a753c99f361d8e230d61f240061959d


Detailed log of new commits:

Author: abv
Date: Mon Aug 31 10:01:12 2020 +0300

    Review comments are accounted; some code clean-up

(0093751)
git   
2020-08-31 10:33   
Branch CR31740_1 has been created by abv.

SHA-1: 9dffa4beb7c2dccebbe2e377181bd9e492533c35


Detailed log of new commits:

Author: abv
Date: Fri Aug 28 18:09:54 2020 +0300

    0031740: Configuration - recover support of Yacc and Lex generation
    
    Scripts adm/cmake/bison.cmake and adm/cmake/flex.cmake are refactored to enable actual search for bison and flex.
    Apart of standard locations, also sub-folders of 3RDPARTY_DIR whose names contain "bison" and "flex", respectively, are added to search.
    Cache variables 3RDPARTY_BISON_EXECUTABLE and 3RDPARTY_FLEX_EXECUTABLE are removed to avoid confusion (they duplicated similar variables without "3RDPARTY_" prefix).
    
    Lex and Yacc files are corrected to match changes made manually in generated files during last years:
    - StepFile/step.yacc: correction missing from 0022972
    - StepFile/step.lex: corrected for compilation (broken by 0031060)
    - MSVC-specific code is synchronized between StepFile/step.lex and ExprIntrp/ExprIntrp.lex
    - Old commented code and duplicate code blocks removed
    
    Commands for execution of Flex and Bison tools in CMake scripts are tweaked to avoid embedding line numbers (with local paths) in generated files.
    
    Scanners and parsers are regenerated from updated source files with modified options.
    Note that lex.ExprIntrp.c is regenerated with multiple differences because option -f (fast scanner) was used for generation of previous version (by WOK).
(0093752)
abv   
2020-08-31 10:35   
Code is corrected, please review (see last commit in branch CR31740; branch CR31740_1 is complete fix with all commits squashed, rebased on current master)
(0093756)
dpasukhi   
2020-08-31 11:32   
Dear abv,
I have a some bug. If cmake-FLEX_EXECUTABLE is not empty or points to in other directory, configure is crush. The same bison
(0093757)
dpasukhi   
2020-08-31 12:12   
Review have been done.
(0093768)
abv   
2020-08-31 16:00   
Tests passed OK, see Jenkins job CR31740-abv. Please integrate:
- OCCT branch CR31740_1
- Products: none
(0094050)
bugmaster   
2020-09-05 12:06   
Combination -
OCCT branch : IR-2020-09-04
master SHA - a516227511f3452f9f55b79c961265b0bf210793
a206de37fbfa0bf71bd534ae47192bbec23b8522
Products branch : IR-2020-09-04 SHA - 134448d280fb82bfb14ffc6255ac1cb53f7fd1cf
was compiled on Linux, MacOS and Windows platforms and tested in optimize mode.

Number of compiler warnings:
No new/fixed warnings

Regressions/Differences/Improvements:
No regressions/differences

CPU differences:
Debian80-64:
OCCT
Total CPU difference: 17319.18000000012 / 17284.370000000214 [+0.20%]
Products
Total CPU difference: 12059.670000000087 / 12007.590000000067 [+0.43%]
Windows-64-VC14:
OCCT
Total CPU difference: 18776.390625 / 18770.796875 [+0.03%]
Products
Total CPU difference: 13251.640625 / 13222.90625 [+0.22%]


Image differences :
No differences that require special attention

Memory differences :
No differences that require special attention
(0094093)
git   
2020-09-05 12:35   
Branch CR31740_1 has been deleted by inv.

SHA-1: 9dffa4beb7c2dccebbe2e377181bd9e492533c35
(0094094)
git   
2020-09-05 12:35   
Branch CR31740 has been deleted by inv.

SHA-1: c3d3eb478a753c99f361d8e230d61f240061959d
(0094096)
git   
2020-09-05 12:35   
Branch CR31740_win_flex_bison has been deleted by inv.

SHA-1: 74475399d21aa1dfc7225c6ee8d3f669fbf36259