View Issue Details

Related ChangesetsJump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0031740Open CASCADEOCCT:Configurationpublic2020-08-29 07:452020-12-02 17:13
ReporterabvAssigned Tobugmaster  
PrioritynormalSeverityminor 
Status closedResolutionfixed 
Target Version7.5.0Fixed in Version7.5.0 
Summary0031740: Configuration - recover support of Yacc and Lex generation
DescriptionSupport of Yacc and Lex generation in OCCT CMake scripts is incomplete:
- we have no ready-to-use package of Flex and Bison matching the generated files currently stored in master
- CMake script cannot find flex and bison automatically even when they are present in the system (on Linux) or in 3rd-party folder
- options used for calling Flex and Bison in CMake script differ from those used for generation of files currently stored in master

Steps To ReproduceN/A
TagsNo tags attached.
Test case numberNot required

Relationships

related to 0027342 closedbugmaster Open CASCADE Data Exchange, STEP - support C++ streams for import 
related to 0031481 closedabv Open CASCADE Data Exchange - provide parser of STEP EXPRESS schema for generation of new STEP entities 

Activities

abv

2020-08-29 21:28

manager   ~0093722

The files generated by flex and bison in the current master (in StepFile and ExprIntrp) were generated by the version of the tools that were distributed with WOK 6.8.0:
• Flex 2.5.3
• Bison 2.7

Note that these versions do not match any available variant of win_flex_bison package: even version 1.0 of that distribution contained flex 2.5.35, i.e. newer than that used by OCCT.
Note that this contradicts with statement in Readme file located together with these binaries in WOK.

git

2020-08-29 21:30

administrator   ~0093723

Branch CR31740 has been created by abv.

SHA-1: 9aa82f7bcbe8c99fa7fba3f0308c4db63230b348


Detailed log of new commits:

Author: abv
Date: Fri Aug 28 18:09:54 2020 +0300

    0031740: Configuration - recover support of Yacc and Lex generation
    
    Scripts adm/cmake/bison.cmake and adm/cmake/flex.cmake are refactored to enable actual search for bison and flex.
    Apart of standard locations, also sub-folders of 3RDPARTY_DIR whose names contain "bison" and "flex", respectively, are added to search.
    Cache variables 3RDPARTY_BISON_EXECUTABLE and 3RDPARTY_FLEX_EXECUTABLE are removed to avoid confusion (they duplicated similar variables without "3RDPARTY_" prefix).
    
    Lex and Yacc files are corrected to match changes made manually in generated files during last years:
    - StepFile/step.yacc: correction missing from 0022972
    - StepFile/step.lex: corrected for compilation (broken by 0031060)
    - MSVC-specific code is synchronized between StepFile/step.lex and ExprIntrp/ExprIntrp.lex
    
    Commands for execution of Flex and Bison tools in CMake scripts are tweaked to avoid embedding line numbers (with local paths) in generated files.
    
    Scanners and parsers are regenerated from updated source files with modified options.
    Note that lex.ExprIntrp.c is regenerated with multiple differences because option -f (fast scanner) was used for generation of previous version (by WOK).

git

2020-08-30 06:11

administrator   ~0093724

Branch CR31740_win_flex_bison has been created by abv.

SHA-1: 74475399d21aa1dfc7225c6ee8d3f669fbf36259


Detailed log of new commits:

Author: abv
Date: Sun Aug 30 06:10:50 2020 +0300

    For testing only: build with latest win_flex_bison
    
    flex 2.6.4
    bison 3.7.1

abv

2020-08-30 06:32

manager   ~0093725

On Ubuntu 16.04 I have

flex 2.6.0
bison 3.0.4

abv

2020-08-30 06:49

manager   ~0093726

Upgrading to bison newer than 2.7 can be helpful at least in one sense: we can get rid of annoying inclusion of local path in header files produced by bison, like this:

#ifndef YY_EXPRINTRP_D_ABV_OCCT_OCCT_SRC_EXPRINTRP_EXPRINTRP_TAB_H_INCLUDED
# define YY_EXPRINTRP_D_ABV_OCCT_OCCT_SRC_EXPRINTRP_EXPRINTRP_TAB_H_INCLUDED

For that, option -M can be added in bison command in adm/cmake/occt_toolkit.cmake, as follows:

          BISON_TARGET (Parser_${CURRENT_BISON_FILE_NAME} ${CURRENT_BISON_FILE} ${CMAKE_SOURCE_DIR}/${RELATIVE_SOURCES_DIR}/${OCCT_PACKAGE}/${BISON_OUTPUT_FILE} COMPILE_FLAGS "-p ${CURRENT_BISON_FILE_NAME} -l -M ${CMAKE_SOURCE_DIR}/${RELATIVE_SOURCES_DIR}/=")

abv

2020-08-30 18:58

manager   ~0093732

Last edited: 2020-08-30 18:59

 Fix pushed to branch CR31740, please review. Tests are OK with flex and bison from both WOK 6.8.0 and latest win_flex_bison 2.5.23, see Jenkins job CR31740-abv.

dpasukhi

2020-08-30 21:57

administrator   ~0093743

Last edited: 2020-08-30 22:01

 Dear abv, some files have a some misprint and inaccuracy
adm/cmake/bison.cmake:
adm/cmake/flex.cmake:
>> 5 # delete obsolete 3RDPARTY_BISON_EXECUTABLE cache variable ((not used any more)
 double '('
 anymore - has the meaning of ‘no longer’

src/ExprIntrp/ExprIntrp.tab.c:
src/ExprIntrp/ExprIntrp.tab.h:
src/StepFile/step.tab.c:
src/StepFile/step.tab.h:
>>
 Not have Open Cascade comments
maybe a solution, update comment in the .yacc file with directive
%code top {
 /* anythings */
 }

git

2020-08-31 10:16

administrator   ~0093749

Branch CR31740 has been updated by abv.

SHA-1: c3d3eb478a753c99f361d8e230d61f240061959d


Detailed log of new commits:

Author: abv
Date: Mon Aug 31 10:01:12 2020 +0300

    Review comments are accounted; some code clean-up

git

2020-08-31 10:33

administrator   ~0093751

Branch CR31740_1 has been created by abv.

SHA-1: 9dffa4beb7c2dccebbe2e377181bd9e492533c35


Detailed log of new commits:

Author: abv
Date: Fri Aug 28 18:09:54 2020 +0300

    0031740: Configuration - recover support of Yacc and Lex generation
    
    Scripts adm/cmake/bison.cmake and adm/cmake/flex.cmake are refactored to enable actual search for bison and flex.
    Apart of standard locations, also sub-folders of 3RDPARTY_DIR whose names contain "bison" and "flex", respectively, are added to search.
    Cache variables 3RDPARTY_BISON_EXECUTABLE and 3RDPARTY_FLEX_EXECUTABLE are removed to avoid confusion (they duplicated similar variables without "3RDPARTY_" prefix).
    
    Lex and Yacc files are corrected to match changes made manually in generated files during last years:
    - StepFile/step.yacc: correction missing from 0022972
    - StepFile/step.lex: corrected for compilation (broken by 0031060)
    - MSVC-specific code is synchronized between StepFile/step.lex and ExprIntrp/ExprIntrp.lex
    - Old commented code and duplicate code blocks removed
    
    Commands for execution of Flex and Bison tools in CMake scripts are tweaked to avoid embedding line numbers (with local paths) in generated files.
    
    Scanners and parsers are regenerated from updated source files with modified options.
    Note that lex.ExprIntrp.c is regenerated with multiple differences because option -f (fast scanner) was used for generation of previous version (by WOK).

abv

2020-08-31 10:35

manager   ~0093752

Code is corrected, please review (see last commit in branch CR31740; branch CR31740_1 is complete fix with all commits squashed, rebased on current master)

dpasukhi

2020-08-31 11:32

administrator   ~0093756

Dear abv,
I have a some bug. If cmake-FLEX_EXECUTABLE is not empty or points to in other directory, configure is crush. The same bison

dpasukhi

2020-08-31 12:12

administrator   ~0093757

Review have been done.

abv

2020-08-31 16:00

manager   ~0093768

Tests passed OK, see Jenkins job CR31740-abv. Please integrate:
- OCCT branch CR31740_1
- Products: none

bugmaster

2020-09-05 12:06

administrator   ~0094050

Combination -
OCCT branch : IR-2020-09-04
master SHA - a516227511f3452f9f55b79c961265b0bf210793
a206de37fbfa0bf71bd534ae47192bbec23b8522
Products branch : IR-2020-09-04 SHA - 134448d280fb82bfb14ffc6255ac1cb53f7fd1cf
was compiled on Linux, MacOS and Windows platforms and tested in optimize mode.

Number of compiler warnings:
No new/fixed warnings

Regressions/Differences/Improvements:
No regressions/differences

CPU differences:
Debian80-64:
OCCT
Total CPU difference: 17319.18000000012 / 17284.370000000214 [+0.20%]
Products
Total CPU difference: 12059.670000000087 / 12007.590000000067 [+0.43%]
Windows-64-VC14:
OCCT
Total CPU difference: 18776.390625 / 18770.796875 [+0.03%]
Products
Total CPU difference: 13251.640625 / 13222.90625 [+0.22%]


Image differences :
No differences that require special attention

Memory differences :
No differences that require special attention

git

2020-09-05 12:35

administrator   ~0094093

Branch CR31740_1 has been deleted by inv.

SHA-1: 9dffa4beb7c2dccebbe2e377181bd9e492533c35

git

2020-09-05 12:35

administrator   ~0094094

Branch CR31740 has been deleted by inv.

SHA-1: c3d3eb478a753c99f361d8e230d61f240061959d

git

2020-09-05 12:35

administrator   ~0094096

Branch CR31740_win_flex_bison has been deleted by inv.

SHA-1: 74475399d21aa1dfc7225c6ee8d3f669fbf36259

Related Changesets

occt: master fba34cf8

2020-08-28 15:09:54

abv


Committer: bugmaster Details Diff		 0031740: Configuration - recover support of Yacc and Lex generation

Scripts adm/cmake/bison.cmake and adm/cmake/flex.cmake are refactored to enable actual search for bison and flex.
Apart of standard locations, also sub-folders of 3RDPARTY_DIR whose names contain "bison" and "flex", respectively, are added to search.
Cache variables 3RDPARTY_BISON_EXECUTABLE and 3RDPARTY_FLEX_EXECUTABLE are removed to avoid confusion (they duplicated similar variables without "3RDPARTY_" prefix).

Lex and Yacc files are corrected to match changes made manually in generated files during last years:
- StepFile/step.yacc: correction missing from 0022972
- StepFile/step.lex: corrected for compilation (broken by 0031060)
- MSVC-specific code is synchronized between StepFile/step.lex and ExprIntrp/ExprIntrp.lex
- Old commented code and duplicate code blocks removed

Commands for execution of Flex and Bison tools in CMake scripts are tweaked to avoid embedding line numbers (with local paths) in generated files.

Scanners and parsers are regenerated from updated source files with modified options.
Note that lex.ExprIntrp.c is regenerated with multiple differences because option -f (fast scanner) was used for generation of previous version (by WOK).		 Affected Issues
0031740
mod - adm/cmake/bison.cmake Diff File
mod - adm/cmake/flex.cmake Diff File
mod - adm/cmake/occt_toolkit.cmake Diff File
mod - src/ExprIntrp/ExprIntrp.lex Diff File
mod - src/ExprIntrp/ExprIntrp.tab.c Diff File
mod - src/ExprIntrp/ExprIntrp.tab.h Diff File
mod - src/ExprIntrp/lex.ExprIntrp.c Diff File
mod - src/StepFile/lex.step.c Diff File
mod - src/StepFile/step.lex Diff File
mod - src/StepFile/step.tab.c Diff File
mod - src/StepFile/step.tab.h Diff File
mod - src/StepFile/step.yacc Diff File

Issue History

Date Modified Username Field Change
2020-08-29 07:45 abv New Issue
2020-08-29 07:45 abv Assigned To => bugmaster
2020-08-29 07:46 abv Relationship added related to 0027342
2020-08-29 21:28 abv Note Added: 0093722
2020-08-29 21:30 git Note Added: 0093723
2020-08-30 06:11 git Note Added: 0093724
2020-08-30 06:32 abv Note Added: 0093725
2020-08-30 06:49 abv Note Added: 0093726
2020-08-30 18:58 abv Note Added: 0093732
2020-08-30 18:58 abv Assigned To bugmaster => dpasukhi
2020-08-30 18:58 abv Status new => resolved
2020-08-30 18:58 abv Steps to Reproduce Updated
2020-08-30 18:59 abv Note Edited: 0093732
2020-08-30 21:57 dpasukhi Note Added: 0093743
2020-08-30 21:57 dpasukhi Assigned To dpasukhi => adv
2020-08-30 21:57 dpasukhi Status resolved => assigned
2020-08-30 21:58 dpasukhi Note Edited: 0093743
2020-08-30 22:00 dpasukhi Note Edited: 0093743
2020-08-30 22:01 dpasukhi Note Edited: 0093743
2020-08-31 09:43 abv Assigned To adv => abv
2020-08-31 10:16 git Note Added: 0093749
2020-08-31 10:33 git Note Added: 0093751
2020-08-31 10:35 abv Note Added: 0093752
2020-08-31 10:35 abv Assigned To abv => bugmaster
2020-08-31 10:35 abv Status assigned => resolved
2020-08-31 10:35 abv Assigned To bugmaster => dpasukhi
2020-08-31 11:32 dpasukhi Note Added: 0093756
2020-08-31 11:32 dpasukhi Status resolved => assigned
2020-08-31 12:11 dpasukhi Status assigned => resolved
2020-08-31 12:12 dpasukhi Note Added: 0093757
2020-08-31 12:12 dpasukhi Assigned To dpasukhi => abv
2020-08-31 12:12 dpasukhi Status resolved => reviewed
2020-08-31 16:00 abv Note Added: 0093768
2020-08-31 16:10 abv Assigned To abv => bugmaster
2020-09-05 12:06 bugmaster Note Added: 0094050
2020-09-05 12:06 bugmaster Status reviewed => tested
2020-09-05 12:16 bugmaster Test case number => Not required
2020-09-05 12:25 bugmaster Changeset attached => occt master fba34cf8
2020-09-05 12:25 bugmaster Status tested => verified
2020-09-05 12:25 bugmaster Resolution open => fixed
2020-09-05 12:35 git Note Added: 0094093
2020-09-05 12:35 git Note Added: 0094094
2020-09-05 12:35 git Note Added: 0094096
2020-09-13 08:22 abv Relationship added related to 0031481
2020-09-14 15:24 bugmaster Status verified => closed
2020-09-14 15:24 bugmaster Assigned To bugmaster => isn
2020-09-14 15:24 bugmaster Status closed => assigned
2020-09-14 15:25 bugmaster Assigned To isn => bugmaster
2020-09-14 15:25 bugmaster Status assigned => resolved
2020-09-14 15:25 bugmaster Status resolved => reviewed
2020-09-14 15:25 bugmaster Status reviewed => verified
2020-12-02 16:43 emo Fixed in Version => 7.5.0
2020-12-02 17:13 emo Status verified => closed