View Issue Details

IDProjectCategoryView StatusLast Update
0027838Open CASCADEOCCT:Foundation Classespublic2019-03-01 17:10
Reporterkgv Assigned Tokgv  
PrioritynormalSeverityminor 
Status closedResolutionfixed 
PlatformWindowsOSVC++ 2015 
Product Version7.0.0 
Target Version7.1.0Fixed in Version7.1.0 
Summary0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString
DescriptionThe most used string classes in OCCT are TCollection_AsciiString and TCollection_ExtendedString.

TCollection_ExtendedString is intended to store UTF-16 string, while TCollection_AsciiString is for UTF-8.

Most systems just use UTF-8 strings in platform APIs - and thus "const char*" and TCollection_AsciiString can be transparently used there.
However, Windows supports only one way for passing Unicode strings - through "wide-char" APIs taking "wchar_t*".

The problem is that TCollection_ExtendedString defined UTF-16 characters in platform-independent manner and does not provide any means to handle "wchar_t*" strings (despite the fact that on Windows these types have the same size).
This requires very annoying conversion routines to pass / take Unicode string from Windows APIs - usually using static casts at every place.

It is proposed to extend TCollection_AsciiString/TCollection_ExtendedString classes to handle "wchar_t*" strings on Windows platform to make syntax more transparent to end-user.
Steps To ReproduceN/A
TagsNo tags attached.
Test case numberNot needed

Relationships

related to 0026380 closedbugmaster Open CASCADE OSD_SharedLibrary - handle UTF-8 file paths 
related to 0025308 newszy Open CASCADE TCollection_ExtendedString, NCollection_String - merge classes for string management 
related to 0027844 closedbugmaster Open CASCADE Foundation Classes - remove method OSD_File::Print() 
related to 0027880 closedbugmaster Open CASCADE Samples - fix handling of Unicode paths within MFC import/export sample 
related to 0027960 closedapn Community Configuration - fix compilation of OSD_Directory with MinGW-w64 
child of 0022484 closedbugmaster Open CASCADE UNICODE characters support. 

Activities

git

2016-09-04 14:33

administrator   ~0057448

Branch CR27838 has been created by kgv.

SHA-1: 4cb3a5dbb128e8e6102d7bfbc43d64f2c0c35dd5


Detailed log of new commits:

Author: kgv
Date: Sun Sep 4 14:31:47 2016 +0300

    0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString
    
    TCollection_ExtendedString/TCollection_AsciiString description
    has been updated to reflect usage of this classes for Unicode strings.
    
    TCollection_ExtendedString now defines constructor taking wchar_t* (all platforms)
    and method ::ToWideString() returning wchar_t* (Windows only).
    TCollection_AsciiString now defines constructor taking wchar_t*.
    
    TCollection_ExtendedString/TCollection_AsciiString now defines
    auxiliary methods ::StartsWith() and ::EndsWith().
    
    TCollection_ExtendedString internals has been updated to eliminate
    duplicated code for converting between UTF-16 and UTF-8.
    The internal method Allocate() in TCollection_ExtendedString.cxx
    now automatically NULL-terminate allocated buffer.
    
    Code has been cleaned up from redundant explicit conversions to wchar_t*.
    Global method OSD_OpenStream()/OSD_OpenFileBuf() have been replaced
    by C++ template to eliminate copy-paste for different STL collections.
    
    OSD_SharedLibrary now uses wide-char system API call LoadLibraryExW()
    on Windows for consistency.

kgv

2016-09-04 14:33

developer   ~0057449

Patch is ready for review.

abv

2016-09-06 13:54

manager   ~0057512

Some remarks:

1. In OSD_File.cxx, 2791:

- if ( !OpenPrinter ( Standard_PCharacter(pName), &hPrinter, NULL ) ) {
+ if ( !OpenPrinterW ((wchar_t* )pName, &hPrinter, NULL ) ) {

this conversion is suspicious since pName is still PCharacter, i.e. char *

2. In TCollection_ExtendedString, methods Re/Allocate():
- perhaps it is better to use size in characters, not in bytes
- it is somehow innatural to delegate the task of setting terminating null symbol to allocation function... the code would be less confusing if null was added explicitly whenever string is filled

3. In TCollection_ExtendedString, run-time "switch (sizeof(Standard_WideChar))" should be replaced by compile-time equivalent, e.g. using template specializations. For an example of the technique, see OCCT_InverseSizeSpecialized in FSD_BinaryFile.cxx

4. In TCollection_ExtendedString.hxx, I suggest removing "has been decided to " and replacing "might not" by "can miss". Idem replace "NULL-terminated symbol" by "terminating NULL symbol"

kgv

2016-09-06 14:34

developer   ~0057516

-  if (  !OpenPrinter ( Standard_PCharacter(pName), &hPrinter, NULL )  ) {
+  if (  !OpenPrinterW ((wchar_t* )pName, &hPrinter, NULL )  ) {

Should be const_cast<wchar_t*>

git

2016-09-06 19:01

administrator   ~0057532

Branch CR27838 has been updated by kgv.

SHA-1: 2c65aca3e8d19ff54774fed1f94b39adfb7438e0


Detailed log of new commits:

Author: kgv
Date: Tue Sep 6 19:03:46 2016 +0300

    apply remarks

git

2016-09-06 19:03

administrator   ~0057533

Branch CR27838_1 has been created by kgv.

SHA-1: 6d82d9da31cb22e0bf588f20292ff73112a194fb


Detailed log of new commits:

Author: kgv
Date: Sun Sep 4 14:31:47 2016 +0300

    0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString
    
    TCollection_ExtendedString/TCollection_AsciiString description
    has been updated to reflect usage of this classes for Unicode strings.
    
    TCollection_ExtendedString now defines constructor taking wchar_t* (all platforms)
    and method ::ToWideString() returning wchar_t* (Windows only).
    TCollection_AsciiString now defines constructor taking wchar_t*.
    
    TCollection_ExtendedString/TCollection_AsciiString now defines
    auxiliary methods ::StartsWith() and ::EndsWith().
    
    TCollection_ExtendedString internals has been updated to eliminate
    duplicated code for converting between UTF-16 and UTF-8.
    
    Code has been cleaned up from redundant explicit conversions to wchar_t*.
    Global method OSD_OpenStream()/OSD_OpenFileBuf() have been replaced
    by C++ template to eliminate copy-paste for different STL collections.
    
    OSD_SharedLibrary now uses wide-char system API call LoadLibraryExW()
    on Windows for consistency.

kgv

2016-09-06 19:04

developer   ~0057534

Remarks have been applied.

abv

2016-09-07 14:20

manager   ~0057561

No remarks, please test

apv

2016-09-08 09:43

tester   ~0057580

Dear BugMaster,

Branch CR27838_1 from occt git-repository (and master from products git-repository) was compiled on Linux, MacOS and Windows platforms.
SHA-1: 6d82d9da31cb22e0bf588f20292ff73112a194fb

Number of compiler warnings:
occt component:
   Linux: Not available
   Windows: 0 (0 on master)
   MasOS: Not available
products component:
   Linux: Not available
   Windows: 0
   MacOS: Not available

During building of branch CR27838_1 from occt git-repository and master from products git-repository following compilation errors have been detected
On Linux:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_1-master/job/CR27838_1-master-OCCT-Debian70-64-opt-compile/1/parsed_console/
On MacOS:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_1-master/job/CR27838_1-master-OCCT-MacOS-opt-compile/1/parsed_console/

Moreover compilation error has been detected during building of
Qt OCCT samples on Windows:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_1-master/job/CR27838_1-master-OCCT-Windows-64-VC10-qt-samples-windows/1/parsed_console/
Qt Products samples on Windows:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_1-master/job/CR27838_1-master-Products-Windows-64-VC10-qt-samples-windows/1/parsed_console/

apv

2016-09-08 09:44

tester   ~0057581

Dear Kirill,

Branch CR27838_1 has been rejected due to:
- compilation errors

git

2016-09-08 10:13

administrator   ~0057582

Branch CR27838_1 has been updated by kgv.

SHA-1: acbb3bb805272d0853acfe46f11d3f4952c1fc66


Detailed log of new commits:

Author: kgv
Date: Thu Sep 8 10:16:04 2016 +0300

    fix compilation issues

git

2016-09-08 10:49

administrator   ~0057585

Branch CR27838_1 has been updated forcibly by kgv.

SHA-1: 88bb3801663a0d1aa567cf86b0d7857a00356607

git

2016-09-08 10:50

administrator   ~0057586

Branch CR27838_2 has been created by kgv.

SHA-1: 53d3d3359f8f31600aa3387dab8fe29f7fdb9240


Detailed log of new commits:

Author: kgv
Date: Sun Sep 4 14:31:47 2016 +0300

    0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString
    
    TCollection_ExtendedString/TCollection_AsciiString description
    has been updated to reflect usage of this classes for Unicode strings.
    
    TCollection_ExtendedString now defines constructor taking wchar_t* (all platforms)
    and method ::ToWideString() returning wchar_t* (Windows only).
    TCollection_AsciiString now defines constructor taking wchar_t*.
    
    TCollection_ExtendedString/TCollection_AsciiString now defines
    auxiliary methods ::StartsWith() and ::EndsWith().
    
    TCollection_ExtendedString internals has been updated to eliminate
    duplicated code for converting between UTF-16 and UTF-8.
    
    Code has been cleaned up from redundant explicit conversions to wchar_t*.
    Global method OSD_OpenStream()/OSD_OpenFileBuf() have been replaced
    by C++ template to eliminate copy-paste for different STL collections.
    
    OSD_SharedLibrary now uses wide-char system API call LoadLibraryExW()
    on Windows for consistency.

kgv

2016-09-08 10:55

developer   ~0057587

Last edited: 2016-09-08 10:55

Please check updated patch in branch CR27838_2 which should workaround awkward compatibility issues.

It seems that compilation issues in Qt samples caused by using deprecated msvc compilation flag "-Zc:wchar_t-" which makes wchar_t as not built-in type.
This issue has been fixed in Qt5, so I suppose old Qt4 is used for building samples:
https://bugreports.qt.io/browse/QTBUG-9617

apv

2016-09-09 11:05

tester   ~0057625

Dear Kirill,

Please, rebase CR27838_2 on the current master. Thank you in advance.

git

2016-09-09 11:19

administrator   ~0057628

Branch CR27838_2 has been updated forcibly by kgv.

SHA-1: baa5f5802bb3428537160deb783394acdf599958

kgv

2016-09-09 11:20

developer   ~0057629

Branch has been rebased.

apv

2016-09-12 13:45

tester   ~0057682

Dear BugMaster,

Branch CR27838_2 from occt git-repository (and master from products git-repository) was compiled on Linux, MacOS and Windows platforms and tested.
SHA-1: baa5f5802bb3428537160deb783394acdf599958

Number of compiler warnings:
occt component:
   Linux: 0 (0 on master)
   Windows: 0 (0 on master)
   MasOS: 1 (0 on master)
products component:
   Linux: 64
   Windows: 0
   MacOS: 1129
New warnings have been detected during OCCT component building
on MacOS:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_2-master/job/CR27838_2-master-OCCT-MacOS-opt-compile/1/warnings7Result/

Regressions/Differences:
Not detected

Testing cases:
Not needed

Testing on Linux:
Total MEMORY difference: 89720825 / 90408349 [-0.76%]
Total CPU difference: 19452.11000000009 / 19445.350000000133 [+0.03%]

Testing on Windows:
Total MEMORY difference: 57157566 / 57172575 [-0.03%]
Total CPU difference: 18246.641364898747 / 18100.047225198865 [+0.81%]

apv

2016-09-12 13:46

tester   ~0057683

Dear Kirill,

Branch CR27838_2 has been rejected due to:
- additional warnings

git

2016-09-12 15:46

administrator   ~0057688

Branch CR27838_2 has been updated by kgv.

SHA-1: 5c146df1a4bbdcd1d0d6990998efd7468f1d9b22


Detailed log of new commits:

Author: kgv
Date: Mon Sep 12 15:46:37 2016 +0300

    mark fromWideString() as inline in TCollection_ExtendedString.cxx

kgv

2016-09-12 15:47

developer   ~0057689

Please re-check compilation of updated patch on OS X, regression testing is not needed.

apv

2016-09-13 13:48

tester   ~0057738

Dear BugMaster,

Branch CR27838_2 from occt git-repository (and master from products git-repository) was compiled on Linux, MacOS and Windows platforms and tested.
SHA-1: 5c146df1a4bbdcd1d0d6990998efd7468f1d9b22

Number of compiler warnings:
occt component:
   Linux: 0 (0 on master)
   Windows: 0 (0 on master)
   MasOS: 1 (0 on master)
products component:
   Linux: 64
   Windows: 0
   MacOS: 1134
New warnings have been detected during OCCT component building
on MacOS:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_2-master/job/CR27838_2-master-OCCT-MacOS-opt-compile/1/warnings7Result/

Regressions/Differences:
Not detected

Testing cases:
Not needed

Testing on Linux:
Total MEMORY difference: 89462472 / 90412700 [-1.05%]
Total CPU difference: 19406.839999999967 / 19445.440000000133 [-0.20%]

Testing on Windows:
Total MEMORY difference: 57157530 / 57172575 [-0.03%]
Total CPU difference: 18357.324074398828 / 18100.047225198865 [+1.42%]

apv

2016-09-13 13:49

tester   ~0057739

Dear Kirill,

Branch CR27838_2 has been rejected due to:
- additional warnings

git

2016-09-13 14:13

administrator   ~0057740

Branch CR27838_2 has been updated by kgv.

SHA-1: de5403ed5b30d46dd74f2f33a9a00bc1242caac8


Detailed log of new commits:

Author: kgv
Date: Tue Sep 13 14:12:59 2016 +0300

    fromWideString() in TCollection_ExtendedString.cxx has been marked with Standard_UNUSED.

kgv

2016-09-13 14:14

developer   ~0057741

Please try compiling updated patch, regression testing is not needed.

git

2016-09-13 14:16

administrator   ~0057742

Branch CR27838_2 has been updated forcibly by kgv.

SHA-1: bc2f92295c3b32e119a15a33a5329b92d151c701

apv

2016-09-14 13:12

tester   ~0057772

Dear BugMaster,

Branch CR27838_2 from occt git-repository (and master from products git-repository) was compiled on Linux, MacOS and Windows platforms and tested.
SHA-1: bc2f92295c3b32e119a15a33a5329b92d151c701

Number of compiler warnings:
occt component:
   Linux: 0 (0 on master)
   Windows: 0 (0 on master)
   MasOS: 0 (0 on master)
products component:
   Linux: 64
   Windows: 0
   MacOS: 1139

Regressions/Differences:
Not detected

Testing cases:
Not needed

Testing on Linux:
Total MEMORY difference: 89615541 / 90384642 [-0.85%]
Total CPU difference: 19413.290000000125 / 19445.440000000133 [-0.17%]

Testing on Windows:
Total MEMORY difference: 57157795 / 57172575 [-0.03%]
Total CPU difference: 18255.861023998856 / 18100.047225198865 [+0.86%]

BenjaminBihler

2016-10-13 14:33

updater   ~0058663

From commit fb0b05319f52b12ba581e70f561234fe582c3dc3 on, OCCT cannot be compiled with MinGw-w64 anymore. Should I create a new issue for that?

kgv

2016-10-13 14:39

developer   ~0058664

> Should I create a new issue for that?
Yes, please.

git

2016-10-28 21:45

administrator   ~0059505

Branch CR27838_2 has been deleted by kgv.

SHA-1: bc2f92295c3b32e119a15a33a5329b92d151c701

git

2016-10-28 21:48

administrator   ~0059581

Branch CR27838 has been deleted by kgv.

SHA-1: 2c65aca3e8d19ff54774fed1f94b39adfb7438e0

git

2016-10-28 21:48

administrator   ~0059582

Branch CR27838_1 has been deleted by kgv.

SHA-1: 88bb3801663a0d1aa567cf86b0d7857a00356607

Related Changesets

occt: master fb0b0531

2016-09-04 11:31:47

kgv

Details Diff
0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString

TCollection_ExtendedString/TCollection_AsciiString description
has been updated to reflect usage of this classes for Unicode strings.

TCollection_ExtendedString now defines constructor taking wchar_t* (all platforms)
and method ::ToWideString() returning wchar_t* (Windows only).
TCollection_AsciiString now defines constructor taking wchar_t*.

TCollection_ExtendedString/TCollection_AsciiString now defines
auxiliary methods ::StartsWith() and ::EndsWith().

TCollection_ExtendedString internals has been updated to eliminate
duplicated code for converting between UTF-16 and UTF-8.

Code has been cleaned up from redundant explicit conversions to wchar_t*.
Global method OSD_OpenStream()/OSD_OpenFileBuf() have been replaced
by C++ template to eliminate copy-paste for different STL collections.

OSD_SharedLibrary now uses wide-char system API call LoadLibraryExW()
on Windows for consistency.

New macro Standard_UNUSED has been added for marking possibly unused functions and variables
(to suppress gcc/clang compiler warnings).
Affected Issues
0027838
mod - src/AIS/AIS_Dimension.cxx Diff File
mod - src/AIS/AIS_TextLabel.cxx Diff File
mod - src/BinTools/BinTools.cxx Diff File
mod - src/BRepTools/BRepTools.cxx Diff File
mod - src/Draw/Draw_VariableCommands.cxx Diff File
mod - src/FSD/FSD_CmpFile.cxx Diff File
mod - src/FSD/FSD_File.cxx Diff File
mod - src/Graphic3d/Graphic3d_Group.cxx Diff File
mod - src/Image/Image_AlienPixMap.cxx Diff File
mod - src/NCollection/NCollection_UtfIterator.hxx Diff File
mod - src/NCollection/NCollection_UtfIterator.lxx Diff File
mod - src/NCollection/NCollection_UtfString.hxx Diff File
mod - src/NCollection/NCollection_UtfString.lxx Diff File
mod - src/OpenGl/OpenGl_GraphicDriver.cxx Diff File
mod - src/OpenGl/OpenGl_Text.cxx Diff File
mod - src/OSD/OSD_Directory.cxx Diff File
mod - src/OSD/OSD_DirectoryIterator.cxx Diff File
mod - src/OSD/OSD_Disk.cxx Diff File
mod - src/OSD/OSD_File.cxx Diff File
mod - src/OSD/OSD_FileIterator.cxx Diff File
mod - src/OSD/OSD_FileNode.cxx Diff File
mod - src/OSD/OSD_OpenFile.cxx Diff File
mod - src/OSD/OSD_OpenFile.hxx Diff File
mod - src/OSD/OSD_Process.cxx Diff File
mod - src/OSD/OSD_SharedLibrary.cxx Diff File
mod - src/Standard/Standard.cxx Diff File
mod - src/Standard/Standard_Macro.hxx Diff File
mod - src/TCollection/FILES Diff File
mod - src/TCollection/TCollection_AsciiString.cxx Diff File
mod - src/TCollection/TCollection_AsciiString.hxx Diff File
mod - src/TCollection/TCollection_ExtendedString.cxx Diff File
mod - src/TCollection/TCollection_ExtendedString.hxx Diff File
rm - src/TCollection/TCollection_ExtendedString.lxx Diff File

Issue History

Date Modified Username Field Change
2016-09-04 12:16 kgv New Issue
2016-09-04 12:16 kgv Assigned To => abv
2016-09-04 14:33 git Note Added: 0057448
2016-09-04 14:33 kgv Note Added: 0057449
2016-09-04 14:33 kgv Status new => resolved
2016-09-04 14:34 kgv Relationship added child of 0022484
2016-09-04 14:35 kgv Relationship added related to 0026380
2016-09-04 14:35 kgv Relationship added related to 0025308
2016-09-06 13:54 abv Note Added: 0057512
2016-09-06 13:54 abv Assigned To abv => kgv
2016-09-06 13:54 abv Status resolved => assigned
2016-09-06 14:34 kgv Note Added: 0057516
2016-09-06 14:38 kgv Relationship added related to 0027844
2016-09-06 19:01 git Note Added: 0057532
2016-09-06 19:03 git Note Added: 0057533
2016-09-06 19:04 kgv Note Added: 0057534
2016-09-06 19:04 kgv Assigned To kgv => abv
2016-09-06 19:04 kgv Status assigned => resolved
2016-09-07 14:20 abv Note Added: 0057561
2016-09-07 14:20 abv Assigned To abv => bugmaster
2016-09-07 14:20 abv Status resolved => reviewed
2016-09-07 15:55 apv Assigned To bugmaster => apv
2016-09-08 09:43 apv Note Added: 0057580
2016-09-08 09:43 apv Assigned To apv => kgv
2016-09-08 09:43 apv Status reviewed => assigned
2016-09-08 09:44 apv Note Added: 0057581
2016-09-08 10:13 git Note Added: 0057582
2016-09-08 10:49 git Note Added: 0057585
2016-09-08 10:50 git Note Added: 0057586
2016-09-08 10:55 kgv Note Added: 0057587
2016-09-08 10:55 kgv Assigned To kgv => bugmaster
2016-09-08 10:55 kgv Status assigned => resolved
2016-09-08 10:55 kgv Status resolved => reviewed
2016-09-08 10:55 kgv Note Edited: 0057587
2016-09-09 09:08 apv Assigned To bugmaster => apv
2016-09-09 11:05 apv Note Added: 0057625
2016-09-09 11:05 apv Assigned To apv => kgv
2016-09-09 11:05 apv Status reviewed => feedback
2016-09-09 11:19 git Note Added: 0057628
2016-09-09 11:20 kgv Note Added: 0057629
2016-09-09 11:20 kgv Assigned To kgv => bugmaster
2016-09-09 11:20 kgv Status feedback => reviewed
2016-09-09 11:25 apv Assigned To bugmaster => apv
2016-09-12 13:02 apv Test case number => Not needed
2016-09-12 13:45 apv Note Added: 0057682
2016-09-12 13:45 apv Assigned To apv => kgv
2016-09-12 13:45 apv Status reviewed => assigned
2016-09-12 13:46 apv Note Added: 0057683
2016-09-12 15:46 git Note Added: 0057688
2016-09-12 15:47 kgv Note Added: 0057689
2016-09-12 15:47 kgv Assigned To kgv => bugmaster
2016-09-12 15:47 kgv Status assigned => resolved
2016-09-12 15:47 kgv Status resolved => reviewed
2016-09-12 16:13 apv Assigned To bugmaster => apv
2016-09-13 13:48 apv Note Added: 0057738
2016-09-13 13:48 apv Assigned To apv => kgv
2016-09-13 13:48 apv Status reviewed => assigned
2016-09-13 13:49 apv Note Added: 0057739
2016-09-13 14:13 git Note Added: 0057740
2016-09-13 14:14 kgv Note Added: 0057741
2016-09-13 14:14 kgv Assigned To kgv => bugmaster
2016-09-13 14:14 kgv Status assigned => resolved
2016-09-13 14:14 kgv Status resolved => reviewed
2016-09-13 14:16 git Note Added: 0057742
2016-09-13 14:51 apv Assigned To bugmaster => apv
2016-09-14 13:12 apv Note Added: 0057772
2016-09-14 13:12 apv Assigned To apv => bugmaster
2016-09-14 13:12 apv Status reviewed => tested
2016-09-16 09:43 kgv Changeset attached => occt master fb0b0531
2016-09-16 09:43 kgv Assigned To bugmaster => kgv
2016-09-16 09:43 kgv Status tested => verified
2016-09-16 09:43 kgv Resolution open => fixed
2016-09-19 11:47 kgv Relationship added related to 0027880
2016-10-13 14:33 BenjaminBihler Note Added: 0058663
2016-10-13 14:39 kgv Note Added: 0058664
2016-10-13 14:50 BenjaminBihler Relationship added related to 0027960
2016-10-28 21:45 git Note Added: 0059505
2016-10-28 21:48 git Note Added: 0059581
2016-10-28 21:48 git Note Added: 0059582
2016-12-09 16:30 aiv Status verified => closed
2016-12-09 16:39 aiv Fixed in Version => 7.1.0