MantisBT - Open CASCADE
View Issue Details
0027838Open CASCADE[OCCT] OCCT:Foundation Classespublic2016-09-04 12:162019-03-01 17:10
kgv 
kgv 
normalminor 
closedfixed 
WindowsVC++ 201564 bit
[OCCT] 7.0.0 
[OCCT] 7.1.0[OCCT] 7.1.0 
Not needed
0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString
The most used string classes in OCCT are TCollection_AsciiString and TCollection_ExtendedString.

TCollection_ExtendedString is intended to store UTF-16 string, while TCollection_AsciiString is for UTF-8.

Most systems just use UTF-8 strings in platform APIs - and thus "const char*" and TCollection_AsciiString can be transparently used there.
However, Windows supports only one way for passing Unicode strings - through "wide-char" APIs taking "wchar_t*".

The problem is that TCollection_ExtendedString defined UTF-16 characters in platform-independent manner and does not provide any means to handle "wchar_t*" strings (despite the fact that on Windows these types have the same size).
This requires very annoying conversion routines to pass / take Unicode string from Windows APIs - usually using static casts at every place.

It is proposed to extend TCollection_AsciiString/TCollection_ExtendedString classes to handle "wchar_t*" strings on Windows platform to make syntax more transparent to end-user.
N/A
No tags attached.
related to 0026380closed bugmaster Open CASCADE OSD_SharedLibrary - handle UTF-8 file paths 
related to 0025308new pdn Open CASCADE TCollection_ExtendedString, NCollection_String - merge classes for string management 
related to 0027844closed bugmaster Open CASCADE Foundation Classes - remove method OSD_File::Print() 
related to 0027880closed bugmaster Open CASCADE Samples - fix handling of Unicode paths within MFC import/export sample 
related to 0027960closed apn Community Configuration - fix compilation of OSD_Directory with MinGW-w64 
child of 0022484closed bugmaster Open CASCADE UNICODE characters support. 
Issue History
2016-09-04 12:16kgvNew Issue
2016-09-04 12:16kgvAssigned To => abv
2016-09-04 14:33gitNote Added: 0057448
2016-09-04 14:33kgvNote Added: 0057449
2016-09-04 14:33kgvStatusnew => resolved
2016-09-04 14:34kgvRelationship addedchild of 0022484
2016-09-04 14:35kgvRelationship addedrelated to 0026380
2016-09-04 14:35kgvRelationship addedrelated to 0025308
2016-09-04 14:36kgvRelationship addedrelated to 0027779
2016-09-06 13:54abvNote Added: 0057512
2016-09-06 13:54abvAssigned Toabv => kgv
2016-09-06 13:54abvStatusresolved => assigned
2016-09-06 14:34kgvNote Added: 0057516
2016-09-06 14:38kgvRelationship addedrelated to 0027844
2016-09-06 19:01gitNote Added: 0057532
2016-09-06 19:03gitNote Added: 0057533
2016-09-06 19:04kgvNote Added: 0057534
2016-09-06 19:04kgvAssigned Tokgv => abv
2016-09-06 19:04kgvStatusassigned => resolved
2016-09-07 14:20abvNote Added: 0057561
2016-09-07 14:20abvAssigned Toabv => bugmaster
2016-09-07 14:20abvStatusresolved => reviewed
2016-09-07 15:55apvAssigned Tobugmaster => apv
2016-09-08 09:43apvNote Added: 0057580
2016-09-08 09:43apvAssigned Toapv => kgv
2016-09-08 09:43apvStatusreviewed => assigned
2016-09-08 09:44apvNote Added: 0057581
2016-09-08 10:13gitNote Added: 0057582
2016-09-08 10:49gitNote Added: 0057585
2016-09-08 10:50gitNote Added: 0057586
2016-09-08 10:55kgvNote Added: 0057587
2016-09-08 10:55kgvAssigned Tokgv => bugmaster
2016-09-08 10:55kgvStatusassigned => resolved
2016-09-08 10:55kgvStatusresolved => reviewed
2016-09-08 10:55kgvNote Edited: 0057587bug_revision_view_page.php?bugnote_id=57587#r14602
2016-09-09 09:08apvAssigned Tobugmaster => apv
2016-09-09 11:05apvNote Added: 0057625
2016-09-09 11:05apvAssigned Toapv => kgv
2016-09-09 11:05apvStatusreviewed => feedback
2016-09-09 11:19gitNote Added: 0057628
2016-09-09 11:20kgvNote Added: 0057629
2016-09-09 11:20kgvAssigned Tokgv => bugmaster
2016-09-09 11:20kgvStatusfeedback => reviewed
2016-09-09 11:25apvAssigned Tobugmaster => apv
2016-09-12 13:02apvTest case number => Not needed
2016-09-12 13:45apvNote Added: 0057682
2016-09-12 13:45apvAssigned Toapv => kgv
2016-09-12 13:45apvStatusreviewed => assigned
2016-09-12 13:46apvNote Added: 0057683
2016-09-12 15:46gitNote Added: 0057688
2016-09-12 15:47kgvNote Added: 0057689
2016-09-12 15:47kgvAssigned Tokgv => bugmaster
2016-09-12 15:47kgvStatusassigned => resolved
2016-09-12 15:47kgvStatusresolved => reviewed
2016-09-12 16:13apvAssigned Tobugmaster => apv
2016-09-13 13:48apvNote Added: 0057738
2016-09-13 13:48apvAssigned Toapv => kgv
2016-09-13 13:48apvStatusreviewed => assigned
2016-09-13 13:49apvNote Added: 0057739
2016-09-13 14:13gitNote Added: 0057740
2016-09-13 14:14kgvNote Added: 0057741
2016-09-13 14:14kgvAssigned Tokgv => bugmaster
2016-09-13 14:14kgvStatusassigned => resolved
2016-09-13 14:14kgvStatusresolved => reviewed
2016-09-13 14:16gitNote Added: 0057742
2016-09-13 14:51apvAssigned Tobugmaster => apv
2016-09-14 13:12apvNote Added: 0057772
2016-09-14 13:12apvAssigned Toapv => bugmaster
2016-09-14 13:12apvStatusreviewed => tested
2016-09-16 09:43kgvChangeset attached => occt master fb0b0531
2016-09-16 09:43kgvAssigned Tobugmaster => kgv
2016-09-16 09:43kgvStatustested => verified
2016-09-16 09:43kgvResolutionopen => fixed
2016-09-19 11:47kgvRelationship addedrelated to 0027880
2016-10-13 14:33BenjaminBihlerNote Added: 0058663
2016-10-13 14:39kgvNote Added: 0058664
2016-10-13 14:50BenjaminBihlerRelationship addedrelated to 0027960
2016-10-28 21:45gitNote Added: 0059505
2016-10-28 21:48gitNote Added: 0059581
2016-10-28 21:48gitNote Added: 0059582
2016-12-09 16:30aivStatusverified => closed
2016-12-09 16:39aivFixed in Version => 7.1.0
2019-03-01 17:10kgvRelationship addedparent of 0030536

Notes
(0057448)
git   
2016-09-04 14:33   
Branch CR27838 has been created by kgv.

SHA-1: 4cb3a5dbb128e8e6102d7bfbc43d64f2c0c35dd5


Detailed log of new commits:

Author: kgv
Date: Sun Sep 4 14:31:47 2016 +0300

    0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString
    
    TCollection_ExtendedString/TCollection_AsciiString description
    has been updated to reflect usage of this classes for Unicode strings.
    
    TCollection_ExtendedString now defines constructor taking wchar_t* (all platforms)
    and method ::ToWideString() returning wchar_t* (Windows only).
    TCollection_AsciiString now defines constructor taking wchar_t*.
    
    TCollection_ExtendedString/TCollection_AsciiString now defines
    auxiliary methods ::StartsWith() and ::EndsWith().
    
    TCollection_ExtendedString internals has been updated to eliminate
    duplicated code for converting between UTF-16 and UTF-8.
    The internal method Allocate() in TCollection_ExtendedString.cxx
    now automatically NULL-terminate allocated buffer.
    
    Code has been cleaned up from redundant explicit conversions to wchar_t*.
    Global method OSD_OpenStream()/OSD_OpenFileBuf() have been replaced
    by C++ template to eliminate copy-paste for different STL collections.
    
    OSD_SharedLibrary now uses wide-char system API call LoadLibraryExW()
    on Windows for consistency.
(0057449)
kgv   
2016-09-04 14:33   
Patch is ready for review.
(0057512)
abv   
2016-09-06 13:54   
Some remarks:

1. In OSD_File.cxx, 2791:

- if ( !OpenPrinter ( Standard_PCharacter(pName), &hPrinter, NULL ) ) {
+ if ( !OpenPrinterW ((wchar_t* )pName, &hPrinter, NULL ) ) {

this conversion is suspicious since pName is still PCharacter, i.e. char *

2. In TCollection_ExtendedString, methods Re/Allocate():
- perhaps it is better to use size in characters, not in bytes
- it is somehow innatural to delegate the task of setting terminating null symbol to allocation function... the code would be less confusing if null was added explicitly whenever string is filled

3. In TCollection_ExtendedString, run-time "switch (sizeof(Standard_WideChar))" should be replaced by compile-time equivalent, e.g. using template specializations. For an example of the technique, see OCCT_InverseSizeSpecialized in FSD_BinaryFile.cxx

4. In TCollection_ExtendedString.hxx, I suggest removing "has been decided to " and replacing "might not" by "can miss". Idem replace "NULL-terminated symbol" by "terminating NULL symbol"
(0057516)
kgv   
2016-09-06 14:34   
-  if (  !OpenPrinter ( Standard_PCharacter(pName), &hPrinter, NULL )  ) {
+  if (  !OpenPrinterW ((wchar_t* )pName, &hPrinter, NULL )  ) {

Should be const_cast<wchar_t*>
(0057532)
git   
2016-09-06 19:01   
Branch CR27838 has been updated by kgv.

SHA-1: 2c65aca3e8d19ff54774fed1f94b39adfb7438e0


Detailed log of new commits:

Author: kgv
Date: Tue Sep 6 19:03:46 2016 +0300

    apply remarks

(0057533)
git   
2016-09-06 19:03   
Branch CR27838_1 has been created by kgv.

SHA-1: 6d82d9da31cb22e0bf588f20292ff73112a194fb


Detailed log of new commits:

Author: kgv
Date: Sun Sep 4 14:31:47 2016 +0300

    0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString
    
    TCollection_ExtendedString/TCollection_AsciiString description
    has been updated to reflect usage of this classes for Unicode strings.
    
    TCollection_ExtendedString now defines constructor taking wchar_t* (all platforms)
    and method ::ToWideString() returning wchar_t* (Windows only).
    TCollection_AsciiString now defines constructor taking wchar_t*.
    
    TCollection_ExtendedString/TCollection_AsciiString now defines
    auxiliary methods ::StartsWith() and ::EndsWith().
    
    TCollection_ExtendedString internals has been updated to eliminate
    duplicated code for converting between UTF-16 and UTF-8.
    
    Code has been cleaned up from redundant explicit conversions to wchar_t*.
    Global method OSD_OpenStream()/OSD_OpenFileBuf() have been replaced
    by C++ template to eliminate copy-paste for different STL collections.
    
    OSD_SharedLibrary now uses wide-char system API call LoadLibraryExW()
    on Windows for consistency.
(0057534)
kgv   
2016-09-06 19:04   
Remarks have been applied.
(0057561)
abv   
2016-09-07 14:20   
No remarks, please test
(0057580)
apv   
2016-09-08 09:43   
Dear BugMaster,

Branch CR27838_1 from occt git-repository (and master from products git-repository) was compiled on Linux, MacOS and Windows platforms.
SHA-1: 6d82d9da31cb22e0bf588f20292ff73112a194fb

Number of compiler warnings:
occt component:
   Linux: Not available
   Windows: 0 (0 on master)
   MasOS: Not available
products component:
   Linux: Not available
   Windows: 0
   MacOS: Not available

During building of branch CR27838_1 from occt git-repository and master from products git-repository following compilation errors have been detected
On Linux:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_1-master/job/CR27838_1-master-OCCT-Debian70-64-opt-compile/1/parsed_console/ [^]
On MacOS:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_1-master/job/CR27838_1-master-OCCT-MacOS-opt-compile/1/parsed_console/ [^]

Moreover compilation error has been detected during building of
Qt OCCT samples on Windows:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_1-master/job/CR27838_1-master-OCCT-Windows-64-VC10-qt-samples-windows/1/parsed_console/ [^]
Qt Products samples on Windows:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_1-master/job/CR27838_1-master-Products-Windows-64-VC10-qt-samples-windows/1/parsed_console/ [^]
(0057581)
apv   
2016-09-08 09:44   
Dear Kirill,

Branch CR27838_1 has been rejected due to:
- compilation errors
(0057582)
git   
2016-09-08 10:13   
Branch CR27838_1 has been updated by kgv.

SHA-1: acbb3bb805272d0853acfe46f11d3f4952c1fc66


Detailed log of new commits:

Author: kgv
Date: Thu Sep 8 10:16:04 2016 +0300

    fix compilation issues

(0057585)
git   
2016-09-08 10:49   
Branch CR27838_1 has been updated forcibly by kgv.

SHA-1: 88bb3801663a0d1aa567cf86b0d7857a00356607
(0057586)
git   
2016-09-08 10:50   
Branch CR27838_2 has been created by kgv.

SHA-1: 53d3d3359f8f31600aa3387dab8fe29f7fdb9240


Detailed log of new commits:

Author: kgv
Date: Sun Sep 4 14:31:47 2016 +0300

    0027838: Foundation Classes - support wchar_t* input within TCollection_AsciiString and TCollection_ExtendedString
    
    TCollection_ExtendedString/TCollection_AsciiString description
    has been updated to reflect usage of this classes for Unicode strings.
    
    TCollection_ExtendedString now defines constructor taking wchar_t* (all platforms)
    and method ::ToWideString() returning wchar_t* (Windows only).
    TCollection_AsciiString now defines constructor taking wchar_t*.
    
    TCollection_ExtendedString/TCollection_AsciiString now defines
    auxiliary methods ::StartsWith() and ::EndsWith().
    
    TCollection_ExtendedString internals has been updated to eliminate
    duplicated code for converting between UTF-16 and UTF-8.
    
    Code has been cleaned up from redundant explicit conversions to wchar_t*.
    Global method OSD_OpenStream()/OSD_OpenFileBuf() have been replaced
    by C++ template to eliminate copy-paste for different STL collections.
    
    OSD_SharedLibrary now uses wide-char system API call LoadLibraryExW()
    on Windows for consistency.
(0057587)
kgv   
2016-09-08 10:55   
Please check updated patch in branch CR27838_2 which should workaround awkward compatibility issues.

It seems that compilation issues in Qt samples caused by using deprecated msvc compilation flag "-Zc:wchar_t-" which makes wchar_t as not built-in type.
This issue has been fixed in Qt5, so I suppose old Qt4 is used for building samples:
https://bugreports.qt.io/browse/QTBUG-9617 [^]

(0057625)
apv   
2016-09-09 11:05   
Dear Kirill,

Please, rebase CR27838_2 on the current master. Thank you in advance.
(0057628)
git   
2016-09-09 11:19   
Branch CR27838_2 has been updated forcibly by kgv.

SHA-1: baa5f5802bb3428537160deb783394acdf599958
(0057629)
kgv   
2016-09-09 11:20   
Branch has been rebased.
(0057682)
apv   
2016-09-12 13:45   
Dear BugMaster,

Branch CR27838_2 from occt git-repository (and master from products git-repository) was compiled on Linux, MacOS and Windows platforms and tested.
SHA-1: baa5f5802bb3428537160deb783394acdf599958

Number of compiler warnings:
occt component:
   Linux: 0 (0 on master)
   Windows: 0 (0 on master)
   MasOS: 1 (0 on master)
products component:
   Linux: 64
   Windows: 0
   MacOS: 1129
New warnings have been detected during OCCT component building
on MacOS:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_2-master/job/CR27838_2-master-OCCT-MacOS-opt-compile/1/warnings7Result/ [^]

Regressions/Differences:
Not detected

Testing cases:
Not needed

Testing on Linux:
Total MEMORY difference: 89720825 / 90408349 [-0.76%]
Total CPU difference: 19452.11000000009 / 19445.350000000133 [+0.03%]

Testing on Windows:
Total MEMORY difference: 57157566 / 57172575 [-0.03%]
Total CPU difference: 18246.641364898747 / 18100.047225198865 [+0.81%]
(0057683)
apv   
2016-09-12 13:46   
Dear Kirill,

Branch CR27838_2 has been rejected due to:
- additional warnings
(0057688)
git   
2016-09-12 15:46   
Branch CR27838_2 has been updated by kgv.

SHA-1: 5c146df1a4bbdcd1d0d6990998efd7468f1d9b22


Detailed log of new commits:

Author: kgv
Date: Mon Sep 12 15:46:37 2016 +0300

    mark fromWideString() as inline in TCollection_ExtendedString.cxx

(0057689)
kgv   
2016-09-12 15:47   
Please re-check compilation of updated patch on OS X, regression testing is not needed.
(0057738)
apv   
2016-09-13 13:48   
Dear BugMaster,

Branch CR27838_2 from occt git-repository (and master from products git-repository) was compiled on Linux, MacOS and Windows platforms and tested.
SHA-1: 5c146df1a4bbdcd1d0d6990998efd7468f1d9b22

Number of compiler warnings:
occt component:
   Linux: 0 (0 on master)
   Windows: 0 (0 on master)
   MasOS: 1 (0 on master)
products component:
   Linux: 64
   Windows: 0
   MacOS: 1134
New warnings have been detected during OCCT component building
on MacOS:
http://jenkins-test-05.nnov.opencascade.com/view/CR27838_2-master/job/CR27838_2-master-OCCT-MacOS-opt-compile/1/warnings7Result/ [^]

Regressions/Differences:
Not detected

Testing cases:
Not needed

Testing on Linux:
Total MEMORY difference: 89462472 / 90412700 [-1.05%]
Total CPU difference: 19406.839999999967 / 19445.440000000133 [-0.20%]

Testing on Windows:
Total MEMORY difference: 57157530 / 57172575 [-0.03%]
Total CPU difference: 18357.324074398828 / 18100.047225198865 [+1.42%]
(0057739)
apv   
2016-09-13 13:49   
Dear Kirill,

Branch CR27838_2 has been rejected due to:
- additional warnings
(0057740)
git   
2016-09-13 14:13   
Branch CR27838_2 has been updated by kgv.

SHA-1: de5403ed5b30d46dd74f2f33a9a00bc1242caac8


Detailed log of new commits:

Author: kgv
Date: Tue Sep 13 14:12:59 2016 +0300

    fromWideString() in TCollection_ExtendedString.cxx has been marked with Standard_UNUSED.

(0057741)
kgv   
2016-09-13 14:14   
Please try compiling updated patch, regression testing is not needed.
(0057742)
git   
2016-09-13 14:16   
Branch CR27838_2 has been updated forcibly by kgv.

SHA-1: bc2f92295c3b32e119a15a33a5329b92d151c701
(0057772)
apv   
2016-09-14 13:12   
Dear BugMaster,

Branch CR27838_2 from occt git-repository (and master from products git-repository) was compiled on Linux, MacOS and Windows platforms and tested.
SHA-1: bc2f92295c3b32e119a15a33a5329b92d151c701

Number of compiler warnings:
occt component:
   Linux: 0 (0 on master)
   Windows: 0 (0 on master)
   MasOS: 0 (0 on master)
products component:
   Linux: 64
   Windows: 0
   MacOS: 1139

Regressions/Differences:
Not detected

Testing cases:
Not needed

Testing on Linux:
Total MEMORY difference: 89615541 / 90384642 [-0.85%]
Total CPU difference: 19413.290000000125 / 19445.440000000133 [-0.17%]

Testing on Windows:
Total MEMORY difference: 57157795 / 57172575 [-0.03%]
Total CPU difference: 18255.861023998856 / 18100.047225198865 [+0.86%]
(0058663)
BenjaminBihler   
2016-10-13 14:33   
From commit fb0b05319f52b12ba581e70f561234fe582c3dc3 on, OCCT cannot be compiled with MinGw-w64 anymore. Should I create a new issue for that?
(0058664)
kgv   
2016-10-13 14:39   
> Should I create a new issue for that?
Yes, please.
(0059505)
git   
2016-10-28 21:45   
Branch CR27838_2 has been deleted by kgv.

SHA-1: bc2f92295c3b32e119a15a33a5329b92d151c701
(0059581)
git   
2016-10-28 21:48   
Branch CR27838 has been deleted by kgv.

SHA-1: 2c65aca3e8d19ff54774fed1f94b39adfb7438e0
(0059582)
git   
2016-10-28 21:48   
Branch CR27838_1 has been deleted by kgv.

SHA-1: 88bb3801663a0d1aa567cf86b0d7857a00356607