MantisBT
Mantis Bug Tracker Workflow

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0025748Open CASCADE[OCCT] OCCT:Foundation Classespublic2015-01-23 19:272019-05-20 09:25
Reporterabv 
Assigned Toabv 
PrioritynormalSeverityminor 
StatusresolvedResolutionopen 
PlatformOSOS Version
Product Version 
Target Version[OCCT] 7.4.0*Fixed in Version 
Summary0025748: Foundation Classes - Parallel version of progress indicator
DescriptionIn order to provide progress indication functionality in algorithms working in parallel mode (BOP, BRepMesh), it is necessary to make progress indicator (classes Message_Progress*) capable of working in parallel threads.
Steps To Reproduceperf fclasses progress_par
TagsNo tags attached.
Test case number
Attached Files

- Relationships
related to 0025113resolvedmsv Community Progress indication and user break functionality for BRepMesh component 
related to 0030697newkgv Open CASCADE Draw Harness - Draw_Printer should not be set to Message::DefaultMessenger() by default 

-  Notes
(0076845)
git (administrator)
2018-06-20 11:54

Branch CR25748 has been created by msv.

SHA-1: 21a68e71d603039611304c2a68b385c37287b968


Detailed log of new commits:

Author: msv
Date: Wed Jun 20 11:54:41 2018 +0300

    0025748: Parallel version of progress indicator
    
    The fix revises the progress indication mechanism. Now, the class Handle(Message_ProgressIndicator) performs the only function of calling back to user application. It just accumulates the progress provided by progress scopes.
    
    The new class Message_ProgressScope serves to represent a scope of execution. The new instance of it is created to provide progress advancement of a new coming operation. The scopes are nested to each other to reflect the nested nature of operations.
    
    All codes involving progress indication have been updated to new API.
(0076866)
git (administrator)
2018-06-21 13:05

Branch CR25748 has been updated by msv.

SHA-1: df083f1fd8a7647b189776260534167270ec0f0d


Detailed log of new commits:

Author: msv
Date: Thu Jun 21 13:04:43 2018 +0300

    # Improved version of indicator and scope.
    # More codes are moved to new API.

(0077434)
git (administrator)
2018-07-10 20:53

Branch CR25748_1 has been created by msv.

SHA-1: c89b4b85c527c84e723f42e8d8079a9f5d2d6c8a


Detailed log of new commits:

Author: msv
Date: Wed Jun 20 11:54:41 2018 +0300

    0025748: Parallel version of progress indicator
    
    The fix revises the progress indication mechanism. Now, the class Handle(Message_ProgressIndicator) performs the only function of calling back to user application. It just accumulates the progress provided by progress scopes. The counter is protected by mutex for thread-safety.
    
    The new class Message_ProgressScope serves to represent a scope of execution. The new instance of it is created to provide progress advancement of a new coming operation. The scopes are nested to each other to reflect the nested nature of operations. Each instance of progress scope can call the method Increment of the progress indicator concurrently with other scopes, which make the work of the overall mechanism thread-safe.
    
    All OCCT algorithms involving progress indication have been updated to new API.
    
    Improvements in Draw_ProgressIndicator:
    
    - Console mode has been added in order to make possible to put the progress into cout instead of draw interpreter.
    - Treatment of Ctrl-Break signal has been added. Now any operation can be aborted by Ctrl-C or Ctrl-Break keystroke.
(0077439)
git (administrator)
2018-07-10 23:14

Branch CR25748_1 has been updated by msv.

SHA-1: 5d09214f94d19fd57f2283fd9d11646b7b7ba9db


Detailed log of new commits:

Author: msv
Date: Tue Jul 10 23:13:53 2018 +0300

    #correct compilation error

(0077529)
git (administrator)
2018-07-12 15:20

Branch CR25748_1 has been updated forcibly by msv.

SHA-1: 4fd60394052d7ea81389416d68011bdcc46cc423
(0077561)
git (administrator)
2018-07-13 08:40

Branch CR25748_1 has been updated forcibly by msv.

SHA-1: 63d200d4d3c1e03c1ba0062ed899bcc79d53903e
(0077588)
git (administrator)
2018-07-13 18:19

Branch CR25748_1 has been updated forcibly by msv.

SHA-1: 05365f941b15c0e2714dd705221d820a28243e81
(0077652)
git (administrator)
2018-07-16 12:52

Branch CR25748_2 has been created by msv.

SHA-1: 87033320e01ee9f7a56c73bc59153afc0e7c356d


Detailed log of new commits:

Author: msv
Date: Wed Jun 20 11:54:41 2018 +0300

    0025748: Parallel version of progress indicator
    
    The fix revises the progress indication mechanism. Now, the class Handle(Message_ProgressIndicator) performs the only function of calling back to user application. It just accumulates the progress provided by progress scopes. The counter is protected by mutex for thread-safety.
    
    The new class Message_ProgressScope serves to represent a scope of execution. The new instance of it is created to provide progress advancement of a new coming operation. The scopes are nested to each other to reflect the nested nature of operations. Each instance of progress scope can call the method Increment of the progress indicator concurrently with other scopes, which make the work of the overall mechanism thread-safe.
    
    All OCCT algorithms involving progress indication have been updated to new API.
    
    Improvements in Draw_ProgressIndicator:
    
    - Console mode has been added in order to make possible to put the progress into cout instead of draw interpreter.
    - Treatment of Ctrl-Break signal has been added. Now any operation can be aborted by Ctrl-C or Ctrl-Break keystroke.
(0077747)
git (administrator)
2018-07-17 17:33

Branch CR25748_2 has been updated forcibly by msv.

SHA-1: c201ed8ca8bae7307ecd21d4c9e70541b3412452
(0077749)
msv (developer)
2018-07-17 17:42

Please review the branches in occt and products.
Jenkins tests are now executed http://jenkins-test-11.nnov.opencascade.com:8080/view/CR25748_2-CR25748_1-msv/view/COMPARE/ [^]
(0084216)
git (administrator)
2019-05-07 11:19

Branch CR25748_3 has been created by msv.

SHA-1: 92d57ddcdcd47bfce8ade716a42d2eda07998c42


Detailed log of new commits:

Author: msv
Date: Wed Jun 20 11:54:41 2018 +0300

    0025748: Parallel version of progress indicator
    
    The fix revises the progress indication mechanism. Now, the class Handle(Message_ProgressIndicator) performs the only function of calling back to user application. It just accumulates the progress provided by progress scopes. The counter is protected by mutex for thread-safety.
    
    The new class Message_ProgressScope serves to represent a scope of execution. The new instance of it is created to provide progress advancement of a new coming operation. The scopes are nested to each other to reflect the nested nature of operations. Each instance of progress scope can call the method Increment of the progress indicator concurrently with other scopes, which makes the work of the overall mechanism thread-safe.
    
    All OCCT algorithms involving progress indication have been updated to new API.
    
    Improvements in Draw_ProgressIndicator:
    
    - Console mode has been added in order to make possible to put the progress into cout instead of draw interpreter.
    - Treatment of Ctrl-Break signal has been added. Now any operation can be aborted by Ctrl-C or Ctrl-Break keystroke.
(0084232)
msv (developer)
2019-05-07 17:24

Please review.
(0084233)
msv (developer)
2019-05-07 17:25

Tests results http://jenkins-test-12.nnov.opencascade.com/view/CR25748-CR25748-MSV/view/COMPARE/ [^]
(0084293)
kgv (developer)
2019-05-12 17:26

   // Print textual progress info
-  if ( myTextMode )
+  if (myConsoleMode)
+    cout << text << endl;
+  else if (myTextMode)
     Message::DefaultMessenger()->Send (text, Message_Info);

Do we have any use cases, where printing progress to Draw_Printer instead of std::cout will be of any use apart from synthetic test case bugs/fclasses/bug28478?

+  if (OSD_Thread::Current() != myThreadId)
+    // avoid showing in another thread

What are consequences of this check - no any progress in case if algorithm performs all jobs in non-main thread (or unpredictable progress from main thread when using OSD_Parallel::For())?
I suppose that the main purpose for this check is a crash on using Tcl-interpretor from multiple threads (which is not that a blocking issue in case of std::cout).

+  const Message_ProgressScope& GetRootScope() const
+  {
...
+  //! Returns the name of the scope (may be null)
+  Standard_CString GetName() const
+
+  //! Returns parent scope (null for top-level scope)
+  const Message_ProgressScope* GetParent() const
+
+  //! Returns the maximal value of progress in this scope
+  Standard_Real GetMax() const
+
+  //! Returns the current value of progress in this scope
+  Standard_Real GetValue() const
+
+  //! Get start position of this scope on total progress scale
+  Standard_Real GetStart() const
+
+  //! Get end position of this scope on total progress scale
+  Standard_Real GetEnd() const

Shouldn't this be RootScope()/End()/Parent()/Start()?

+  //! Sets the name of the scope.
+  Standard_EXPORT void SetName(Standard_CString theName);
..
+
+  //! Creates a sub-scope covering current step of the parent scope, 
+  //! and selects parameters of the current scale. The top most scope 
+  //! must be got using the method GetRoopScope() of Message_ProgressIndicator.
+  Standard_EXPORT Message_ProgressScope(const Message_ProgressScope& theParent,
+                                        Standard_CString theName,
+                                        Standard_Real theMin = 0, Standard_Real theMax = 100,
+                                        Standard_Real theStep = 1, Standard_Boolean isInfinite = false)

...
+  Standard_CString   myName;        //!< Name of the operation being done in this scope, or null


This should be well-documented that the given name will NOT be copied, hence application should never pass string from a temporary variable.

+public: //! @name Destrucion, allocation

Destrucion

+  mutable Standard_Real myPosBySS;  //!< Current position reserved by sub-scopes
+  mutable Standard_Integer myNbChild; //!< Number of children - used to control code consistency


I suppose that the main reason for these fields to be mutable is to pass Message_ProgressScope as const reference, which in turn is done to allow defining default argument value "const Message_ProgressScope& theProgress = Message_ProgressScope()". I think its worth to be documented.
(0084294)
kgv (developer)
2019-05-12 22:37
edited on: 2019-05-12 22:38

Example of usage in parallel process:
+//!
+//! @code{.cpp}
+//! struct Task
+//! {
+//!   Data* Data;
+//!   Message_ProgressScope Progr;
+//!
+//!   Task(const Data& theData, Message_ProgressScope& theProgr)
+//!     : Data(theData), Progr(theProgr, NULL, 0, 1, 1) {}
+//!
+//!   void operator()(Task& theTask) const
+//!   {
+//!     if (theTask.Progr.More())
+//!     {
+//!       // ... process data
+//!       theTask.Progr.Next();
+//!     }
+//!   }
+//! };
+//! {
+//!   std::vector<Data*> aData; // somehow got data
+//!   std::vector<Task> aTasks;
+//!
+//!   Message_ProgressScope aPS(aRootPS, "Data processing", 0, aData.size(), 1);
+//!   for (Standard_Integer i = 0; i < aData.size(); ++i)
+//!     aTasks.push_back(Task(aData[i], aPS));
+//!   
+//!   OSD_Parallel::ForEach(aTasks.begin(), aTasks.end(), Task());
+//! }
+//! @endcode
...
+void Message_ProgressIndicator::Increment(const Standard_Real theStep,
+                                          const Message_ProgressScope& theScope)
 {
+  Standard_Mutex::Sentry aSentry(myMutex);
+  myPosition = Min(myPosition + theStep, 1.);
+  Show(Standard_False, theScope);

Since proposed API requires explicit barrier at spawning and passing to multiple threads, I would expect this mutex NOT to be used by default - it is unneeded by majority of algorithms / waste of time. Instead, the parallelization can be assigned to Progress Indicator first time algorithm is about to spawn working threads and forking progress to them.

The new parallelization design relies on the idea that enumerated entities are of significant size to spawn individual Progress Sentry for each such task - the main thread-safety idea is possibility to sub-divide Progress Sentry further within nested algorithms. But this mechanism cannot be used for OSD_Parallel::For() on big amount of small elements due to extra overhead of individual Progress Sentries and serialization via global mutex in Progress Indicator itself. For such small tasks with unpredictable balancing (within perfect balancing algorithm might spawn Progress Sentry per working thread by equally splitting full range), thread-safe updating of single shared Progress Sentry will be still responsibility of local to algorithm code - in exactly the same way, as existing Progress Indicator API.

(0084407)
git (administrator)
2019-05-16 17:47

Branch CR25748_3 has been updated forcibly by msv.

SHA-1: 864a80af9414e0b3b351b69e30745b83899f2fa2
(0084411)
git (administrator)
2019-05-17 09:31

Branch CR25748_3 has been updated forcibly by msv.

SHA-1: 2f2d36be0b37a6ea656dadfd077756dd77f4fa17
(0084436)
msv (developer)
2019-05-17 22:16

I have updated the code according to KGV's remarks.
New test results are here: http://jenkins-test-12.nnov.opencascade.com/view/CR25748-CR25748-MSV2/view/COMPARE/ [^]
(0084469)
msv (developer)
2019-05-20 09:25

Dear Andrey, please review the current version.

- Issue History
Date Modified Username Field Change
2015-01-23 19:27 abv New Issue
2015-01-23 19:27 abv Assigned To => abv
2015-01-23 19:27 abv Relationship added related to 0025113
2015-03-27 21:35 abv Target Version 6.9.0 => 7.1.0
2016-11-01 06:41 abv Target Version 7.1.0 => 7.2.0
2017-07-27 09:43 abv Target Version 7.2.0 => 7.4.0*
2018-06-19 15:49 msv Relationship added related to 0029872
2018-06-20 11:54 git Note Added: 0076845
2018-06-20 11:55 msv Assigned To abv => msv
2018-06-20 11:55 msv Status new => assigned
2018-06-20 11:55 kgv Summary Parallel version of progress indicator => Foundation Classes - Parallel version of progress indicator
2018-06-21 13:05 git Note Added: 0076866
2018-07-10 20:53 git Note Added: 0077434
2018-07-10 23:14 git Note Added: 0077439
2018-07-12 15:20 git Note Added: 0077529
2018-07-13 08:40 git Note Added: 0077561
2018-07-13 18:19 git Note Added: 0077588
2018-07-16 12:52 git Note Added: 0077652
2018-07-17 17:33 git Note Added: 0077747
2018-07-17 17:42 msv Note Added: 0077749
2018-07-17 17:42 msv Assigned To msv => abv
2018-07-17 17:42 msv Status assigned => resolved
2018-07-17 17:42 msv Steps to Reproduce Updated View Revisions
2019-05-07 09:03 msv Assigned To abv => msv
2019-05-07 09:03 msv Status resolved => assigned
2019-05-07 11:19 git Note Added: 0084216
2019-05-07 17:24 msv Note Added: 0084232
2019-05-07 17:24 msv Assigned To msv => kgv
2019-05-07 17:24 msv Status assigned => resolved
2019-05-07 17:25 msv Note Added: 0084233
2019-05-12 16:46 kgv Relationship added related to 0030697
2019-05-12 17:26 kgv Note Added: 0084293
2019-05-12 22:37 kgv Note Added: 0084294
2019-05-12 22:38 kgv Note Edited: 0084294 View Revisions
2019-05-13 12:25 kgv Assigned To kgv => abv
2019-05-16 17:47 git Note Added: 0084407
2019-05-17 09:31 git Note Added: 0084411
2019-05-17 22:16 msv Note Added: 0084436
2019-05-20 09:25 msv Note Added: 0084469


Copyright © 2000 - 2019 MantisBT Team
Powered by Mantis Bugtracker