Skip to content

MINIFICPP-2746 Allow python processors to run in parallel#2135

Open
lordgamez wants to merge 1 commit intoapache:mainfrom
lordgamez:MINIFICPP-2746
Open

MINIFICPP-2746 Allow python processors to run in parallel#2135
lordgamez wants to merge 1 commit intoapache:mainfrom
lordgamez:MINIFICPP-2746

Conversation

@lordgamez
Copy link
Contributor

https://issues.apache.org/jira/browse/MINIFICPP-2746


Thank you for submitting a contribution to Apache NiFi - MiNiFi C++.

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

For all changes:

  • Is there a JIRA ticket associated with this PR? Is it referenced
    in the commit message?

  • Does your PR title start with MINIFICPP-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

  • Has your PR been rebased against the latest commit within the target branch (typically main)?

  • Is your initial contribution a single, squashed commit?

For code changes:

  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE file?
  • If applicable, have you updated the NOTICE file?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered?

Note:

Please ensure that once the PR is submitted, you check GitHub Actions CI results for build issues and submit an update to your PR as soon as possible.


### Setting processor to be single threaded

One feature that is currently only available in MiNiFi C++ is the ability to set a python processor to be single threaded. This is not yet available in NiFi's API, but similarly to Nifi's `@TriggerSerially` annotation, in MiNiFi C++ the `@trigger_serially` decorator can be used to make a processor single threaded. Setting this will make sure that only one thread executes the processor, and setting the max concurrent tasks to more than 1 in the flow configuration will not have any effect.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
One feature that is currently only available in MiNiFi C++ is the ability to set a python processor to be single threaded. This is not yet available in NiFi's API, but similarly to Nifi's `@TriggerSerially` annotation, in MiNiFi C++ the `@trigger_serially` decorator can be used to make a processor single threaded. Setting this will make sure that only one thread executes the processor, and setting the max concurrent tasks to more than 1 in the flow configuration will not have any effect.
One feature that is currently only available in MiNiFi C++ is the ability to set a python processor to be single threaded. This is not yet available in NiFi's Python API, but similarly to Nifi's `@TriggerSerially` Java API annotation, in MiNiFi C++ the `@trigger_serially` decorator can be used to make a processor single threaded. Setting this will make sure that only one thread executes the processor, and setting the max concurrent tasks to more than 1 in the flow configuration will not have any effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants