Support for uvicorn with multiple workers

[jennykupzig]

Is your feature request related to a problem?

Uvicorn enables process management itself (Uvicorn 0.30.0 Release), claiming to replace Gunicorn entirely. However, opentelementy-autoinstrumenation for python does only support either:

• multiple workers with gunicorn
• one worker with uvicorn.

When I have tried to run my application with ucivorn and multiple workers, the logs just don't show up in Grafana.

Describe the solution you'd like

An opentelemetry support for multiple workers for uvicorn would be great.

Describe alternatives you've considered

• Using gunicorn with multiple workers:
  Using gunicorn, we have problems when starting a subprocess, which we do not have when usong uvicorn. Therefore, we would really like to integrate uvicorn with mutliple workers instead of gunicorn with multiple workers.

• Using uvicorn with one worker:
  As we run our application in production we need to be able to scale our application. Therefore, using only one worker is not suitable.

Additional Context

Our current installation of opentelemetry is:

ARG version_nr="0.60b0"
RUN pip install opentelemetry-distro[otlp]==${version_nr}
RUN pip install opentelemetry-instrumentation-system-metrics==${version_nr}
RUN pip install opentelemetry-instrumentation-asyncio==${version_nr}
RUN pip install opentelemetry-instrumentation-logging==${version_nr}
RUN pip install opentelemetry-instrumentation-httpx==${version_nr}
RUN pip install opentelemetry-instrumentation-sqlalchemy==${version_nr}
RUN pip install opentelemetry-instrumentation-fastapi==${version_nr}
RUN pip install opentelemetry-instrumentation-requests==${version_nr}

Would you like to implement a fix?

None

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[jennykupzig]

Is there any update on this?

[herin049]

@jennykupzig One approach that should work is using the opentelemetry-sitecustomize package. The only caveat is that this approach will instrument ALL Python processes created by the environment this package is installed in. I would need to look at how Uvicorn implements process management to see what changes would need to be made to enable auto-instrumentation with the traditional CLI method.

[jennykupzig]

Thank you for your reply @herin049!

Regarding the note on the page of the opentelemetry-sitecustomize package 'If you control the start command of your application, preferably use the opentelemetry-instrument CLI.', I would prefer a solution for uvicorn that enables auto-instrumentation using the traditional CLI method. However, I would be very interested if you could provide more information on why the traditional CLI method is preferred over the opentelemetry-sitecustomize method.

I would greatly appreciate any updates on this.

[xrmx]

An alternative would be:
https://opentelemetry.io/docs/zero-code/python/troubleshooting/#use-programmatic-auto-instrumentation

[herin049]

Thank you for your reply @herin049!

Regarding the note on the page of the opentelemetry-sitecustomize package 'If you control the start command of your application, preferably use the opentelemetry-instrument CLI.', I would prefer a solution for uvicorn that enables auto-instrumentation using the traditional CLI method. However, I would be very interested if you could provide more information on why the traditional CLI method is preferred over the opentelemetry-sitecustomize method.

I would greatly appreciate any updates on this.

The reason I suggest sticking with the CLI wrapper is because it can potentially lead to situations where you unintentionally instrument your application. For example, if you install this package in a shared environment, it could lead to other applications being instrumented. However, assuming this is a single isolated environment, you should be fine.

I will take a look and see what we can do to support Uvicorn workers with the CLI wrapper.

[herin049]

@jennykupzig I have opened a PR which successfully addresses your original issue by adding opt-in support for auto-instrumenting subprocesses. Once the linked PR is merged, you will be able to use the following command to auto-instrument your application while using Uvicorn with multiple workers:

opentelemetry-instrument --auto_instrumentation_instrument_subprocesses true uvicorn main:app --workers 4 --host 0.0.0.0 --port 8000

Alternatively, you can set the OTEL_PYTHON_AUTO_INSTRUMENTATION_INSTRUMENT_SUBPROCESSES environment variable to true.