The upload ID may be invalid, or the upload may have been aborted or completed.

[Zevrap-81]

Apache Iceberg version

0.11.0 (latest release)

Please describe the bug 🐞

Hello,

I am using a setup with pyiceberg and polaris as rest catalog along with Rustfs for S3 filestorage type. When i am getting this error during table commits.

OSError: When completing multiple part upload for key 'Senec/Stage/Bronze/Ampace_P4/BmsAmpaceModules/data/month_timestamp=2025-09/bucket_mainControllerSerial=7/bucket_moduleSerial=17/00000-0-ab52d0d3-77d2-4d72-b10f-146e1d40e866.parquet' in bucket 'senec-bucket': AWS Error NO_SUCH_UPLOAD during CompleteMultipartUpload operation: The specified multipart upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.

And here is the entire traceback:

Traceback (most recent call last):
  File "/home/parvez/23-0325_senec_monitoring/novum-iceberg/scripts/transfer_monthly.py", line 205, in <module>
    transfer_lfp_data_for_module(
  File "/home/parvez/23-0325_senec_monitoring/novum-iceberg/scripts/transfer_monthly.py", line 147, in transfer_lfp_data_for_module
    iceberg_client.write(                                                                                                            File "/home/parvez/23-0325_senec_monitoring/novum-iceberg/novum_iceberg/iceberg_client.py", line 428, in write                       table.append(pa_table)
  File "/home/parvez/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.12/site-packages/pyiceberg/table/__init__.py", line 1427, in append
    tx.append(df=df, snapshot_properties=snapshot_properties, branch=branch)
  File "/home/parvez/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.12/site-packages/pyiceberg/table/__init__.py", line 523, in append                                                                                                                            data_files = list(
                 ^^^^^
  File "/home/parvez/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.12/site-packages/pyiceberg/io/pyarrow.py", line 2875, in _dataframe_to_data_files
    yield from write_file(
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 619, in result_iterator
    yield _result_or_cancel(fs.pop())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 317, in _result_or_cancel
    return fut.result(timeout)
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/parvez/23-0325_senec_monitoring/novum-iceberg/env/lib/python3.12/site-packages/pyiceberg/io/pyarrow.py", line 2666, in write_parquet
    with fo.create(overwrite=True) as fos:
  File "pyarrow/io.pxi", line 136, in pyarrow.lib.NativeFile.__exit__
  File "pyarrow/io.pxi", line 206, in pyarrow.lib.NativeFile.close
  File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
OSError: When completing multiple part upload for key 'Senec/Stage/Bronze/Ampace_P4/BmsAmpaceModules/data/month_timestamp=2025-09/bucket_mainControllerSerial=7/bucket_moduleSerial=17/00000-0-ab52d0d3-77d2-4d72-b10f-146e1d40e866.parquet' in bucket 'senec-bucket': AWS Error NO_SUCH_UPLOAD during CompleteMultipartUpload operation: The specified multipart upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed.

I have tried to find what is causing the issue but with no luck. I would appreciate it if someone can point to me what could be the problem and a way to fix this issue.

I am working on creating a MCE for this.

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time