ord: add cross-process CPU throttle to prevent overcommit by oharboe · Pull Request #10053 · The-OpenROAD-Project/OpenROAD

oharboe · 2026-04-04T09:00:58Z

When multiple OpenROAD instances run in parallel (e.g. Bazel tests), each creating its own thread pool, total threads can far exceed available cores. This adds an always-on, zero-config throttle using per-CPU file locks in /tmp/openroad_cpu_sem/. Each instance acquires N slots (matching its thread count) before proceeding, blocking if the machine is fully committed. Crash-safe: OS releases locks on process death. Linux-only; no-op on macOS. Use -disable_throttle for debugging (e.g. tracking down non-determinism).

Type of Change

Bug fix
New feature

When multiple OpenROAD instances run in parallel (e.g. Bazel tests), each creating its own thread pool, total threads can far exceed available cores. This adds an always-on, zero-config throttle using per-CPU file locks in /tmp/openroad_cpu_sem/. Each instance acquires N slots (matching its thread count) before proceeding, blocking if the machine is fully committed. Crash-safe: OS releases locks on process death. Linux-only; no-op on macOS. Use -disable_throttle for debugging (e.g. tracking down non-determinism). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>

gemini-code-assist

Code Review

This pull request introduces a CPU throttling mechanism to coordinate thread usage across multiple OpenROAD instances using file-based locks on Linux. It includes a new CpuThreadThrottle class, documentation, and a command-line flag to disable the feature. The review feedback highlights several improvements for robustness and modern C++ standards, including the use of O_CLOEXEC to prevent file descriptor leakage, handling potential infinite loops during resource acquisition, replacing deprecated usleep with std::this_thread::sleep_for, and cleaning up unused member variables.

gemini-code-assist · 2026-04-04T09:03:51Z

src/CpuThrottle.cpp

+{
+  // Open and lock the coordinator to serialize allocation attempts.
+  // This prevents deadlock: only one process tries to grab CPU slots at a time.
+  int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDWR, 0666);


The open call should include the O_CLOEXEC flag. This ensures that the file descriptor is automatically closed if the process executes another program (e.g., calling external tools like ABC), preventing file descriptor leakage.

Suggested change

int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDWR, 0666);

int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDWR | O_CLOEXEC, 0666);

gemini-code-assist · 2026-04-04T09:03:51Z

src/CpuThrottle.cpp

+  if (coord_fd < 0) {
+    logger_->warn(utl::ORD,
+                  44,
+                  "Throttle: cannot open coordinator lock: {}",
+                  std::strerror(errno));
+    return false;
+  }


If open fails due to a permanent error (such as EACCES if the directory permissions are incorrect), tryAcquire will return false and the constructor will loop indefinitely, spamming the log with warnings every 100ms. Consider handling permanent failures by either terminating the program using logger_->error or disabling the throttle mechanism for the current session.

References

Calls to logger_->error are expected to terminate the program, so additional error handling like returning an error code or throwing an exception is not necessary.

gemini-code-assist · 2026-04-04T09:03:51Z

src/CpuThrottle.cpp

+       i < total_cpus && static_cast<int>(acquired.size()) < num_threads;
+       ++i) {
+    std::string path = std::string(kSemDir) + "/cpu." + std::to_string(i);
+    int fd = open(path.c_str(), O_CREAT | O_RDWR, 0666);


Similar to the coordinator lock, these per-CPU file descriptors should be opened with O_CLOEXEC to prevent leakage to child processes.

Suggested change

int fd = open(path.c_str(), O_CREAT | O_RDWR, 0666);

int fd = open(path.c_str(), O_CREAT | O_RDWR | O_CLOEXEC, 0666);

gemini-code-assist · 2026-04-04T09:03:51Z

src/CpuThrottle.cpp

+#ifdef __linux__
+#include <sys/file.h>
+#include <sys/stat.h>
+#include <unistd.h>


Include <fcntl.h> to provide the definition for the open system call and the O_CLOEXEC flag, ensuring better adherence to POSIX standards.

Suggested change

#include <unistd.h>

#include <fcntl.h>

#include <unistd.h>

gemini-code-assist · 2026-04-04T09:03:51Z

src/CpuThrottle.cpp

+#endif
+
+#include <string>
+#include <thread>


Include to support the use of std::chrono types for thread sleeping, which is the modern C++ approach.

Suggested change

#include <thread>

#include <chrono>

#include <thread>

gemini-code-assist · 2026-04-04T09:03:51Z

src/CpuThrottle.cpp

+          num_threads);
+      logged_waiting = true;
+    }
+    usleep(100000);  // 100ms


usleep is deprecated in POSIX.1-2008. Use std::this_thread::sleep_for instead. Additionally, using the kRetryDelayMs constant avoids the use of magic numbers, following the repository's general rules.

Suggested change

usleep(100000); // 100ms

std::this_thread::sleep_for(std::chrono::milliseconds(kRetryDelayMs));

References

Define tunable parameters as named constants instead of using hardcoded magic numbers.

gemini-code-assist · 2026-04-04T09:03:51Z

src/CpuThrottle.h

+  int coordinator_fd_ = -1;
+  utl::Logger* logger_;


The coordinator_fd_ member variable is unused and should be removed. Additionally, defining kRetryDelayMs here avoids magic numbers in the implementation.

Suggested change

int coordinator_fd_ = -1;

utl::Logger* logger_;

static constexpr int kRetryDelayMs = 100;

utl::Logger* logger_;

References

Define tunable parameters as named constants instead of using hardcoded magic numbers.

github-actions

clang-tidy made some suggestions

github-actions · 2026-04-04T09:07:27Z

src/CpuThrottle.cpp

+
+  if (num_threads > total_cpus) {
+    num_threads = total_cpus;
+  }


warning: use std::min instead of > [readability-use-std-min-max]

src/CpuThrottle.cpp:10:

- #include <cerrno> + #include <algorithm> + #include <cerrno>

Suggested change

}

num_threads = std::min(num_threads, total_cpus);

github-actions · 2026-04-04T09:07:27Z

src/CpuThrottle.cpp

+}
+
+CpuThreadThrottle::CpuThreadThrottle(CpuThreadThrottle&& other) noexcept
+    : held_fds_(std::move(other.held_fds_)),


warning: no header providing "std::move" is directly included [misc-include-cleaner]

src/CpuThrottle.cpp:4:

+ #include <utility>

github-actions · 2026-04-04T09:07:27Z

src/CpuThrottle.cpp

+{
+  // Open and lock the coordinator to serialize allocation attempts.
+  // This prevents deadlock: only one process tries to grab CPU slots at a time.
+  int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDWR, 0666);


warning: no header providing "O_CREAT" is directly included [misc-include-cleaner]

. ^

github-actions · 2026-04-04T09:07:27Z

src/CpuThrottle.cpp

+{
+  // Open and lock the coordinator to serialize allocation attempts.
+  // This prevents deadlock: only one process tries to grab CPU slots at a time.
+  int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDWR, 0666);


warning: no header providing "O_RDWR" is directly included [misc-include-cleaner]

. ^

github-actions · 2026-04-04T09:07:27Z

src/CpuThrottle.cpp

+{
+  // Open and lock the coordinator to serialize allocation attempts.
+  // This prevents deadlock: only one process tries to grab CPU slots at a time.
+  int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDWR, 0666);


warning: no header providing "open" is directly included [misc-include-cleaner]

src/CpuThrottle.cpp:4:

+ #include <fcntl.h>

github-actions · 2026-04-04T09:07:27Z

src/CpuThrottle.cpp

+    return false;
+  }
+
+  std::vector<int> acquired;


warning: no header providing "std::vector" is directly included [misc-include-cleaner]

src/CpuThrottle.cpp:4:

+ #include <vector>

github-actions · 2026-04-04T09:07:27Z

src/OpenRoad.cc

  sta_->setThreadCount(threads_);
+
+  if (!disable_throttle_ && threads_ > 0) {
+    cpu_throttle_ = std::make_unique<CpuThreadThrottle>(threads_, logger_);


warning: no header providing "std::make_unique" is directly included [misc-include-cleaner]

src/OpenRoad.cc:10:

- #include <stdexcept> + #include <memory> + #include <stdexcept>

oharboe · 2026-04-04T10:45:59Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a cross-process CPU throttle mechanism using file-based locks to prevent core oversubscription when running multiple OpenROAD instances. It includes a new CpuThreadThrottle class, RAII guards for thread management, and a Python-based monitoring tool. Feedback focuses on improving robustness in multi-user environments by handling umask and file permissions (using O_RDONLY for locks), preventing log spam during contention, and ensuring the throttle is initialized by default. Additionally, several calls to acquireGlobalCpuThreads should be updated to explicitly pass the local thread count to ensure accurate slot reservation.

gemini-code-assist · 2026-04-04T10:49:21Z

src/CpuThrottle.cpp

+  mkdir(kSemDir, 0777);
+  // Ignore EEXIST — directory may already exist from another process.


The mkdir call is subject to the process's umask. In a multi-user environment, if the first user to run OpenROAD has a restrictive umask (e.g., 0022), the directory will be created with 0755 permissions, preventing other users from creating lock files inside it. To ensure the directory is truly shared, consider calling chmod(kSemDir, 0777) if mkdir succeeds.

gemini-code-assist · 2026-04-04T10:49:21Z

src/CpuThrottle.cpp

+bool CpuThreadThrottle::tryAcquireAdditional(int additional)
+{
+  // Open and lock the coordinator to serialize allocation attempts.
+  int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDWR | O_CLOEXEC, 0666);


Using O_RDWR for the coordinator lock requires write permission on the file. If the file was created by another user with a restrictive umask (e.g., 0644), subsequent users will fail to open it in O_RDWR mode. Since flock only requires an open file descriptor and does not check the open mode for exclusive locks, using O_RDONLY is more robust for a machine-wide shared resource.

Suggested change

int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDWR | O_CLOEXEC, 0666);

int coord_fd = open(kCoordinatorLock, O_CREAT | O_RDONLY | O_CLOEXEC, 0666);

gemini-code-assist · 2026-04-04T10:49:21Z

src/CpuThrottle.cpp

+    logger_->warn(utl::ORD,
+                  80,
+                  "Throttle: cannot open coordinator lock: {}",
+                  std::strerror(errno));
+    return false;


This warning is inside a loop that retries every 100ms for up to 30 seconds. If the error is persistent (e.g., EACCES due to permission issues), it will spam the log with hundreds of identical warnings. Consider logging the error only once or handling persistent failures by disabling the throttle for the current session.

References

When analyzing code within a loop, consider the entire loop structure to avoid redundant operations or repetitive logging within the loop body.

gemini-code-assist · 2026-04-04T10:49:21Z

src/CpuThrottle.cpp

+       i < total_cpus_ && static_cast<int>(acquired.size()) < additional;
+       ++i) {
+    std::string path = std::string(kSemDir) + "/cpu." + std::to_string(i);
+    int fd = open(path.c_str(), O_CREAT | O_RDWR | O_CLOEXEC, 0666);


Similar to the coordinator lock, using O_RDONLY for the per-CPU lock files is preferred to avoid permission issues when multiple users share the same machine-wide pool.

Suggested change

int fd = open(path.c_str(), O_CREAT | O_RDWR | O_CLOEXEC, 0666);

int fd = open(path.c_str(), O_CREAT | O_RDONLY | O_CLOEXEC, 0666);

gemini-code-assist · 2026-04-04T10:49:22Z

src/OpenRoad.cc

+  if (!disable_throttle_ && !cpu_throttle_ && threads_ > 0) {
+    cpu_throttle_ = std::make_unique<CpuThreadThrottle>(logger_);
+    setGlobalCpuThrottle(cpu_throttle_.get());
+  }


The throttle is currently only initialized if setThreadCount is called (e.g., via the -threads command line flag). If OpenROAD is run with default settings, the throttle remains uninitialized, and the process will not participate in machine-wide CPU coordination. To ensure the throttle is "always-on" as described, it should be initialized during OpenRoad startup even if no explicit thread count is provided.

gemini-code-assist · 2026-04-04T10:49:22Z

src/drt/src/TritonRoute.cpp

 void TritonRoute::updateDesign(const std::vector<std::string>& updatesStrs,
                               int num_threads)
 {
+  auto cpu_guard = ord::acquireGlobalCpuThreads();


The acquireGlobalCpuThreads() call should explicitly use the num_threads parameter to ensure the throttle reserves the correct number of slots for the subsequent OpenMP parallel section.

Suggested change

auto cpu_guard = ord::acquireGlobalCpuThreads();

auto cpu_guard = ord::acquireGlobalCpuThreads(num_threads);

gemini-code-assist · 2026-04-04T10:49:22Z

src/drt/src/TritonRoute.cpp


 void TritonRoute::updateDesign(const std::string& path, int num_threads)
 {
+  auto cpu_guard = ord::acquireGlobalCpuThreads();


The acquireGlobalCpuThreads() call should explicitly use the num_threads parameter.

Suggested change

auto cpu_guard = ord::acquireGlobalCpuThreads();

auto cpu_guard = ord::acquireGlobalCpuThreads(num_threads);

gemini-code-assist · 2026-04-04T10:49:22Z

src/drt/src/TritonRoute.cpp

 void TritonRoute::sendDesignUpdates(const std::string& router_cfg_path,
                                    int num_threads)
 {
+  auto cpu_guard = ord::acquireGlobalCpuThreads();


The acquireGlobalCpuThreads() call should explicitly use the num_threads parameter.

Suggested change

auto cpu_guard = ord::acquireGlobalCpuThreads();

auto cpu_guard = ord::acquireGlobalCpuThreads(num_threads);

gemini-code-assist · 2026-04-04T10:49:22Z

src/grt/src/RepairAntennas.cpp


  arc_->makeNetWiresFromGuides(nets_to_repair);
  arc_->initAntennaRules();
+  auto cpu_guard = ord::acquireGlobalCpuThreads();


The acquireGlobalCpuThreads() call should explicitly use the num_threads parameter to match the subsequent omp_set_num_threads call.

Suggested change

auto cpu_guard = ord::acquireGlobalCpuThreads();

auto cpu_guard = ord::acquireGlobalCpuThreads(num_threads);

gemini-code-assist · 2026-04-04T10:49:22Z

src/par/src/Multilevel.cpp

    }

    // Parallel refine all the solutions
+    auto cpu_guard = ord::acquireGlobalCpuThreads();


The throttle should reserve a number of slots equal to the number of threads being launched, which is top_solutions.size() in this context.

Suggested change

auto cpu_guard = ord::acquireGlobalCpuThreads();

auto cpu_guard = ord::acquireGlobalCpuThreads(top_solutions.size());

github-actions

clang-tidy made some suggestions

github-actions · 2026-04-04T10:59:16Z

src/OpenRoad.cc

+  // Create the throttle on first call (holds 1 slot for main thread).
+  // It will be resized to threads_ dynamically when tools need parallelism.
+  if (!disable_throttle_ && !cpu_throttle_ && threads_ > 0) {
+    cpu_throttle_ = std::make_unique<CpuThreadThrottle>(logger_);


warning: no header providing "std::make_unique" is directly included [misc-include-cleaner]

src/OpenRoad.cc:10:

- #include <stdexcept> + #include <memory> + #include <stdexcept>

github-actions · 2026-04-04T10:59:16Z

src/ant/src/AntennaChecker.cc

@@ -29,6 +29,7 @@
 #include "odb/dbWireGraph.h"
 #include "odb/geom.h"
 #include "omp.h"
