Skip to content

test: add screenshots#86

Merged
ReenigneArcher merged 19 commits intomasterfrom
test/add-screenshots
Mar 23, 2026
Merged

test: add screenshots#86
ReenigneArcher merged 19 commits intomasterfrom
test/add-screenshots

Conversation

@ReenigneArcher
Copy link
Member

@ReenigneArcher ReenigneArcher commented Dec 25, 2025

Description

This PR adds screenshots to tests. The goal is to be able to easily review what the tray icon looks like in different states.

TODO:

  • fix sonar warnings
  • post images in PR comment and/or github step summary
    • secondary workflow to:
      • download artifacts from first job
      • publish images to a screenshot branch
      • comment on PR
        • compare master branch images to PR images, make obvious if images are mismatched (need to remove date/time for reliable comparison)
  • capture more states/events of tray
  • minimize all windows before tests (on windows)
  • improve test coverage
  • misc issues
    • show menu
      • windows
      • linux
      • macos
    • ensure notifications appear
      • Linux
      • macOS
        • fix permissions on macOS runner?
      • Windows
        • disabled do not disturb
        • disabled quiet time (1st hour after initial setup) by adjusting system time
        • works locally, but not in GitHub runner?
    • tests hanging on macOS
    • multiple tray icons appearing on macOS, not properly cleaned up between tests
    • libappindicator version is not building (probably due to dependencies added by new virtual display action)
    • merge (feat(actions): add setup_virtual_desktop action actions#103) and release virtual display action
    • show menu in more states, like checked / unchecked checkboxes and whatnot

Screenshot

Issues Fixed or Closed

Roadmap Issues

Type of Change

  • feat: New feature (non-breaking change which adds functionality)
  • fix: Bug fix (non-breaking change which fixes an issue)
  • docs: Documentation only changes
  • style: Changes that do not affect the meaning of the code (white-space, formatting, missing semicolons, etc.)
  • refactor: Code change that neither fixes a bug nor adds a feature
  • perf: Code change that improves performance
  • test: Adding missing tests or correcting existing tests
  • build: Changes that affect the build system or external dependencies
  • ci: Changes to CI configuration files and scripts
  • chore: Other changes that don't modify src or test files
  • revert: Reverts a previous commit
  • BREAKING CHANGE: Introduces a breaking change (can be combined with any type above)

Checklist

  • Code follows the style guidelines of this project
  • Code has been self-reviewed
  • Code has been commented, particularly in hard-to-understand areas
  • Code docstring/documentation-blocks for new or existing methods/components have been added or updated
  • Unit tests have been added or updated for any new or modified functionality

AI Usage

  • None: No AI tools were used in creating this PR
  • Light: AI provided minor assistance (formatting, simple suggestions)
  • Moderate: AI helped with code generation or debugging specific parts
  • Heavy: AI generated most or all of the code changes

@codecov
Copy link

codecov bot commented Dec 25, 2025

Codecov Report

❌ Patch coverage is 77.41935% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.22%. Comparing base (7ce2adc) to head (7b11e95).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/tray_linux.c 70.49% 11 Missing and 7 partials ⚠️
src/tray_darwin.m 88.88% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master      #86       +/-   ##
===========================================
+ Coverage   62.01%   73.22%   +11.20%     
===========================================
  Files           4        4               
  Lines         337      422       +85     
  Branches       63       81       +18     
===========================================
+ Hits          209      309      +100     
+ Misses         92       70       -22     
- Partials       36       43        +7     
Flag Coverage Δ
Linux 60.00% <70.49%> (+10.94%) ⬆️
Windows 71.85% <100.00%> (+15.15%) ⬆️
libappindicator3-dev 60.00% <70.49%> (+10.94%) ⬆️
libayatana-appindicator3-dev 60.00% <70.49%> (?)
macOS 64.42% <88.88%> (+7.79%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/tray_windows.c 81.25% <100.00%> (+16.92%) ⬆️
src/tray_darwin.m 87.01% <88.88%> (+3.08%) ⬆️
src/tray_linux.c 69.71% <70.49%> (+7.06%) ⬆️

@ReenigneArcher ReenigneArcher force-pushed the test/add-screenshots branch 29 times, most recently from d361a0c to c6d4b9b Compare December 31, 2025 05:00
@ReenigneArcher ReenigneArcher force-pushed the test/add-screenshots branch 5 times, most recently from e9ed38e to cd13799 Compare January 1, 2026 23:18
@ReenigneArcher
Copy link
Member Author

@ThomVanL I think I'm facing a similar problem with permissions in this PR. Wondering if you can provide some assistance. We use this library for Sunshine's tray icon. Any idea how to grant the proper permission to avoid this popup warning/error?

tray_notification_with_callback

Adds logic to create an invisible toplevel window as an anchor for the tray menu, ensuring compatibility with AppIndicator and headless environments. Uses modern GTK API when available and falls back to legacy methods as needed.
Tray items on macOS now display a tooltip if the 'tooltip' field is set. This improves user experience by providing additional context when hovering over the tray icon.
Eliminated unnecessary tray_loop(1) calls in unit tests for tooltip, checkbox, and icon updates. WaitForTrayReady is sufficient for synchronization, simplifying the test logic.
Updated tray_exit to remove the status item from the status bar using dispatch_async on the main thread, ensuring thread safety for NSStatusBar operations.
Replaces or removes calls to tray_loop(1) in unit tests to prevent hanging during test execution. The TestTrayLoop now uses tray_loop(0) for non-blocking behavior, and other tests no longer invoke tray_loop, improving test reliability and speed.
Updated tray_exit to check if the current thread is the main thread before removing the status item. If not on the main thread, the removal is dispatched synchronously to the main thread to ensure thread safety.
Enhanced the WaitForTrayReady method to handle macOS (AppKit) by adding a delay, ensuring the tray icon appears before screenshots. Updated comments for clarity and maintained Linux (AppIndicator) event processing.
Introduces a step to grant screen recording permissions on macOS runners by modifying the TCC database, preventing popup dialogs during CI runs. This ensures smoother automated workflows and addresses issues with screen capture access in GitHub Actions.
Introduces an appindicator_type variable to the CI matrix for distinguishing between 'ayatana' and 'legacy' appindicator packages. Passes this variable to the setup_virtual_desktop action to allow for more flexible environment configuration.
Pin setup_virtual_desktop action to a specific commit and add extensive D-Bus environment checks in the CI test step: print DBUS_SESSION_BUS_ADDRESS, attempt fallback detection from a dbus-daemon process, verify the D-Bus session responds, and warn if AppIndicator service is missing before running tests. In tests, ensure BaseTest::SetUp() is called in the platform-specific SetUp overrides to run common setup logic. Clean up includes and tighten const-correctness in screenshot_utils.cpp (remove unused headers and make local variables const). These changes aim to reduce flaky Linux AppIndicator test failures by ensuring a working D-Bus session and to improve code quality in tests.
Remove verbose D-Bus environment detection, fallback logic, and verification steps from the CI test job. The workflow now runs ./test_tray --gtest_color=yes --gtest_output=xml:test_results.xml directly (working-directory: build/tests). This trims the CI step by deleting the DBUS_SESSION_BUS_ADDRESS checks, pgrep-/proc fallback, dbus-send responsiveness tests, and AppIndicator registration check that were previously logging and potentially causing hangs.
Update the 'Setup virtual desktop' step in the CI workflow to use LizardByte/actions@70bb8d3 (v2026.227.200013) instead of the older setup_virtual_desktop action. This switches to the newer action tag for the Linux virtual desktop setup while preserving the existing inputs and environment.
Remove brittle WaitForTrayReady() and inline EXPECT_TRUE(captureScreenshot(...)) calls across tray unit tests. Replace them with deterministic capture threads that open the tray/menu (tray_show_menu), sleep briefly, capture screenshots, and then programmatically close the menu (platform-specific: PostMessage WM_CANCELMODE on Windows, synthesize Escape on macOS) before calling tray_exit(). Adjust checkbox test to reinitialize the tray to exercise checked/unchecked captures. Overall this makes visual-capture steps explicit and less timing-dependent.
Introduce a dedicated _tray_popup helper on Linux to manage popup menus and their anchor window: track current_popup and menu_anchor_window, dismiss previous popups, create/destroy a tiny toplevel anchor, and use gtk_menu_popup_at_rect when available. Ensure cleanup of popups and anchor window in tray exit. Change Windows tray_show_menu to PostMessage to avoid synchronous SendMessage reentrancy. Minor whitespace/format adjustments in tray.h and tray_darwin.m. Update unit tests to remove fragile fixed sleeps and single-step tray_loop(1) calls: replace with polling while(tray_loop(0) == 0) and simplify screenshot/capture threads and platform-specific dismissal sequences to make tests more reliable.
@ThomVanL
Copy link

@ReenigneArcher I think the particular permission you might be looking could be kTCCServiceScreenCapture. I did a test just now after removing all of my machine's permissions with sudo tccutil reset All (case-sensitive), firing up Sunshine and I got the following results:

# system TCC.DB
sudo sqlite3 "/Library/Application Support/com.apple.TCC/TCC.db" "                     
SELECT * FROM access
WHERE service='kTCCServiceScreenCapture'"
# kTCCServiceScreenCapture|com.apple.Terminal|0|2|4|1|??^L||0|UNUSED||0|1774218150|||UNUSED|1774218150
# kTCCServiceScreenCapture|com.microsoft.VSCode|0|2|4|1|??^L||0|UNUSED||0|1774217534|||UNUSED|1774217534

# user TCC.DB
sudo sqlite3 "$HOME/Library/Application Support/com.apple.TCC/TCC.db" "                
SELECT * FROM access
WHERE service='kTCCServiceScreenCapture'"
# <empty>

Just based off this, I think that particular permission needs to be in the system TCC database, not the one under the user's home. You might already know this but accessing the TCC databases also requires your process to have "Full Disk Access" permissions, but that should already be the case with Github runners (though I'm not entirely sure if you're using those).

If you're looking to add a client with an absolute path, then the record structure changes slightly and you could use the script I referenced for my core audio tap PR.

https://github.com/actions/runner-images/blob/563ad669b646f6f3d68e17b9d83d4028806becb6/images/macos/scripts/build/configure-tccdb-macos.sh#L37

@ThomVanL
Copy link

So to be more complete... I think this might help:

set -euo pipefail

configure_system_tccdb() {
  local values=$1
  local dbPath="/Library/Application Support/com.apple.TCC/TCC.db"
  local sqlQuery="INSERT OR IGNORE INTO access VALUES($values);"
  sudo sqlite3 "$dbPath" "$sqlQuery"
}

configure_user_tccdb() {
  local values=$1
  local dbPath="$HOME/Library/Application Support/com.apple.TCC/TCC.db"
  local sqlQuery="INSERT OR IGNORE INTO access VALUES($values);"
  sqlite3 "$dbPath" "$sqlQuery"
}

systemValuesArray=(
  "'kTCCServiceScreenCapture','/bin/bash',1,2,0,1,NULL,NULL,NULL,'UNUSED',NULL,0,1599831148"
)
for values in "${systemValuesArray[@]}"; do
  configure_system_tccdb "$values,NULL,NULL,'UNUSED',${values##*,}"
done

userValuesArray=(
  "'kTCCServiceScreenCapture','/bin/bash',1,2,0,1,NULL,NULL,NULL,'UNUSED',NULL,0,1583997993"
)
for values in "${userValuesArray[@]}"; do
  configure_user_tccdb "$values,NULL,NULL,'UNUSED',${values##*,}"
done

echo "macOS TCC permissions configured."

@ReenigneArcher
Copy link
Member Author

ReenigneArcher commented Mar 22, 2026

@ThomVanL thank you! I will give your suggestion a try. I am just using GitHub's hosted runners, nothing self hosted.

Edit: still giving me the same popup

ReenigneArcher and others added 4 commits March 22, 2026 19:32
Refactor the macOS screen-recording permission step in the CI workflow: enable strict shell flags, replace the previous dynamic find/loop logic with helper functions (configure_system_tccdb/configure_user_tccdb) and fixed arrays of SQL insert values, and insert entries directly into the system and user TCC databases. Removes verbose logging, Sonoma-specific branching, and dynamic path discovery in favor of a simpler, more deterministic insertion approach and adds a final confirmation echo.

Co-Authored-By: Thomas Van Laere <4990509+thomvanl@users.noreply.github.com>
Pin artifact actions in CI and extend the screenshots publishing workflow. ci.yml: update upload-artifact usage to the v7 SHA. publish-screenshots.yml: add actions read permission, pin download-artifact and checkout actions, change artifact download to include repo/token and save to run-screenshots, and prepare per-matrix directories. Add steps to detect workflow context (PR vs master push), sync prepared screenshots into a screenshots branch (baseline or pull-requests/PR-<n>) using rsync, generate a PR comparison markdown with embedded raw image links, commit and push changes if any, and post the comparison as a PR comment. Includes validation, debug output, and early-exit handling when no screenshots are present.
@sonarqubecloud
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants