Skip to content

NF: Add Jupyter notebook widget for WebGL viewer#608

Open
mvdoc wants to merge 19 commits intogallantlab:mainfrom
mvdoc:claude/webgl-jupyter-widget-dvTkg
Open

NF: Add Jupyter notebook widget for WebGL viewer#608
mvdoc wants to merge 19 commits intogallantlab:mainfrom
mvdoc:claude/webgl-jupyter-widget-dvTkg

Conversation

@mvdoc
Copy link
Copy Markdown
Contributor

@mvdoc mvdoc commented Mar 23, 2026

Summary

Add cortex.webgl.jupyter module for embedding the WebGL brain viewer in Jupyter notebooks:

  • display(data) — IFrame mode: starts Tornado server in background thread, embeds via IFrame. Full interactivity with WebSocket/JSMixer control.
  • display(data, method="static") — Static mode: generates self-contained viewer directory served via local HTTP server. Returns StaticViewer handle with close() for cleanup.
  • make_notebook_html(data) — returns raw HTML string for custom embedding.
  • Reliable port selection via socket.bind(('', 0)) instead of random.randint
  • StaticViewer class with:
    • Proper resource cleanup (HTTP server shutdown + temp dir removal)
    • Public closed property
    • Context manager protocol (with viewer:)
    • _repr_html_() that shows "closed" message after cleanup
    • Automatic removal from active viewer registry on close
  • IFrame URL uses 127.0.0.1 by default (configurable via CORTEX_JUPYTER_IFRAME_HOST and CORTEX_JUPYTER_STATIC_HOST env vars for remote setups)
  • Resource leak protection: HTTP server and temp files cleaned up even if ipydisplay() or HTTPServer() fails
  • close_all() releases lock before blocking shutdown to prevent deadlocks; hardened for interpreter teardown
  • Thread join timeout warnings logged when server thread doesn't stop
  • nbsphinx guarded with try/except in docs config so local builds work without it
  • 44 unit tests covering dispatch, port handling, lifecycle, cleanup, error paths, resource leaks, and lock safety

New/modified files

  • cortex/webgl/jupyter.py — main module
  • cortex/webgl/__init__.py — guarded import of jupyter submodule
  • cortex/tests/test_jupyter_widget.py — 44 tests
  • docs/notebooks/jupyter_notebook.ipynb — example notebook (auto-executed at docs build time)
  • docs/conf.py — optional nbsphinx integration

Test plan

  • python -m pytest cortex/tests/test_jupyter_widget.py -v — 44 tests pass
  • Verify notebook executes via jupyter nbconvert --execute
  • Test display(vol) in live Jupyter notebook
  • Test display(vol, method="static") in live Jupyter notebook
  • Test viewer.close() cleans up server and temp files
  • Test context manager (with viewer:) auto-cleanup
  • Test close_all() cleans up multiple viewers
  • Verify docs/conf.py works with and without nbsphinx installed

claude and others added 12 commits March 23, 2026 09:51
Implements two display methods for embedding pycortex WebGL brain
viewers in Jupyter notebooks:

- display_iframe: Embeds the Tornado-served viewer in an IFrame for
  full interactivity (surface morphing, data switching, WebSocket)
- display_static: Generates self-contained HTML served via a lightweight
  HTTP server, suitable for sharing

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Covers IFrame mode, static mode, programmatic control via JSMixer,
viewer customization options, and raw HTML generation.

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Sphinx-gallery only processes files matching the /plot_ pattern.
Renamed jupyter_notebook.py -> plot_jupyter_notebook.py and added
sphinx_gallery_dummy_images directive since this example cannot
execute during doc builds (requires Jupyter + browser).

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
- Add nbsphinx extension to Sphinx conf with execute='never'
- Exclude auto_examples/*.ipynb to avoid conflict with sphinx-gallery
- Replace plot_jupyter_notebook.py with jupyter_notebook.ipynb
- Symlink notebook from examples/webgl/ into docs/notebooks/
- Add "Jupyter Notebooks" section to docs index toctree

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Restructured the notebook so the static viewer uses
make_notebook_html() + IPython.display.HTML with an srcdoc IFrame,
which persists the full 3D viewer in the cell output. The IFrame
mode and other server-dependent examples are shown as markdown
code blocks since they require a live Jupyter session.

Executed via nbconvert --execute --inplace so nbsphinx renders
the static viewer output directly in the built docs.

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
The executed outputs embedded ~4MB of HTML in the .ipynb (74k lines).
Strip outputs so the notebook stays lightweight; nbsphinx is configured
with nbsphinx_execute = 'never' so the docs render the code cells
without re-executing.

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Set nbsphinx_execute = 'always' so the notebook cells run during
sphinx-build. The committed .ipynb stays clean (no outputs).

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
- Fix invalid port 65536 by using socket.bind(('', 0)) to find free ports
- Remove dead html_content read in display_static
- Tighten import guard: only suppress ImportError when IPython is missing
- Add StaticViewer class with close() for server/tmpdir cleanup
- Wrap make_static in try/except with clear RuntimeError on failure
- Log HTTP 4xx/5xx errors instead of suppressing all server output
- Use TemporaryDirectory context manager in make_notebook_html
- Use port 0 for HTTPServer to get OS-assigned free port
- Expand tests from 10 to 20: dispatch, port handling, cleanup, errors
The previous approach embedded ~4MB of HTML in an iframe srcdoc
attribute, which broke JavaScript due to escaping issues. Now the
notebook calls make_static() to generate the full viewer directory
(HTML + CTM + data files) and references it via a regular IFrame src.

Added a build-finished hook in conf.py to copy the generated
static_viewer/ directory into the Sphinx build output.

Verified with Playwright: brain renders correctly in ~2s.

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
@mvdoc mvdoc requested a review from kroq-gar78 March 23, 2026 17:36
@mvdoc
Copy link
Copy Markdown
Contributor Author

mvdoc commented Mar 23, 2026

The goal is to have something like this in a jupyter notebook. The widgets should allow that (need to test it)

image

(This is another one of those "I wonder if Claude can do it" things coded on my phone)

mvdoc added 2 commits March 23, 2026 10:41
The Jupyter notebook example requires nbsphinx (Sphinx extension),
ipykernel (notebook execution), and pandoc (notebook conversion)
to build during CI.

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
The upstream CI workflow doesn't install nbsphinx. Make it optional:
only add the extension if importable, otherwise exclude the notebooks
directory so the build doesn't fail on .ipynb files.

Also keep the workflow update (nbsphinx + pandoc + ipykernel) for
when the workflow is merged upstream.

https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds first-class Jupyter notebook support for embedding the cortex.webgl viewer, along with documentation and CI updates to build/render an example notebook in the Sphinx docs.

Changes:

  • Add cortex.webgl.jupyter with display() dispatch plus iframe/static helpers and a StaticViewer lifecycle handle.
  • Add a notebook-optimized WebGL HTML template and an example executed notebook in the docs.
  • Update docs build configuration and workflows to support rendering notebooks (nbsphinx + dependencies).

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
cortex/webgl/jupyter.py New Jupyter integration module (iframe/static display + HTML generation helpers).
cortex/webgl/notebook.html New HTML template variant tailored for notebook iframe usage.
cortex/webgl/__init__.py Exposes cortex.webgl.jupyter when IPython is available.
cortex/tests/test_jupyter_widget.py Adds unit tests for dispatch, port selection, and static viewer lifecycle.
docs/notebooks/jupyter_notebook.ipynb New example notebook demonstrating usage.
docs/notebooks/.gitignore Ignores artifacts generated during notebook execution.
docs/index.rst Adds the notebook page to the docs toctree.
docs/conf.py Conditionally enables nbsphinx and copies notebook artifacts into build output.
.github/workflows/build_docs.yml Installs pandoc/nbsphinx/ipykernel and builds docs in CI.
docs/.github/workflows/build_docs.yml Adds a second (duplicate) docs workflow under docs/ (note: not a standard Actions location).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

import tempfile
import threading

from IPython.display import HTML, IFrame
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HTML is imported from IPython.display but never used in this module. This adds an unnecessary hard dependency at import-time and may trigger unused-import checks in downstream tooling; please remove it or use it.

Suggested change
from IPython.display import HTML, IFrame
from IPython.display import IFrame

Copilot uses AI. Check for mistakes.
Comment on lines +55 to +77

def close(self):
"""Shut down the HTTP server and remove temp files."""
try:
self._httpd.shutdown()
except Exception:
logger.warning("Failed to shut down static viewer server", exc_info=True)
try:
shutil.rmtree(self._tmpdir, ignore_errors=True)
except Exception:
logger.warning(
"Failed to clean up temp dir %s", self._tmpdir, exc_info=True
)

def __del__(self):
try:
self._httpd.shutdown()
except Exception:
pass
try:
shutil.rmtree(self._tmpdir, ignore_errors=True)
except Exception:
pass
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

StaticViewer.close() only calls HTTPServer.shutdown() but never calls server_close() and never joins the serving thread. This can leave the listening socket open longer than intended and makes cleanup less reliable; consider making close() idempotent, calling shutdown() + server_close(), and joining the thread with a short timeout.

Suggested change
def close(self):
"""Shut down the HTTP server and remove temp files."""
try:
self._httpd.shutdown()
except Exception:
logger.warning("Failed to shut down static viewer server", exc_info=True)
try:
shutil.rmtree(self._tmpdir, ignore_errors=True)
except Exception:
logger.warning(
"Failed to clean up temp dir %s", self._tmpdir, exc_info=True
)
def __del__(self):
try:
self._httpd.shutdown()
except Exception:
pass
try:
shutil.rmtree(self._tmpdir, ignore_errors=True)
except Exception:
pass
# Track shutdown state to make close() idempotent and thread-safe.
self._closed = False
self._lock = threading.Lock()
def close(self, timeout=1.0):
"""Shut down the HTTP server, wait for the thread, and remove temp files.
Parameters
----------
timeout : float, optional
Maximum number of seconds to wait for the server thread to finish.
"""
# Ensure idempotent, thread-safe shutdown.
try:
lock = self._lock
except AttributeError:
# If attributes are partially initialized or being torn down,
# fall back to best-effort cleanup without synchronization.
lock = None
if lock is not None:
with lock:
if getattr(self, "_closed", False):
return
self._closed = True
else:
if getattr(self, "_closed", False):
return
self._closed = True
# Stop the HTTP server and close its listening socket.
httpd = getattr(self, "_httpd", None)
if httpd is not None:
try:
httpd.shutdown()
httpd.server_close()
except Exception:
logger.warning(
"Failed to shut down static viewer server",
exc_info=True,
)
# Give the serving thread a chance to exit cleanly.
thread = getattr(self, "_thread", None)
if thread is not None and thread.is_alive():
try:
thread.join(timeout=timeout)
except Exception:
logger.warning(
"Failed to join static viewer server thread",
exc_info=True,
)
# Finally, remove the temporary directory.
tmpdir = getattr(self, "_tmpdir", None)
if tmpdir is not None:
try:
shutil.rmtree(tmpdir, ignore_errors=True)
except Exception:
logger.warning(
"Failed to clean up temp dir %s", tmpdir, exc_info=True
)
def __del__(self):
# Best-effort cleanup during garbage collection; avoid raising.
try:
self.close(timeout=0.1)
except Exception:
pass

Copilot uses AI. Check for mistakes.
Comment on lines +11 to +16
Usage
-----
>>> import cortex
>>> vol = cortex.Volume.random("S1", "fullhead")
>>> cortex.webgl.jupyter.display(vol) # auto-detects best approach
"""
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The module docstring example says cortex.webgl.jupyter.display(vol) “auto-detects best approach”, but display() defaults to method="iframe" and does not perform any auto-detection. Please update the docstring/example to match the actual behavior (or implement auto-detection if that’s intended).

Copilot uses AI. Check for mistakes.
mvdoc added 2 commits March 24, 2026 08:17
- Remove unused HTML import
- Make StaticViewer.close() idempotent with server_close() and thread.join()
- Add viewer registry with close_all() and atexit cleanup to prevent tmp buildup
- Fix misleading docstrings (auto-detect claim, self-contained HTML claims)
- Make static host configurable via CORTEX_JUPYTER_STATIC_HOST env var
- Add TOCTOU race documentation to _find_free_port
- Make nbsphinx a required docs dependency (fixes toctree mismatch)
- Fix _copy_notebook_artifacts to skip on failure and overwrite stale builds
- Remove misplaced docs/.github/workflows/build_docs.yml
- Fix test_width_int_converted to actually assert IFrame width
- Add tests for viewer registry and close_all
Address all issues from code review:

Production fixes (cortex/webgl/jupyter.py):
- Use 127.0.0.1 for IFrame URL instead of socket.gethostname() (#1)
- Add CORTEX_JUPYTER_IFRAME_HOST env var for remote setups
- Pass display_url=False to suppress redundant URL output (gallantlab#2)
- Release _viewer_lock before blocking close() in close_all() (gallantlab#4)
- Clean up HTTP server and tmpdir if ipydisplay() fails (gallantlab#7)
- Clean up tmpdir if HTTPServer() constructor fails (gallantlab#8)
- Guard _repr_html_() against use-after-close (gallantlab#9)
- Add public `closed` property (gallantlab#10)
- Add context manager protocol (__enter__/__exit__) (gallantlab#11)
- Warn when server thread doesn't terminate after join (gallantlab#14)
- Remove viewer from _active_viewers on close
- Harden close_all() for interpreter shutdown

Config fixes:
- Guard nbsphinx import in docs/conf.py (gallantlab#5)
- Change nbsphinx_execute to "auto" (gallantlab#6)
- Protect jupyter import in __init__.py (gallantlab#16)
- Remove dead notebook.html template (gallantlab#3)

Tests (21 new, 1 fixed):
- IFrame URL correctness and env var configurability
- display_url=False verification
- closed property, _repr_html_ after close
- Context manager lifecycle
- Double-close idempotency
- Graceful degradation when httpd.shutdown() fails
- Thread join timeout warning
- close_all() lock scope (deadlock prevention)
- Resource leak on ipydisplay/HTTPServer failure
- FileNotFoundError when index.html missing
- CORTEX_JUPYTER_STATIC_HOST env var
- HTTP server integration test (serves content, logs 404s)
- make_notebook_html kwargs forwarding
- Registry removal on close
- Replace flaky port uniqueness test (gallantlab#24)

Docs notebook:
- Add disk-write warning for static viewer
- Rewrite to be executable during docs build
@mvdoc mvdoc changed the title WIP: Add Jupyter notebook widget for WebGL viewer NF: Add Jupyter notebook widget for WebGL viewer Apr 4, 2026
@mvdoc mvdoc marked this pull request as ready for review April 4, 2026 21:17
mvdoc added 3 commits April 4, 2026 14:19
…widget-dvTkg

* origin/main:
  FIX NaN values in Volume/Vertex rendering as black instead of transparent (gallantlab#612)
  Fix docs version display and update copyright to Gallant Lab (gallantlab#614)
  Disable flatmap tilt by default; respect allow_tilt checkbox in controls (gallantlab#618)
  Split 'r' hotkey: fold-only vs full view reset (gallantlab#617)
  Bump codecov/codecov-action from 5 to 6 (gallantlab#615)
  FIX deprecated scipy APIs (gallantlab#609)

# Conflicts:
#	docs/conf.py
- Add output_dir parameter to display_static() so the viewer can write
  to a persistent directory with a relative IFrame path (no ephemeral
  HTTP server). This makes it work in both live Jupyter and rendered docs.
- Add docs dependency group to pyproject.toml (sphinx, numpydoc,
  sphinx-gallery, alabaster, nbsphinx, pandoc, ipykernel).
- Rewrite notebook example to use cortex.webgl.jupyter.display() API.
- Add Jupyter Notebook Examples section with nbsphinx-gallery cards.
- Add 5 tests for the new output_dir parameter.
New notebook showing how to use cortex.export.plot_panels with
headless=True to render composite brain views without opening a
browser. Demonstrates the flatmap + lateral/medial/ventral preset
and all predefined panel layouts.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants