NF: Add Jupyter notebook widget for WebGL viewer#608
NF: Add Jupyter notebook widget for WebGL viewer#608mvdoc wants to merge 19 commits intogallantlab:mainfrom
Conversation
Implements two display methods for embedding pycortex WebGL brain viewers in Jupyter notebooks: - display_iframe: Embeds the Tornado-served viewer in an IFrame for full interactivity (surface morphing, data switching, WebSocket) - display_static: Generates self-contained HTML served via a lightweight HTTP server, suitable for sharing https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Covers IFrame mode, static mode, programmatic control via JSMixer, viewer customization options, and raw HTML generation. https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Sphinx-gallery only processes files matching the /plot_ pattern. Renamed jupyter_notebook.py -> plot_jupyter_notebook.py and added sphinx_gallery_dummy_images directive since this example cannot execute during doc builds (requires Jupyter + browser). https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
- Add nbsphinx extension to Sphinx conf with execute='never' - Exclude auto_examples/*.ipynb to avoid conflict with sphinx-gallery - Replace plot_jupyter_notebook.py with jupyter_notebook.ipynb - Symlink notebook from examples/webgl/ into docs/notebooks/ - Add "Jupyter Notebooks" section to docs index toctree https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Restructured the notebook so the static viewer uses make_notebook_html() + IPython.display.HTML with an srcdoc IFrame, which persists the full 3D viewer in the cell output. The IFrame mode and other server-dependent examples are shown as markdown code blocks since they require a live Jupyter session. Executed via nbconvert --execute --inplace so nbsphinx renders the static viewer output directly in the built docs. https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
The executed outputs embedded ~4MB of HTML in the .ipynb (74k lines). Strip outputs so the notebook stays lightweight; nbsphinx is configured with nbsphinx_execute = 'never' so the docs render the code cells without re-executing. https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
Set nbsphinx_execute = 'always' so the notebook cells run during sphinx-build. The committed .ipynb stays clean (no outputs). https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
This file is generated during docs build and should not be tracked. https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
- Fix invalid port 65536 by using socket.bind(('', 0)) to find free ports
- Remove dead html_content read in display_static
- Tighten import guard: only suppress ImportError when IPython is missing
- Add StaticViewer class with close() for server/tmpdir cleanup
- Wrap make_static in try/except with clear RuntimeError on failure
- Log HTTP 4xx/5xx errors instead of suppressing all server output
- Use TemporaryDirectory context manager in make_notebook_html
- Use port 0 for HTTPServer to get OS-assigned free port
- Expand tests from 10 to 20: dispatch, port handling, cleanup, errors
The previous approach embedded ~4MB of HTML in an iframe srcdoc attribute, which broke JavaScript due to escaping issues. Now the notebook calls make_static() to generate the full viewer directory (HTML + CTM + data files) and references it via a regular IFrame src. Added a build-finished hook in conf.py to copy the generated static_viewer/ directory into the Sphinx build output. Verified with Playwright: brain renders correctly in ~2s. https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
The Jupyter notebook example requires nbsphinx (Sphinx extension), ipykernel (notebook execution), and pandoc (notebook conversion) to build during CI. https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
The upstream CI workflow doesn't install nbsphinx. Make it optional: only add the extension if importable, otherwise exclude the notebooks directory so the build doesn't fail on .ipynb files. Also keep the workflow update (nbsphinx + pandoc + ipykernel) for when the workflow is merged upstream. https://claude.ai/code/session_018vcqQBXvodW9oJozRwEpmd
There was a problem hiding this comment.
Pull request overview
This PR adds first-class Jupyter notebook support for embedding the cortex.webgl viewer, along with documentation and CI updates to build/render an example notebook in the Sphinx docs.
Changes:
- Add
cortex.webgl.jupyterwithdisplay()dispatch plus iframe/static helpers and aStaticViewerlifecycle handle. - Add a notebook-optimized WebGL HTML template and an example executed notebook in the docs.
- Update docs build configuration and workflows to support rendering notebooks (nbsphinx + dependencies).
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
cortex/webgl/jupyter.py |
New Jupyter integration module (iframe/static display + HTML generation helpers). |
cortex/webgl/notebook.html |
New HTML template variant tailored for notebook iframe usage. |
cortex/webgl/__init__.py |
Exposes cortex.webgl.jupyter when IPython is available. |
cortex/tests/test_jupyter_widget.py |
Adds unit tests for dispatch, port selection, and static viewer lifecycle. |
docs/notebooks/jupyter_notebook.ipynb |
New example notebook demonstrating usage. |
docs/notebooks/.gitignore |
Ignores artifacts generated during notebook execution. |
docs/index.rst |
Adds the notebook page to the docs toctree. |
docs/conf.py |
Conditionally enables nbsphinx and copies notebook artifacts into build output. |
.github/workflows/build_docs.yml |
Installs pandoc/nbsphinx/ipykernel and builds docs in CI. |
docs/.github/workflows/build_docs.yml |
Adds a second (duplicate) docs workflow under docs/ (note: not a standard Actions location). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
cortex/webgl/jupyter.py
Outdated
| import tempfile | ||
| import threading | ||
|
|
||
| from IPython.display import HTML, IFrame |
There was a problem hiding this comment.
HTML is imported from IPython.display but never used in this module. This adds an unnecessary hard dependency at import-time and may trigger unused-import checks in downstream tooling; please remove it or use it.
| from IPython.display import HTML, IFrame | |
| from IPython.display import IFrame |
cortex/webgl/jupyter.py
Outdated
|
|
||
| def close(self): | ||
| """Shut down the HTTP server and remove temp files.""" | ||
| try: | ||
| self._httpd.shutdown() | ||
| except Exception: | ||
| logger.warning("Failed to shut down static viewer server", exc_info=True) | ||
| try: | ||
| shutil.rmtree(self._tmpdir, ignore_errors=True) | ||
| except Exception: | ||
| logger.warning( | ||
| "Failed to clean up temp dir %s", self._tmpdir, exc_info=True | ||
| ) | ||
|
|
||
| def __del__(self): | ||
| try: | ||
| self._httpd.shutdown() | ||
| except Exception: | ||
| pass | ||
| try: | ||
| shutil.rmtree(self._tmpdir, ignore_errors=True) | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
StaticViewer.close() only calls HTTPServer.shutdown() but never calls server_close() and never joins the serving thread. This can leave the listening socket open longer than intended and makes cleanup less reliable; consider making close() idempotent, calling shutdown() + server_close(), and joining the thread with a short timeout.
| def close(self): | |
| """Shut down the HTTP server and remove temp files.""" | |
| try: | |
| self._httpd.shutdown() | |
| except Exception: | |
| logger.warning("Failed to shut down static viewer server", exc_info=True) | |
| try: | |
| shutil.rmtree(self._tmpdir, ignore_errors=True) | |
| except Exception: | |
| logger.warning( | |
| "Failed to clean up temp dir %s", self._tmpdir, exc_info=True | |
| ) | |
| def __del__(self): | |
| try: | |
| self._httpd.shutdown() | |
| except Exception: | |
| pass | |
| try: | |
| shutil.rmtree(self._tmpdir, ignore_errors=True) | |
| except Exception: | |
| pass | |
| # Track shutdown state to make close() idempotent and thread-safe. | |
| self._closed = False | |
| self._lock = threading.Lock() | |
| def close(self, timeout=1.0): | |
| """Shut down the HTTP server, wait for the thread, and remove temp files. | |
| Parameters | |
| ---------- | |
| timeout : float, optional | |
| Maximum number of seconds to wait for the server thread to finish. | |
| """ | |
| # Ensure idempotent, thread-safe shutdown. | |
| try: | |
| lock = self._lock | |
| except AttributeError: | |
| # If attributes are partially initialized or being torn down, | |
| # fall back to best-effort cleanup without synchronization. | |
| lock = None | |
| if lock is not None: | |
| with lock: | |
| if getattr(self, "_closed", False): | |
| return | |
| self._closed = True | |
| else: | |
| if getattr(self, "_closed", False): | |
| return | |
| self._closed = True | |
| # Stop the HTTP server and close its listening socket. | |
| httpd = getattr(self, "_httpd", None) | |
| if httpd is not None: | |
| try: | |
| httpd.shutdown() | |
| httpd.server_close() | |
| except Exception: | |
| logger.warning( | |
| "Failed to shut down static viewer server", | |
| exc_info=True, | |
| ) | |
| # Give the serving thread a chance to exit cleanly. | |
| thread = getattr(self, "_thread", None) | |
| if thread is not None and thread.is_alive(): | |
| try: | |
| thread.join(timeout=timeout) | |
| except Exception: | |
| logger.warning( | |
| "Failed to join static viewer server thread", | |
| exc_info=True, | |
| ) | |
| # Finally, remove the temporary directory. | |
| tmpdir = getattr(self, "_tmpdir", None) | |
| if tmpdir is not None: | |
| try: | |
| shutil.rmtree(tmpdir, ignore_errors=True) | |
| except Exception: | |
| logger.warning( | |
| "Failed to clean up temp dir %s", tmpdir, exc_info=True | |
| ) | |
| def __del__(self): | |
| # Best-effort cleanup during garbage collection; avoid raising. | |
| try: | |
| self.close(timeout=0.1) | |
| except Exception: | |
| pass |
| Usage | ||
| ----- | ||
| >>> import cortex | ||
| >>> vol = cortex.Volume.random("S1", "fullhead") | ||
| >>> cortex.webgl.jupyter.display(vol) # auto-detects best approach | ||
| """ |
There was a problem hiding this comment.
The module docstring example says cortex.webgl.jupyter.display(vol) “auto-detects best approach”, but display() defaults to method="iframe" and does not perform any auto-detection. Please update the docstring/example to match the actual behavior (or implement auto-detection if that’s intended).
- Remove unused HTML import - Make StaticViewer.close() idempotent with server_close() and thread.join() - Add viewer registry with close_all() and atexit cleanup to prevent tmp buildup - Fix misleading docstrings (auto-detect claim, self-contained HTML claims) - Make static host configurable via CORTEX_JUPYTER_STATIC_HOST env var - Add TOCTOU race documentation to _find_free_port - Make nbsphinx a required docs dependency (fixes toctree mismatch) - Fix _copy_notebook_artifacts to skip on failure and overwrite stale builds - Remove misplaced docs/.github/workflows/build_docs.yml - Fix test_width_int_converted to actually assert IFrame width - Add tests for viewer registry and close_all
Address all issues from code review: Production fixes (cortex/webgl/jupyter.py): - Use 127.0.0.1 for IFrame URL instead of socket.gethostname() (#1) - Add CORTEX_JUPYTER_IFRAME_HOST env var for remote setups - Pass display_url=False to suppress redundant URL output (gallantlab#2) - Release _viewer_lock before blocking close() in close_all() (gallantlab#4) - Clean up HTTP server and tmpdir if ipydisplay() fails (gallantlab#7) - Clean up tmpdir if HTTPServer() constructor fails (gallantlab#8) - Guard _repr_html_() against use-after-close (gallantlab#9) - Add public `closed` property (gallantlab#10) - Add context manager protocol (__enter__/__exit__) (gallantlab#11) - Warn when server thread doesn't terminate after join (gallantlab#14) - Remove viewer from _active_viewers on close - Harden close_all() for interpreter shutdown Config fixes: - Guard nbsphinx import in docs/conf.py (gallantlab#5) - Change nbsphinx_execute to "auto" (gallantlab#6) - Protect jupyter import in __init__.py (gallantlab#16) - Remove dead notebook.html template (gallantlab#3) Tests (21 new, 1 fixed): - IFrame URL correctness and env var configurability - display_url=False verification - closed property, _repr_html_ after close - Context manager lifecycle - Double-close idempotency - Graceful degradation when httpd.shutdown() fails - Thread join timeout warning - close_all() lock scope (deadlock prevention) - Resource leak on ipydisplay/HTTPServer failure - FileNotFoundError when index.html missing - CORTEX_JUPYTER_STATIC_HOST env var - HTTP server integration test (serves content, logs 404s) - make_notebook_html kwargs forwarding - Registry removal on close - Replace flaky port uniqueness test (gallantlab#24) Docs notebook: - Add disk-write warning for static viewer - Rewrite to be executable during docs build
…widget-dvTkg * origin/main: FIX NaN values in Volume/Vertex rendering as black instead of transparent (gallantlab#612) Fix docs version display and update copyright to Gallant Lab (gallantlab#614) Disable flatmap tilt by default; respect allow_tilt checkbox in controls (gallantlab#618) Split 'r' hotkey: fold-only vs full view reset (gallantlab#617) Bump codecov/codecov-action from 5 to 6 (gallantlab#615) FIX deprecated scipy APIs (gallantlab#609) # Conflicts: # docs/conf.py
- Add output_dir parameter to display_static() so the viewer can write to a persistent directory with a relative IFrame path (no ephemeral HTTP server). This makes it work in both live Jupyter and rendered docs. - Add docs dependency group to pyproject.toml (sphinx, numpydoc, sphinx-gallery, alabaster, nbsphinx, pandoc, ipykernel). - Rewrite notebook example to use cortex.webgl.jupyter.display() API. - Add Jupyter Notebook Examples section with nbsphinx-gallery cards. - Add 5 tests for the new output_dir parameter.
New notebook showing how to use cortex.export.plot_panels with headless=True to render composite brain views without opening a browser. Demonstrates the flatmap + lateral/medial/ventral preset and all predefined panel layouts.

Summary
Add
cortex.webgl.jupytermodule for embedding the WebGL brain viewer in Jupyter notebooks:display(data)— IFrame mode: starts Tornado server in background thread, embeds via IFrame. Full interactivity with WebSocket/JSMixer control.display(data, method="static")— Static mode: generates self-contained viewer directory served via local HTTP server. ReturnsStaticViewerhandle withclose()for cleanup.make_notebook_html(data)— returns raw HTML string for custom embedding.socket.bind(('', 0))instead ofrandom.randintStaticViewerclass with:closedpropertywith viewer:)_repr_html_()that shows "closed" message after cleanup127.0.0.1by default (configurable viaCORTEX_JUPYTER_IFRAME_HOSTandCORTEX_JUPYTER_STATIC_HOSTenv vars for remote setups)ipydisplay()orHTTPServer()failsclose_all()releases lock before blocking shutdown to prevent deadlocks; hardened for interpreter teardownnbsphinxguarded withtry/exceptin docs config so local builds work without itNew/modified files
cortex/webgl/jupyter.py— main modulecortex/webgl/__init__.py— guarded import of jupyter submodulecortex/tests/test_jupyter_widget.py— 44 testsdocs/notebooks/jupyter_notebook.ipynb— example notebook (auto-executed at docs build time)docs/conf.py— optional nbsphinx integrationTest plan
python -m pytest cortex/tests/test_jupyter_widget.py -v— 44 tests passjupyter nbconvert --executedisplay(vol)in live Jupyter notebookdisplay(vol, method="static")in live Jupyter notebookviewer.close()cleans up server and temp fileswith viewer:) auto-cleanupclose_all()cleans up multiple viewersdocs/conf.pyworks with and without nbsphinx installed