feat: NotebookLM research pipeline + infographic generation#601
Merged
codercatdev merged 24 commits intomainfrom Mar 5, 2026
Merged
feat: NotebookLM research pipeline + infographic generation#601codercatdev merged 24 commits intomainfrom
codercatdev merged 24 commits intomainfrom
Conversation
Adds sceneType (narration|code|list|comparison|mockup) and type-specific objects to script.scenes for richer Remotion compositions. - code: snippet, language, highlightLines - list: items, icon - comparison: leftLabel, rightLabel, rows - mockup: deviceType, screenContent All type-specific fields are optional and conditionally hidden in Studio based on sceneType. Defaults to 'narration' for backward compatibility. Coordinated with @videopipe for Remotion composition support. Co-authored-by: content <content@miriad.systems>
… params - Add raw Cloudinary object detection (old-format docs without _type) - Add stripTransformations() to remove Cloudinary URL params - Add getOriginalUrl() to construct canonical URLs from public_id - Prevents uploading derived variants (avif, webp, resized copies) Re-run results: 433 clean originals uploaded (down from 6,970 with variants) Co-authored-by: builder <builder@miriad.systems>
Deletes unreferenced Sanity assets left over from migration. Safety: checks document references before deleting, preserves all active assets. Supports --dry-run mode. Co-authored-by: builder <builder@miriad.systems>
…s, content-type headers - Add full iTunes namespace to podcast feed (itunes:author, itunes:image, itunes:category, itunes:season, itunes:episode, enclosure tags) - Create buildPodcastFeed() with hand-crafted XML for Apple Podcasts compatibility - Add rssPodcastQuery with podcastFields (spotify, season, episode, guest) - Fix hardcoded Cloudinary image URL in feed channel - Fix content-type headers: text/xml → application/rss+xml - Fix feed links to be content-type-specific (blog, podcasts, courses) - Fix copyright year to be dynamic - Fix YouTube feed links pointing to non-existent routes
The NotebookLM BatchExecute API requires a source-path query parameter
(/notebook/{id}) for all notebook-scoped operations. Without it, the API
returns null for research, artifacts, summary, etc.
Fixed: addSource, startResearch, pollResearch, importResearchSources,
generateInfographic, generateReport, listArtifacts, getSummary
Also adds debug logging to rpcCall and pollResearch for live API debugging.
* fix: migration script downloads only originals, strips transformation params - Add raw Cloudinary object detection (old-format docs without _type) - Add stripTransformations() to remove Cloudinary URL params - Add getOriginalUrl() to construct canonical URLs from public_id - Prevents uploading derived variants (avif, webp, resized copies) Re-run results: 433 clean originals uploaded (down from 6,970 with variants) Co-authored-by: builder <builder@miriad.systems> * chore: add orphan asset cleanup script Deletes unreferenced Sanity assets left over from migration. Safety: checks document references before deleting, preserves all active assets. Supports --dry-run mode. Co-authored-by: builder <builder@miriad.systems> * fix: RSS feed improvements — podcast iTunes support, proper enclosures, content-type headers - Add full iTunes namespace to podcast feed (itunes:author, itunes:image, itunes:category, itunes:season, itunes:episode, enclosure tags) - Create buildPodcastFeed() with hand-crafted XML for Apple Podcasts compatibility - Add rssPodcastQuery with podcastFields (spotify, season, episode, guest) - Fix hardcoded Cloudinary image URL in feed channel - Fix content-type headers: text/xml → application/rss+xml - Fix feed links to be content-type-specific (blog, podcasts, courses) - Fix copyright year to be dynamic - Fix YouTube feed links pointing to non-existent routes --------- Co-authored-by: Miriad <miriad@miriad.systems> Co-authored-by: builder <builder@miriad.systems>
- Add @vercel/analytics and @vercel/speed-insights packages - Create SiteAnalytics component with Vercel + conditional Umami tracking - Add analytics to main layout and dashboard layout - Add Umami self-hosted setup documentation Vercel Analytics/Speed Insights work automatically on deploy. Umami activates when NEXT_PUBLIC_UMAMI_WEBSITE_ID env var is set.
Google rotates SIDCC, __Secure-*PSIDCC, and NID cookies on every page load. Python's httpx captures these automatically via its cookie jar, but Node.js fetch() does not. Without the fresh cookies, all RPC calls return error code [5] (null result). Now parses Set-Cookie headers from the homepage GET response and merges them into the cookie record before building the Cookie header for subsequent RPC POST requests.
The createNotebook RPC response format is [title, null, uuid, ...]. We were extracting from index 0 (the title string) instead of index 2 (the UUID). This caused all subsequent RPC calls to use the notebook title as the source-path instead of the UUID, which the API rejects with error code [5]. Also fixed listNotebooks to use the same correct index.
The live API returns a triple-nested structure: [[[reportId, taskInfo, startTs, updateTs]]] where taskInfo = [notebookId, [query, srcType], mode, sources, statusCode, researchMeta] The old parsing expected a different nesting and couldn't find the task data, always returning 'no_research'. Now correctly unwraps the triple-nesting and extracts statusCode from taskInfo[4] (1=in_progress, 2=completed) and taskId from researchMeta[0].
Before starting deep research, now adds up to 10 source URLs from trend discovery to the notebook. This gives NotebookLM real context (blog posts, HN links, etc.) to research from, matching the workflow shown in the video. Also increased research timeout from 5min to 10min since deep research can take 5-10 minutes to complete.
Extracts up to 5 source URLs from the top trending topic's signals and passes them to conductResearch() as sourceUrls. These get added to the NotebookLM notebook before deep research starts, giving it real articles/blog posts to analyze.
…cts, fix nesting - Add getSourceIds() method that fetches source IDs from notebook via GET_NOTEBOOK RPC - generateInfographic() and generateReport() now auto-fetch source IDs if not provided - Fix sourceIdsTriple nesting: was [[[sid]]] (4 levels), now [[sid]] (matching Python's [[[sid]] for sid in source_ids]) Co-authored-by: research <research@miriad.systems>
…Urls array - Generate 5 infographics with different angles (architecture, comparison, workflow, timeline, pros/cons) - Change infographicUrl to infographicUrls (string array) in ResearchPayload - Collect URLs for all completed infographics - Generate report in parallel with infographics Co-authored-by: research <research@miriad.systems>
…load Co-authored-by: research <research@miriad.systems>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
added 4 commits
March 5, 2026 02:16
…ed system instruction
…ring + idempotency
…nts + migration scripts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merges the complete NotebookLM research pipeline into main. This adds automated trend discovery, deep research, and infographic generation to the video content pipeline.
What's New
NotebookLM Client (
lib/services/notebooklm/)types.ts— RPC method IDs, artifact types, interfacesrpc.ts— BatchExecute encoding/decoding, sharedfetchWithTimeout()andsleep()auth.ts— Cookie parsing, CSRF extraction, fresh cookie capture from homepageclient.ts— Full client: notebook CRUD, source management, deep research, infographic/report generation, artifact pollingindex.ts— Barrel exportsResearch Service (
lib/services/research.ts)ResearchPayloadinterface withinfographicUrls: string[]for multiple infographicssafeStep()try/catchTrend Discovery (
lib/services/trend-discovery.ts)Ingest Route (
app/api/cron/ingest/route.ts)discoverTrends()replaces oldfetchTrendingTopics()conductResearch()gated byENABLE_NOTEBOOKLM_RESEARCH=trueFALLBACK_TRENDSfor graceful degradationTest Results (Local)
Full end-to-end run completed successfully:
Environment Variables
ENABLE_NOTEBOOKLM_RESEARCH"true"to enable research pipelineNOTEBOOKLM_AUTH_JSON/path/to/file), or base64 (eyJ...)Debug Logging
Debug logging is intentionally kept for Vercel log inspection. All NotebookLM RPC calls log URL, body length, and response size/preview.
Also Includes