# CLAUDE.md — Context for AI Assistant
This file helps resume development in a new conversation. Read DESIGN.md first
for full architectural context.
---
## What this project is
A Python/Playwright scraper for `grok.com/imagine/saved` — the favorites page
of Grok's AI image/video generator. Downloads all favorited images and videos
with metadata into SQLite + local files.
## Current state
Pass 1 runs and authenticates correctly. The scroll loop exits after one
attempt due to the virtualized masonry layout — only ~47 of hundreds of cards
are captured. **This is the primary thing to fix next.**
Passes 2 and 3 are coded but untested.
---
## The immediate problem to solve
### Virtualized masonry in Pass 1
The favorites page mounts/unmounts cards as you scroll (virtual DOM). Only
cards near the viewport exist in the DOM at any time.
`document.body.scrollHeight` doesn't grow, so the scroll loop exits instantly.
**Preferred solution: XHR interception**
The React app fetches favorites in batches via an internal API during scrolling.
Playwright can intercept these with:
```python
async def handle_response(response):
if 'some-api-path' in response.url:
data = await response.json()
# process batch of favorites
page.on('response', handle_response)
# register BEFORE page.goto()
```
**We don't yet know the API endpoint URL.** The human needs to:
1. Open Chrome DevTools → Network tab → filter XHR/Fetch
2. Load `grok.com/imagine/saved` while logged in
3. Scroll through all favorites
4. Identify the API calls that return favorites data
5. Report the URL, request shape, and response shape
Once the endpoint is known, `fetch_favorites()` (which was removed as
speculation) can be rewritten with the correct URL and payload.
**Alternative solution: incremental DOM scraping**
If XHR interception proves difficult, scroll the masonry container element
(not `document.body`) and capture cards incrementally — after each scroll
step, query listitems, process new UUIDs not yet seen, continue. The container
uses `position: absolute` children at fixed pixel offsets (visible in the
listitem style attributes).
---
## Key technical findings
### Card DOM structure
**Video card** (has both `![]()
` and `