# PRODUCT REQUIREMENTS DOCUMENT — AS-BUILT
# Instagram Harvest

**Version:** 1.0  
**Type:** Personal Automation / AI Content Pipeline  
**Status:** Production  
**Purpose:** Public reference PRD — safe to share. Contains no private credentials, IDs, or personal information.

---

## 1. Executive Summary

**Product Name:** Instagram Harvest  
**Current Status:** Live, production  
**Product Type:** Personal automation tool / AI-assisted content pipeline  
**Primary Objective:** Automatically ingest Instagram saved posts into a structured Notion database and use Claude AI to transform them into production-ready content ideas  
**Current Users:** Solo content creators and personal brand builders  
**Business Context:** Content creators save posts on Instagram for inspiration but have no system to process them. Instagram Harvest closes the gap between passive consumption and active content production.  
**Technical Maturity:** MVP — fully functional, single-user, local deployment on macOS

---

## 2. Problem Statement

**User Problem:**  
Content creators consume large volumes of inspiration daily through Instagram — posts, reels, carousels — but have no reliable system to capture, process, and convert that inspiration into content. Saved posts rot inside Instagram's native saved folder, disconnected from any production workflow or AI tool.

**Business Problem:**  
Without a system to process inspiration, creators spend hours every week manually searching for content ideas — re-scrolling the same feeds, starting from scratch, and experiencing creative block despite having already encountered relevant material.

**Current Impact:**  
- Saved posts are never revisited after saving
- Weekly content planning requires hours of doomscrolling for ideas already in the save folder
- No connection exists between inspiration collection and content production
- AI tools cannot access Instagram saved posts natively

**Evidence:** Assumption — validated through personal experience  
**Constraint:** Instagram does not provide an official public API for saved posts. The system uses the internal web API via session cookies.

---

## 3. Target Users

**Primary Persona:**  
Solo content creator building a personal brand. Posts regularly on Instagram, TikTok, and/or YouTube. Uses Notion as their primary workspace. Has a defined content niche and at least one content pillar system.

**Secondary Persona:**  
Anyone who uses Instagram's save feature to bookmark inspiration and wants to turn those saves into structured content briefs.

**Admin/Internal Users:**  
None — single-user tool. The creator is the only operator.

**User Context:**  
- User scrolls Instagram regularly and saves posts they find inspiring or interesting
- User works in Claude Code and has a Notion workspace
- User runs on macOS (launchd scheduler dependency)
- User has a defined content niche and target audience

**Jobs to Be Done:**  
- "When I save something on Instagram, I want it to automatically appear in Notion so I don't have to do it manually."
- "When I sit down to plan content, I want a ready list of processed ideas — not raw saves."
- "I want Claude to help me reframe things I've seen into original content angles for my audience."

---

## 4. Current Product Overview

**Core Capabilities:**
1. Automatic twice-daily sync from Instagram saved posts to a Notion database
2. Collection-based filtering — only pulls saves from specified Instagram collections
3. Deduplication — each save is written to Notion exactly once, regardless of how many times the sync runs
4. On-demand AI ideation — Claude transforms raw saves into structured content ideas
5. Full content brief generation per idea — hooks, outline, platform breakdown, priority
6. Notion-native storage — both saves and ideas live in queryable Notion databases
7. Slash command interface inside Claude Code for manual operations

**Main Workflows:**
1. **Sync:** Instagram saved posts → filtered → deduplicated → written to Notion (Status: New)
2. **Ideate:** Notion saves (Status: New) → Claude analysis → content ideas → written to Content Ideas DB → saves marked Used/Reviewed
3. **Maintain:** Refresh expired Instagram session, check scheduler status, view recent saves

**Key Screens/Interfaces:**
- Notion: Instagram Saves database (raw saves inbox)
- Notion: Content Ideas database (processed, production-ready ideas)
- Claude Code: `/instagram-sync` slash command menu
- Terminal: `sync.py` output and `sync.log`

**Main Objects:**
- **Save** — a single Instagram post pulled from the saved feed
- **Content Idea** — a fully developed content brief generated from one or more saves
- **Collection** — an Instagram folder (e.g., "Claude Code", "Inspiration") used to filter saves

**Current Limitations:**
- macOS only (launchd scheduler)
- Single Instagram account
- Single Notion workspace
- No web UI — operated via Claude Code and Notion
- Instagram session cookies expire and must be manually refreshed (~every 60–90 days)
- Uses Instagram's unofficial internal web API — not the public Graph API

---

## 5. Value Proposition

**Primary Benefit:**  
Turns passive Instagram scrolling into an active, structured content pipeline with zero behavior change required from the creator.

**Secondary Benefits:**
- Eliminates weekly "what should I post?" paralysis
- Ensures saved inspiration is actually used
- Produces complete content briefs, not just topic ideas
- Claude reframes source material for the creator's specific audience and niche
- All ideas stored in Notion with hooks, outlines, and platform breakdowns ready to execute

**Differentiators:**
- Runs automatically — no manual export or copy-paste
- AI-powered reframing, not just categorization
- Collection-gated — only pulls saves the creator intentionally curated
- Writes directly to Notion in a structured, queryable schema

**Current Alternatives:**
- Manually revisiting the Instagram saved folder (high friction, rarely done)
- Screenshot-based save systems (unstructured, unsearchable)
- Third-party social media tools (don't include AI ideation, don't filter by collection)

---

## 6. Scope

### 6.1 In Scope Today
- Instagram saved posts ingestion via internal web API
- Collection-based filtering (allowlist of named collections)
- Notion database storage with full schema (saves inbox + ideas DB)
- Twice-daily automated sync via macOS launchd
- Claude Code slash command with 6 actions
- AI-powered content idea generation (title, hooks, outline, format, platform, pillar, priority)
- Full content brief written to Notion page body (outline + platform breakdown)
- Status lifecycle management (New → Used / Reviewed)
- Deduplication via local state file
- Session validation before every sync run
- Human-in-the-loop approval during ideation

### 6.2 Out of Scope Today
- Web or mobile UI
- Multi-account Instagram support
- Multi-user or team access
- Publishing or scheduling to social platforms
- TikTok, YouTube, or Twitter bookmark ingestion
- Automatic ideation without user approval
- Analytics on content performance
- Email or push notifications
- Windows or Linux deployment
- AI-generated full scripts or captions (only briefs)

### 6.3 Future Scope
- Multi-platform ingestion (TikTok bookmarks, YouTube Watch Later, Twitter/X bookmarks)
- Fully automated ideation (no approval step) as an optional mode
- Full draft generation (script/caption) after idea approval
- Social publishing integration (Buffer, Later)
- Performance feedback loop from published content back to Notion
- Personal voice profiling so Claude writes ideas in the creator's existing style

---

## 7. Functional Requirements

### FR-01 — Session Validation
**Description:** Before any Instagram data is fetched, the system must validate that the current session cookies are still active.  
**Current Status:** Implemented  
**User Story:** As a creator, I want the sync to fail clearly with a helpful message when my Instagram session has expired, so I know exactly what to fix.  
**Acceptance Criteria:**
- The system calls `GET /api/v1/accounts/edit/web_form_data/` before fetching saves
- HTTP 200 → log the authenticated username and continue
- HTTP 401 → exit with a human-readable "session expired" message pointing to the refresh workflow
- Any other HTTP status → exit with the status code for diagnosis
- No saves are fetched until session is confirmed valid

**Edge Cases:**
- Intermittent network failure may return non-401 error — system should not assume the session is expired
- Session may be partially expired (some endpoints work, others don't)

---

### FR-02 — Collections Fetch
**Description:** The system must fetch the user's Instagram collection list to build a name-to-ID map used for filtering.  
**Current Status:** Implemented  
**User Story:** As a creator, I want only saves from my curated collections to be ingested, not everything I've ever saved.  
**Acceptance Criteria:**
- System calls `GET /api/v1/collections/list/` with `collection_types=["ALL_MEDIA_AUTO_COLLECTION","PRODUCT_AUTO_COLLECTION","MEDIA"]`
- Builds a `{collection_id: collection_name}` lookup dictionary
- If the collections endpoint fails, log a warning and continue without filtering (do not abort the sync)

---

### FR-03 — Collection Filter Enforcement
**Description:** Only saves explicitly belonging to one of the configured collections must be written to Notion.  
**Current Status:** Implemented  
**User Story:** As a creator, I want to control exactly which saved posts get ingested by specifying collection names in the config.  
**Acceptance Criteria:**
- If `collections_filter` is empty, all saves are processed (no filtering)
- If `collections_filter` is non-empty:
  - Posts with no `saved_collection_ids` (not in any collection) must be **skipped**
  - Posts in collections not on the allowlist must be **skipped**
  - Only posts where at least one collection name matches the allowlist are processed
- The filter check uses collection **names** (human-readable), not IDs

**Critical Note — Known Bug History:**  
An earlier implementation used `if collections_filter and saved_collection_ids:` which incorrectly allowed uncollected posts through when `saved_collection_ids` was empty. The correct logic is:

```python
if collections_filter:
    if not saved_collection_ids:
        skip_count += 1
        continue  # post not in any collection — always skip when filter is active
    matched_any = any(
        collections_map.get(str(cid)) in collections_filter
        for cid in saved_collection_ids
        if collections_map.get(str(cid))
    )
    if not matched_any:
        skip_count += 1
        continue
```

**Edge Cases:**
- A post may belong to multiple collections — it passes the filter if ANY of its collections is on the allowlist
- Collection names are case-sensitive and must match exactly as they appear in Instagram

---

### FR-04 — Saved Posts Pagination
**Description:** The system must paginate through the Instagram saved feed to fetch all available saves up to the configured page limit.  
**Current Status:** Implemented  
**User Story:** As a creator, I want all recent saves to be captured, not just the first page.  
**Acceptance Criteria:**
- System calls `GET /api/v1/feed/saved/posts/?count=50&max_id={cursor}`
- Uses `next_max_id` from the response as the cursor for the next page
- Stops when `next_max_id` is absent (no more pages) or `max_pages` limit is reached
- Sleeps 1 second between page requests to avoid rate limiting
- If a page request fails, logs the error and stops pagination (does not abort the entire sync)
- Default `max_pages`: 10 (up to 500 saves per run)

---

### FR-05 — Deduplication
**Description:** Each Instagram post must be written to Notion exactly once, regardless of how many times the sync runs.  
**Current Status:** Implemented  
**User Story:** As a creator, I want to run the sync multiple times without creating duplicate entries in Notion.  
**Acceptance Criteria:**
- System maintains `state.json` with a set of all previously synced `media.pk` IDs
- Before writing a post to Notion, check if its ID is already in state
- If ID exists in state, skip the post (increment skip counter)
- State is written to disk **immediately after each individual successful Notion write** (not at the end of the run)
- If the sync crashes mid-run, already-processed posts are not re-processed on the next run

**Edge Cases:**
- If `state.json` is deleted, all posts up to `max_pages` will be re-evaluated (may create duplicates in Notion)
- State file is local-only — not synced or backed up

---

### FR-06 — Notion Write (Instagram Saves DB)
**Description:** Each qualifying save must be written to the Instagram Saves Notion database with the correct schema.  
**Current Status:** Implemented  
**User Story:** As a creator, I want each Instagram save to appear in Notion with enough context to evaluate it for content ideas.  
**Acceptance Criteria:**
- Each page is created with these properties:
  - **Name** (title): `@username — first 60 characters of caption` (falls back to `@username` if no caption)
  - **URL** (url): full Instagram URL constructed from `media.code` — `/reel/{code}/` for Reels, `/p/{code}/` for Posts and Carousels
  - **Type** (select): Post, Reel, Carousel, or IGTV mapped from `media.media_type` and `product_type`
  - **Author** (rich_text): `media.user.username`
  - **Status** (select): hardcoded `"New"` on creation
  - **Media ID** (rich_text): `media.pk` — used for dedup
  - **Saved** (date): `media.taken_at` converted to UTC ISO 8601
  - **Caption** (rich_text): `media.caption.text` truncated to 1900 characters (Notion rich_text limit)
  - **Collection** (select): first matched collection name from allowlist (omitted if no collection matched)
- If the Notion write fails for a post, log the error, do NOT add the post to state, and continue to the next post

**Note on Notion SDK:**  
`notion-client` Python SDK v3.1.0 does NOT have a `databases.query()` method. All Notion database queries must use `requests.post()` directly:
```python
requests.post(
    f"https://api.notion.com/v1/databases/{db_id}/query",
    headers=headers,
    json=body
)
```

---

### FR-07 — Sync Summary Log
**Description:** Every sync run must produce a structured summary log line.  
**Current Status:** Implemented  
**Acceptance Criteria:**
- Final log line format: `Sync complete: {new} new | {skipped} skipped | {total} total | {errors} errors | {duration}s`
- Log is appended to `sync.log` (never overwritten)
- Log is also printed to stdout
- All timestamps in log are local time with timezone

---

### FR-08 — Scheduled Sync (launchd)
**Description:** The sync must run automatically twice daily without user intervention.  
**Current Status:** Implemented  
**User Story:** As a creator, I want my Instagram saves to appear in Notion automatically so I never have to remember to run the sync manually.  
**Acceptance Criteria:**
- sync.py runs at 09:00 and 21:00 daily via macOS launchd
- launchd plist uses absolute paths (launchd does not source shell profiles)
- WorkingDirectory is set to the project root so relative paths in sync.py resolve correctly
- stdout and stderr from scheduled runs are written to separate log files
- Scheduler survives system restarts (plist loaded at user login)

---

### FR-09 — Slash Command — Run Sync
**Description:** User can manually trigger a sync from within Claude Code.  
**Current Status:** Implemented  
**Acceptance Criteria:**
- `/instagram-sync` → Action 1 runs `sync.py` using the venv Python interpreter
- Full sync output is displayed
- If sync produces N > 0 new saves, automatically chains to the Ideate action

---

### FR-10 — Slash Command — Ideate
**Description:** Claude queries Notion for unprocessed saves and generates structured content ideas for the creator's specific niche and audience.  
**Current Status:** Implemented  
**User Story:** As a creator, I want Claude to turn my raw saves into fully developed content briefs so I can immediately start producing content.  
**Acceptance Criteria:**
- Query Instagram Saves DB for `Status = "New"` rows
- If more than 10 saves pending, process in batches of 10 with user approval between batches
- Silently skip saves clearly irrelevant to the creator's niche (mark as Reviewed)
- For each relevant save, generate:
  - Content title
  - Pillar assignment (based on collection and caption content)
  - Priority (High / Medium / Low)
  - Format (Carousel / Reel / Short Video / Long-form Video)
  - Platform recommendations (Instagram / TikTok / YouTube)
  - 3 hooks: Curiosity-driven, Value-driven, Emotional
  - Structured outline: HOOK → 3-4 key points → CTA
  - Per-platform production notes
- Present all ideas numbered for user review
- Accept: `all`, comma-separated numbers (e.g. `1,3,5`), `skip`, or modification requests
- For each approved idea, create a page in the Content Ideas Notion database
- Mark source saves: `Used` if approved, `Reviewed` if skipped
- Print final summary: saves processed, ideas generated, ideas saved, titles

**Content Idea Notion Page Properties:**
- Name (title)
- Pillar (select): configured by creator
- Platform (multi_select)
- Format (select)
- Status (select): `"Not started"` on creation
- Angle (text): 1-2 sentence positioning
- Hook Options (text): all 3 hooks joined by ` | `
- Priority (select)
- Week Of (date): ISO date of the Monday of the current week
- Page body: full outline + platform breakdowns + source link

---

### FR-11 — Slash Command — Check Sync Status
**Description:** User can check the result of the last sync run from within Claude Code.  
**Current Status:** Implemented  
**Acceptance Criteria:**
- Read last 20 lines of `sync.log`
- Display the most recent `Sync complete:` line prominently
- Display any `ERROR` lines
- Report whether errors need attention

---

### FR-12 — Slash Command — Check Scheduler
**Description:** User can verify that the launchd scheduler is active.  
**Current Status:** Implemented  
**Acceptance Criteria:**
- Run `launchctl list | grep {plist-label}`
- If present with exit code 0: confirm running normally
- If absent: provide install instructions
- If non-zero exit code: direct user to sync.log

---

### FR-13 — Slash Command — Refresh Session
**Description:** User can follow a guided walkthrough to update expired Instagram session cookies.  
**Current Status:** Implemented  
**Acceptance Criteria:**
- Step-by-step instructions: Chrome → instagram.com → DevTools → Application → Cookies → copy 3 values → update config.json
- Instructions are browser-specific (Chrome)
- After update, prompt user to run Action 1 to confirm the new session works

---

### FR-14 — Slash Command — View Recent Saves
**Description:** User can view the 10 most recent saves in Notion from within Claude Code.  
**Current Status:** Implemented  
**Acceptance Criteria:**
- Query Instagram Saves DB, sorted by Saved date descending, limit 10
- Display as table: #, Name, Author, Type, Collection, Status
- If any Status = "New" entries exist, note the count and suggest running Ideate

---

## 8. Non-Functional Requirements

### Performance
- A full sync of 500 posts (10 pages × 50 posts) must complete in under 5 minutes under normal network conditions
- Instagram pagination includes a mandatory 1-second sleep between pages (rate limit protection) — this contributes ~9 seconds of intentional delay for 10 pages
- Notion API write rate: one page creation per save (sequential, not batched) — no parallelism to avoid hitting Notion rate limits
- State file read and write must complete in under 100ms (local I/O)

### Scalability
- The system is designed for a single Instagram account and a single Notion workspace
- `max_pages` is configurable — increasing it scales the number of saves processed per run but increases runtime linearly
- `state.json` grows unbounded over time; for long-running use, it may contain thousands of IDs (acceptable for a single-user tool)

### Security
- All credentials (Instagram cookies, Notion API key, database IDs) must be stored in `config.json` only
- `config.json` must be listed in `.gitignore` and never committed to version control
- `config.json` must not be uploaded, shared, or logged
- The Notion API key must use an internal integration token with the minimum required permissions (read/write to specific databases only)
- All API communication uses HTTPS
- Session cookies are treated as passwords — same security posture as a login credential

### Availability
- The scheduled sync is best-effort — if the machine is asleep at 09:00 or 21:00, launchd does not catch up
- No uptime SLA — personal tool, acceptable for occasional sync gaps
- If Instagram's internal API changes, syncs may silently return empty results or errors — monitor `sync.log`

### Reliability
- State is written after each individual post (crash-safe deduplication)
- A failed Notion write for one post does not abort the remaining posts
- A failed Instagram page fetch stops pagination but does not abort posts already fetched
- Session expiry produces a clear exit message, not a silent failure

### Compliance
- The tool uses Instagram's unofficial internal web API — it is not authorized by Meta's Terms of Service for programmatic access
- The creator assumes full responsibility for usage
- No user data beyond the creator's own saved posts is accessed
- No data is shared with third parties

### Observability
- All sync events logged to `sync.log` (append mode) with timestamps
- Final summary line per run: new / skipped / total / errors / duration
- Individual post saves logged with media ID and username
- ERROR lines logged for every failed Notion write with the exception message
- launchd stdout/stderr captured in separate log files for scheduled runs

### Maintainability
- All configuration in a single file (`config.json`) — no hardcoded values in source code
- Collection filter, max pages, and all Notion IDs are configurable without code changes
- Single Python file (`sync.py`) for the sync logic — no framework, no ORM, minimal dependencies
- Dependencies: `requests`, `notion-client` (two packages only)

---

## 9. System Architecture

**Frontend:** None — Notion databases serve as the UI for viewing data; Claude Code slash command is the operator interface  
**Backend:** Python 3.9 script (`sync.py`), run via CLI or launchd  
**Database:** None — Notion is the data store  
**Authentication (Instagram):** Session cookie authentication — `sessionid`, `csrftoken`, `ds_user_id` cookies set from a Chrome browser session  
**Authentication (Notion):** Internal integration token (`ntn_...`) — Bearer token in Authorization header  
**Authorization:** Single-user, no role system  
**APIs:**
- Instagram Web API (unofficial, internal): base URL `https://www.instagram.com`
- Notion REST API v1: base URL `https://api.notion.com/v1`

**Third-Party Services:**
- Instagram (data source)
- Notion (data storage and UI)
- Claude (AI reasoning, accessed via Claude Code slash command — not a direct API integration)

**Hosting/Infrastructure:** Local macOS machine (no cloud deployment)  
**Scheduler:** macOS launchd — `StartCalendarInterval` at Hour=9 and Hour=21  
**Background Jobs:** sync.py running as a launchd agent  
**File Storage:** Local filesystem only (`state.json`, `sync.log`, `config.json`)  
**AI Components:** Claude (Anthropic) — used interactively via Claude Code, not via API  
**External Dependencies:**
- Python 3.9+ with venv
- `requests` library
- `notion-client` library (v3.1.0 — note: `databases.query()` method missing, use `requests.post` directly)
- macOS (launchd requirement)

**Notion CLI (supplementary):**  
A Go binary (`notion-pp-cli`) generated from a custom OpenAPI 3.0 spec, used for ad-hoc Notion queries. Generated via `cli-printing-press` tool. Optional — not required for core pipeline operation.

---

## 10. Data Model Overview

### Entity: Save (Instagram Saves Database)

| Property | Type | Description |
|----------|------|-------------|
| Name | title | `@username — first 60 chars of caption` |
| URL | url | Full Instagram post or reel URL |
| Type | select | Post / Reel / Carousel / IGTV |
| Author | rich_text | Instagram username of the post creator |
| Status | select | New / Reviewed / Used |
| Media ID | rich_text | Instagram `media.pk` — used for dedup |
| Saved | date | UTC ISO datetime from `media.taken_at` |
| Caption | rich_text | Post caption, truncated to 1900 chars |
| Collection | select | Instagram collection name the post was saved into |

**Status Lifecycle:**
```
New → Used      (idea was approved and written to Content Ideas DB)
New → Reviewed  (save was seen during Ideate but no idea was created)
```

### Entity: Content Idea (Content Ideas Database)

| Property | Type | Description |
|----------|------|-------------|
| Name | title | Content idea title |
| Platform | multi_select | Instagram / TikTok / YouTube |
| Format | select | Carousel / Reel / Short Video / Long-form Video |
| Status | select | Not started / In progress / Done |
| Angle | text | 1-2 sentence positioning/angle |
| Hook Options | text | All 3 hooks joined by ` \| ` |
| Priority | select | High / Medium / Low |
| Week Of | date | ISO date of Monday of the target production week |
| Pillar | select | Creator-defined content pillars (e.g. Teach / Tools / Process) |

**Page Body per Idea:**
- `## Outline` — HOOK + 3-4 key points + CTA
- `## Platform Breakdown` — per-platform production notes
- `## Source` — link to original Instagram post

### Entity: State (state.json — local file)

```json
{
  "synced_ids": ["media_pk_1", "media_pk_2", "..."]
}
```

**Relationships:**
- One Save → zero or one Content Idea (via ideation)
- One Content Idea → one Save (source attribution via URL in page body)
- State file tracks all Saves ever written to Notion (prevents duplicates)

**Sensitive Data:** None in Notion databases. Credentials live only in `config.json` (local, git-ignored).  
**Data Retention:** No automatic deletion. Saves and ideas accumulate indefinitely in Notion.  
**Privacy:** Only the creator's own saved posts are processed. No other user data is accessed.  
**Ownership:** All data belongs to the creator operating the system.

---

## 11. User Flows

### Flow 1 — Automated Daily Sync
**Trigger:** launchd fires at 09:00 or 21:00  
**Steps:**
1. launchd executes `.venv/bin/python3 sync.py`
2. Config and state are loaded
3. Instagram session is validated
4. Collections list is fetched and mapped
5. Saved posts are fetched (up to `max_pages` pages, 50 per page)
6. Each post is evaluated: already synced? skip. Not in allowed collection? skip. Else write to Notion.
7. State is updated after each successful write
8. Summary is logged to `sync.log`

**Success State:** N new saves appear in Notion with Status = "New"  
**Failure States:**
- Session expired → exit with message, no saves written
- Instagram pagination fails → stops at last successful page, logs error
- Notion write fails per post → error logged, post skipped, sync continues

---

### Flow 2 — On-Demand Ideation
**Trigger:** User runs `/instagram-sync` in Claude Code and selects Action 2 (Ideate)  
**Steps:**
1. Claude queries Notion Instagram Saves DB for Status = "New"
2. If 0 saves: inform user and stop
3. If > 0 saves: process in batches of 10
4. For each batch, Claude generates content ideas reframed for the creator's niche
5. Claude presents numbered ideas with full briefs
6. User responds: `all`, numbers, `skip`, or modification request
7. Approved ideas are written to Content Ideas Notion DB
8. Source saves are marked Used (approved) or Reviewed (skipped)
9. Summary is printed

**Success State:** Approved ideas appear in Content Ideas DB with Status = "Not started", source saves marked Used  
**Failure States:**
- No New saves → inform user, suggest running sync first
- Notion write fails for an idea → log error, continue to next idea
- User skips all → all saves marked Reviewed, Content Ideas DB unchanged

---

### Flow 3 — Session Refresh
**Trigger:** Sync fails with "session expired" message, or user proactively refreshes  
**Steps:**
1. User runs `/instagram-sync` → Action 5
2. Claude presents step-by-step instructions for extracting new cookies from Chrome
3. User opens Chrome → instagram.com → DevTools → Application → Cookies
4. User copies `sessionid`, `csrftoken`, `ds_user_id`
5. User updates `config.json` with new values
6. User runs Action 1 to confirm new session works

**Success State:** Sync runs successfully, logs username confirmation  
**Failure State:** New cookies also fail → may indicate account lockout or IP-based block

---

### Flow 4 — First-Time Setup
**Trigger:** New user setting up Instagram Harvest for the first time  
**Steps:**
1. Clone/copy project files
2. Create Python venv: `python3 -m venv .venv && .venv/bin/pip install -r requirements.txt`
3. Copy `config.example.json` to `config.json`
4. Fill in `config.json`: Instagram cookies, Notion API key, database IDs
5. Create two Notion databases with the required schemas (see Section 10)
6. Connect the Notion integration to both databases (••• → Connections in Notion)
7. Run `python sync.py` manually to verify the first sync
8. Install launchd scheduler: `cp com.*.plist ~/Library/LaunchAgents/ && launchctl load ~/Library/LaunchAgents/com.*.plist`
9. Customize `.claude/commands/instagram-sync.md` with creator's niche and content pillars

**Success State:** First saves appear in Notion, scheduler is running, slash command works  
**Failure State:** See Section 14 (Risks and Mitigations)

---

## 12. Edge Cases

| Edge Case | Current Behavior |
|-----------|-----------------|
| Post with no caption | Name falls back to `@username` only, Caption property is omitted |
| Post in multiple collections | First matching collection name is used; post passes filter if any collection is on allowlist |
| Post not in any collection, filter active | Always skipped (critical — see FR-03) |
| `saved_collection_ids` key missing from API response | Treated as empty list — post is skipped when filter is active |
| Notion rate limit hit | Notion API returns 429 — `notion-client` SDK may or may not retry; current code does not handle this explicitly |
| `state.json` deleted or corrupted | All posts up to `max_pages` re-evaluated; may create duplicates in Notion |
| Instagram returns empty `items` array | Page loop ends normally; no error logged |
| Caption longer than 1900 characters | Truncated at 1900 chars (Notion rich_text limit per block) |
| `media.taken_at` is 0 or missing | Date defaults to epoch (1970-01-01T00:00:00Z) — no special handling |
| `media.code` is missing | URL constructed as `https://www.instagram.com/p//` — broken link |
| Machine is asleep at scheduled sync time | launchd misses the trigger — no catch-up; next scheduled run fires normally |
| Notion integration not connected to database | API returns 404 — sync exits with error on first write attempt |
| `collections_filter` contains a name that no longer exists in Instagram | No saves match — sync runs but produces 0 new saves (no error) |
| Batch approval with modifications requested | Claude re-generates modified ideas in the same session before asking for final approval |

---

## 13. Known Issues and Technical Debt

| Issue | Severity | Notes |
|-------|----------|-------|
| `notion-client` v3.1.0 lacks `databases.query()` | Critical | Must use `requests.post()` directly for all DB queries. Upgrading the SDK or pinning a version with the method would fix this. |
| No Notion-level dedup check | Medium | If `state.json` is lost, duplicate pages may be created. A pre-write check against Notion (query by Media ID) would prevent this. |
| `media.taken_at` ≠ actual save date | Low | Notion's "Saved" date reflects when the post was created, not when it was saved. Instagram's API does not expose the save timestamp. |
| No retry logic for Notion 429 errors | Medium | If Notion rate-limits the sync, posts are silently skipped. Exponential backoff on 429 should be added. |
| launchd does not catch up missed runs | Low | If the machine is asleep at sync time, the run is skipped silently. Acceptable for personal use. |
| No Notion-level duplicate prevention | Medium | Two rapid manual sync runs could theoretically write the same post twice if the first run hasn't updated state yet (race condition, unlikely in practice). |
| Instagram internal API is unofficial | High risk | Any Instagram API change may silently break the sync. No official support or versioning guarantee. |
| Session cookie manual refresh (~60–90 days) | Medium | No automation or alert for session expiry. Creator must notice the failure in `sync.log`. |
| Single `sync.log` grows unbounded | Low | No log rotation implemented. Long-running use will produce a very large log file. |
| Notion CLI (`notion-pp-cli`) is supplementary | Low | Binary is platform-specific (macOS/Linux). Rebuild required after Go version changes or platform migration. |

---

## 14. Risks and Mitigations

| Risk | Impact | Likelihood | Mitigation |
|------|--------|------------|------------|
| Instagram changes internal API structure | Sync breaks entirely | Medium | Monitor sync.log for unexpected HTTP errors; be prepared to update endpoint URLs or request format |
| Instagram session expires without notice | Sync silently produces 0 new saves | High | Check sync.log weekly; consider adding an alert (e.g., email or notification) when session fails |
| Notion API key revoked or integration disconnected | Sync fails to write | Low | Reconnect integration in Notion UI; verify key in config.json |
| `config.json` accidentally committed to git | Credential exposure | Low but critical | Always verify `.gitignore` before any `git add`; use `git status` to audit before committing |
| `state.json` deleted accidentally | Duplicate Notion pages | Low | Treat state.json as important data; back up periodically or implement Notion-level dedup |
| macOS system update disables launchd agent | Sync stops running automatically | Low | Check `launchctl list` after major macOS updates; reload plist if needed |
| Notion rate limits during large syncs | Posts silently skipped | Low | Add retry with backoff on 429 responses; keep `max_pages` at a reasonable value |
| Creator changes Instagram collection names | Filter stops matching | Medium | Update `config.json → sync.collections_filter` to match new names exactly |

---

## 15. Assumptions

- The creator is the sole operator — no multi-user or team scenarios
- The machine running the scheduler is a personal macOS computer that is on during scheduled sync windows
- The creator has an active Instagram account with saved posts organized into named collections
- The creator has a Notion workspace with an internal integration that can write to two databases
- The creator's content niche and content pillars are stable enough to define in the slash command prompt
- Instagram's internal web API continues to return saved posts in the current format
- `media.pk` is a stable, globally unique identifier for Instagram posts
- Notion's internal Markdown format is sufficient for storing content briefs (no rich media required in idea pages)
- Claude Code is installed and configured as the operator's primary development interface
- AI-generated content ideas always require human review before being saved (no fully automated mode)
- The creator's Notion integration token has permission to create and update pages in both databases
- A Python 3.9+ interpreter is available on the system

---

## 16. Decision Log

| Decision | Reason | Alternatives Considered | Notes |
|----------|--------|------------------------|-------|
| Use Instagram's internal web API instead of official Graph API | The official Graph API does not expose saved posts | Official API, third-party scraping tools | Unofficial API risk acknowledged by the creator |
| Session cookie auth instead of OAuth | Only option available for personal saved post access | OAuth (not available for saved posts) | Cookies must be refreshed manually every ~60-90 days |
| Python for sync script | Simple, readable, fast to write, good library support | Node.js, Go, shell script | Single-file implementation, minimal dependencies |
| `requests.post` directly for Notion queries instead of SDK | `notion-client` v3.1.0 lacks `databases.query()` | Upgrading SDK, forking SDK | Pragmatic workaround; SDK still used for page creation |
| State written per-post instead of end-of-run | Crash safety — partial runs don't re-process successful writes | Write state at end of run | Critical for reliability |
| launchd instead of cron | More reliable on macOS, integrates with user session | cron, Automator, third-party schedulers | launchd is the macOS-native approach |
| Notion as both storage and UI | No web UI needed — Notion provides filtering, sorting, and views | Custom database + admin panel | Reduces build complexity to zero for the UI layer |
| Claude Code slash command instead of a web interface | Creator already works in Claude Code; no new tool needed | Web app, mobile app, Telegram bot | Fits the creator's existing workflow |
| Human approval step in Ideate | AI-generated content ideas require editorial judgment | Fully automated pipeline | Creator wants to review every idea before it enters production |
| Batches of 10 in Ideate | Manageable review session, avoids overwhelming the user | Process all at once, one by one | Balance between throughput and review quality |
| Content Ideas as separate Notion DB from Saves | Different lifecycle, different schema, different use (inbox vs. production) | Single combined table | Clean separation of concerns |

---

## 17. Success Metrics

### Product Metrics
- **Saves captured per week:** Number of Instagram saves successfully written to Notion (target: 100% of saves from allowed collections)
- **Ideas generated per ideation session:** Number of content ideas produced per `/instagram-sync ideate` run
- **Approval rate:** Percentage of generated ideas approved by the creator (signal of relevance quality)
- **Zero status backlog:** Instagram Saves DB has 0 rows with Status = "New" after each ideation session
- **Sync reliability:** Percentage of scheduled sync runs that complete successfully (target: >95%)

### Technical Metrics
- **Sync duration:** Time to complete a full 10-page sync (target: under 5 minutes)
- **Dedup accuracy:** Zero duplicate pages in Notion across multiple sync runs
- **Error rate:** Number of ERROR log lines per sync run (target: 0 per run under normal conditions)
- **State integrity:** `state.json` accurately reflects all pages in the Instagram Saves DB

### Business Metrics (for the creator)
- **Weekly Active Use:** Creator runs Ideate at least once per week
- **Content Ideas in pipeline:** Number of Not Started ideas in the Content Ideas DB at any time (target: always > 10)
- **Time to brief:** Time from Instagram save to approved content brief (target: same day)

---

## 18. Roadmap

### Phase 1 — Stabilization (Current)
- [x] Core sync pipeline implemented
- [x] Collection filter bug fixed
- [x] State written per-post for crash safety
- [x] Slash command with 6 actions
- [x] launchd scheduler
- [ ] Add retry logic for Notion 429 (rate limit) responses
- [ ] Add Notion-level dedup check by Media ID (prevents duplicates if state.json is lost)
- [ ] Add log rotation for `sync.log`
- [ ] Alert mechanism when Instagram session expires (e.g., log a warning prominently, optional OS notification)

### Phase 2 — Improvement
- [ ] Automated session expiry detection with actionable alert
- [ ] Config validation at startup with clear per-field error messages
- [ ] Multi-collection priority (if a save is in both "Claude Code" and "Content Ideas", let the creator define which wins)
- [ ] Richer Notion views (pre-built Notion databases with views already configured)
- [ ] Ideation context: pass the creator's previously generated ideas to Claude so it avoids generating duplicates
- [ ] Personal voice profiling: maintain a voice guide that Claude applies when writing hooks and outlines

### Phase 3 — Scaling
- [ ] Multi-platform ingestion: TikTok bookmarks, YouTube Watch Later, Twitter/X bookmarks — all feeding the same Saves inbox
- [ ] Full draft generation: after approving an idea, one command generates the complete script or caption
- [ ] Social publishing integration: push approved ideas to Buffer or Later for scheduling
- [ ] Performance feedback loop: pull published post metrics back into Notion to surface which angles perform best
- [ ] Automated ideation mode (no approval step) — for creators who trust the output quality
- [ ] Cross-platform (Linux support, removal of launchd dependency in favor of cron or a scheduler service)

---

## 19. Open Questions

1. **Session refresh automation:** Is there a way to refresh Instagram session cookies programmatically (e.g., via a headless browser) to eliminate the manual ~60-day maintenance cycle?

2. **Official API path:** Does Meta's Graph API have any endpoint — even a restricted or partner one — that exposes saved posts? This would eliminate the unofficial API risk entirely.

3. **Notion SDK version:** What version of `notion-client` first introduced `databases.query()`? Is there a stable version worth upgrading to?

4. **State backup:** Should `state.json` be backed up to Notion itself (as a config page) to survive local disk failures?

5. **Batch size:** Is 10 ideas per Ideate batch the right size? Should the creator be able to configure this?

6. **Pillar system portability:** The current slash command hardcodes the creator's specific content pillars and niche. How should this be configurable for other creators adapting the system?

7. **Content quality floor:** How should the system handle saves with very little caption text (e.g., a post with only hashtags)? Currently these generate weak idea titles.

8. **Week Of assignment:** The current system assigns ideas to the current week's Monday. Should this be configurable (e.g., assign to next week if it's already Thursday)?

9. **Platform for Linux/Windows users:** launchd is macOS-only. What is the recommended equivalent for Linux (systemd timer) and Windows (Task Scheduler) users?

10. **Notion integration permissions:** What is the minimum required scope for the Notion internal integration? Currently it has full workspace access — this should be scoped to the two specific databases only.