System Documentation
Data Flow
Data Flow Documentation - CleanTextLab
Last Updated: January 5, 2026 Purpose: Document how data moves through the system Audience: Developers, architects, and AI assistants
Table of Contents
- Client-Side Data Flow
- Authentication Flow
- History Sync Flow
- API Request Flow
- Payment Flow
- Workflow Execution Flow
- Share Link Flow
- State Management
Client-Side Data Flow
Standard Tool Usage (Anonymous User)
┌─────────────────────────────────────────────────────────────┐
│ 1. USER PASTES TEXT INTO TOOL │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ useUndoRedo Hook │
│ setState(input) │
└──────────┬───────────┘
│
├─→ Add to history stack (past, present, future)
│
├─→ Trigger re-render
│
▼
┌──────────────────────┐
│ useMemo(() => │
│ processInput() │
│ ) │
└──────────┬───────────┘
│
├─→ Run tool-specific logic (e.g., JSON.parse())
│
├─→ Return output
│
▼
┌──────────────────────┐
│ Display Output │
│ (Textarea) │
└──────────┬───────────┘
│
┌────────────┼────────────┐
│ │ │
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Copy │ │ Download │ │ Share │
│ Button │ │ Button │ │ Button │
└──────────┘ └──────────┘ └──────────┘
│ │ │
│ │ │
▼ ▼ ▼
Clipboard File Blob URL with
base64 state
Data Persistence (localStorage)
Tool Usage
│
▼
useToolHistory Hook
│
├─→ Create HistoryEntry
│ {
│ id: uuid,
│ timestamp: Date.now(),
│ data: { input, output, metadata }
│ }
│
├─→ Add to history array
│
├─→ Limit to user preference (default 10 entries)
│
└─→ localStorage.setItem(
`clt_history_${toolSlug}`,
JSON.stringify(history)
)
Key: clt_history_${toolSlug} (anonymous) or clt_history_user:${userId}:${toolSlug} (signed-in)
Value: HistoryEntry[] (JSON stringified)
Max Size: configurable; default 10 entries per tool
Authentication Flow
Sign-In Flow (OAuth)
┌─────────────────────────────────────────────────────────────┐
│ 1. USER CLICKS "Sign In with Google" │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ Clerk OAuth Handler │
└──────────┬───────────┘
│
├─→ Redirect to Google OAuth consent screen
│
▼
┌──────────────────────┐
│ User Approves │
└──────────┬───────────┘
│
├─→ Google returns OAuth token
│
▼
┌──────────────────────┐
│ Clerk Creates │
│ User + Session │
└──────────┬───────────┘
│
├─→ Set cookie: __session
│
├─→ Redirect to app
│
▼
┌──────────────────────┐
│ middleware.ts │
│ Checks session │
└──────────┬───────────┘
│
├─→ Session valid → Continue
│
▼
┌──────────────────────┐
│ useUser() hook │
│ Provides user data │
└──────────────────────┘
Session Management
Cookie: __session (HTTP-only, secure, SameSite=Lax)
Expiration: 7 days (configurable)
Refresh: Auto-refresh via Clerk client
Session Data Structure:
{
userId: "user_abc123",
sessionId: "sess_xyz789",
publicMetadata: {
stripePlan: "pro" | null,
apiKey: "hashed_key" | null
}
}
History Sync Flow
Cloud Sync (Pro Users Only)
┌─────────────────────────────────────────────────────────────┐
│ 1. PRO USER COMPLETES TOOL USAGE │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ useToolHistory │
│ saveToHistory() │
└──────────┬───────────┘
│
┌────────────┴─────────────┐
│ │
▼ ▼
┌────────────────┐ ┌──────────────────┐
│ localStorage │ │ Check if Pro │
│ (immediate) │ │ getUserEntitle │
└────────────────┘ │ ments() │
└──────┬───────────┘
│
├─→ isPro === false → Skip cloud sync
│
├─→ isPro === true → Continue
│
▼
┌──────────────────────┐
│ Encrypt History │
│ (AES-256-GCM) │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ POST /api/user/ │
│ history │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Clerk Auth Check │
│ auth().userId │
└──────────┬───────────┘
│
├─→ No userId → Return 401
│
▼
┌──────────────────────┐
│ Upstash Redis │
│ SET user:{userId}: │
│ history:{toolSlug} │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Return 200 OK │
└──────────────────────┘
Encryption Details
Algorithm: AES-256-GCM Key Derivation: PBKDF2 from user's userId + secret salt IV: Random 12 bytes (stored with ciphertext)
Encryption Code:
import crypto from "crypto";
function encryptHistory(data: any, userId: string): string {
const key = crypto.pbkdf2Sync(
userId,
process.env.ENCRYPTION_SALT!,
100000,
32,
"sha256"
);
const iv = crypto.randomBytes(12);
const cipher = crypto.createCipheriv("aes-256-gcm", key, iv);
const plaintext = JSON.stringify(data);
let ciphertext = cipher.update(plaintext, "utf8", "hex");
ciphertext += cipher.final("hex");
const authTag = cipher.getAuthTag();
// Format: iv:authTag:ciphertext
return `${iv.toString("hex")}:${authTag.toString("hex")}:${ciphertext}`;
}
Storage in Redis:
Key: user:user_abc123:history:json-formatter
Value: "f3a2b1c4d5e6:a1b2c3d4e5f6:e9f8d7c6b5a4..."
TTL: 90 days
API Request Flow
Public API Call (/api/v1/run)
┌─────────────────────────────────────────────────────────────┐
│ 1. EXTERNAL CLIENT SENDS API REQUEST │
│ │
│ POST /api/v1/run │
│ x-api-key: sk_live_abc123 │
│ { │
│ "input": "messy text", │
│ "steps": ["trim-lines", "upper-case"] │
│ } │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ Vercel Edge │
│ Function │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ API Key + Rate │
│ Limit Check │
└──────────┬───────────┘
│
├─→ Missing key → Return 401
│
├─→ Invalid key → Return 401
│
└─→ Valid key → INCR rl:api:{key}
└─→ > plan limit/day → Return 429
│
▼
┌──────────────────────┐
│ Input Validation │
└──────────┬───────────┘
│
├─→ Missing input → Return 400
├─→ Missing steps → Return 400
├─→ Invalid step → Return 400
│
▼
┌──────────────────────┐
│ Workflow Engine │
└──────────┬───────────┘
│
├─→ For each step:
│ 1. Lookup executor in map
│ 2. Run: output = executor(input)
│ 3. Set input = output for next step
│
▼
┌──────────────────────┐
│ Build Response │
│ { │
│ "result": "...", │
│ "steps": [...], │
│ "duration": 42 │
│ } │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Return JSON │
│ (200 OK) │
└──────────────────────┘
Rate Limiting Logic
Authenticated API (With API Key):
- All
/api/v1/*endpoints: Pro only, 5,000/day - Redis Key:
rl:api:{key}
Behavior:
- Missing or invalid keys return 401.
- Limits are enforced per key, per day.
MCP (Model Context Protocol) Flow
Agent Call (Claude / MCP Client)
┌─────────────────────────────────────────────────────────────┐
│ 1. AGENT CALLS MCP TOOL (JSON-RPC) │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ MCP CLI (stdio) │
│ cleantextlab-mcp │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ POST /api/v1/run │
│ steps: [toolSlug] │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Standard API Flow │
│ (auth + rate limit) │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ MCP Response │
│ content: [{ text }] │
└──────────────────────┘
Payment Flow
Subscription Purchase Flow
┌─────────────────────────────────────────────────────────────┐
│ 1. USER CLICKS "Upgrade to Pro" │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ POST /api/stripe/ │
│ checkout │
└──────────┬───────────┘
│
├─→ Clerk auth check
│ └─→ No session → Return 401
│
▼
┌──────────────────────┐
│ Stripe Checkout │
│ Session Create │
└──────────┬───────────┘
│
├─→ stripe.checkout.sessions.create({
│ customer_email: user.email,
│ line_items: [{ price: "price_pro_monthly" }],
│ mode: "subscription",
│ success_url: "/dashboard?upgraded=true",
│ cancel_url: "/pricing",
│ metadata: { userId: user.id }
│ })
│
▼
┌──────────────────────┐
│ Redirect to Stripe │
│ Hosted Checkout │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ User Enters Payment │
│ Details │
└──────────┬───────────┘
│
┌────────────┴─────────────┐
│ │
▼ ▼
┌────────────────┐ ┌──────────────────┐
│ Payment │ │ User Cancels │
│ Succeeds │ │ (Return 303) │
└────────┬───────┘ └──────────────────┘
│
▼
┌──────────────────────┐
│ Stripe Webhook │
│ customer.subscription│
│ .created │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ POST /api/stripe/ │
│ webhook │
└──────────┬───────────┘
│
├─→ Verify webhook signature
│
├─→ Parse event.data.object
│
▼
┌──────────────────────┐
│ Update Clerk User │
│ Metadata │
└──────────┬───────────┘
│
├─→ clerkClient.users.updateUserMetadata(userId, {
│ publicMetadata: {
│ stripePlan: "pro",
│ stripeCustomerId: customer.id,
│ subscriptionId: subscription.id
│ }
│ })
│
▼
┌──────────────────────┐
│ Generate API Key │
│ (if first Pro sub) │
└──────────┬───────────┘
│
├─→ apiKey = `sk_live_${randomBytes(32).toString("hex")}`
│
├─→ Store in Upstash:
│ SET api-key:{apiKey} → userId
│
▼
┌──────────────────────┐
│ Redirect User to │
│ /dashboard │
│ (Success Page) │
└──────────────────────┘
Webhook Event Handling
Supported Events:
customer.subscription.created→ Grant Pro accesscustomer.subscription.updated→ Update billing periodcustomer.subscription.deleted→ Revoke Pro accessinvoice.payment_succeeded→ Extend subscriptioninvoice.payment_failed→ Send dunning email
Webhook Handler:
// src/app/api/stripe/webhook/route.ts
export async function POST(req: Request) {
const body = await req.text();
const signature = req.headers.get("stripe-signature")!;
let event: Stripe.Event;
try {
event = stripe.webhooks.constructEvent(
body,
signature,
process.env.STRIPE_WEBHOOK_SECRET!
);
} catch (err) {
return new Response("Webhook signature verification failed", { status: 400 });
}
switch (event.type) {
case "customer.subscription.created":
await handleSubscriptionCreated(event.data.object);
break;
case "customer.subscription.deleted":
await handleSubscriptionDeleted(event.data.object);
break;
// ... other events
}
return new Response("OK", { status: 200 });
}
Workflow Execution Flow
Multi-Step Tool Chaining
┌─────────────────────────────────────────────────────────────┐
│ 1. USER DEFINES WORKFLOW │
│ │
│ Steps: │
│ 1. Trim lines │
│ 2. Remove duplicates │
│ 3. Sort (asc) │
│ 4. Title case │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ Workflow Engine │
│ executeWorkflow() │
└──────────┬───────────┘
│
├─→ Initialize: currentOutput = input
│
▼
┌──────────────────────┐
│ Step 1: Trim Lines │
└──────────┬───────────┘
│
├─→ Import: import("@/lib/tools/trim-lines")
│
├─→ Execute: output = trimLines(currentOutput)
│
├─→ Track: steps.push({ tool: "trim-lines", duration: 3ms })
│
├─→ Update: currentOutput = output
│
▼
┌──────────────────────┐
│ Step 2: Dedupe │
└──────────┬───────────┘
│
├─→ Import: import("@/lib/tools/dedupe")
│
├─→ Execute: output = dedupe(currentOutput, { ignoreCase: false })
│
├─→ Track: steps.push({ tool: "dedupe", duration: 5ms })
│
├─→ Update: currentOutput = output
│
▼
┌──────────────────────┐
│ Step 3: Sort │
└──────────┬───────────┘
│
├─→ Import: import("@/lib/tools/sort")
│
├─→ Execute: output = sort(currentOutput, { direction: "asc" })
│
├─→ Track: steps.push({ tool: "sort", duration: 2ms })
│
├─→ Update: currentOutput = output
│
▼
┌──────────────────────┐
│ Step 4: Title Case │
└──────────┬───────────┘
│
├─→ Import: import("@/lib/tools/title-case")
│
├─→ Execute: output = titleCase(currentOutput, { style: "simple" })
│
├─→ Track: steps.push({ tool: "title-case", duration: 4ms })
│
├─→ Update: currentOutput = output
│
▼
┌──────────────────────┐
│ Return Result │
│ { │
│ output: "...", │
│ steps: [4], │
│ duration: 14ms │
│ } │
└──────────────────────┘
Error Handling in Workflows
Step 2: JSON Format
│
├─→ Try: JSON.parse(input)
│
└─→ Catch: SyntaxError
│
├─→ Return partial result: {
│ success: false,
│ output: currentOutput (from previous step),
│ steps: [
│ { tool: "trim-lines", status: "success" },
│ { tool: "json-format", status: "error", error: "Invalid JSON" }
│ ],
│ error: "Workflow stopped at step 2: Invalid JSON"
│ }
│
└─→ Stop execution (don't run remaining steps)
Share Link Flow
Generate Shareable Link
┌─────────────────────────────────────────────────────────────┐
│ 1. USER CLICKS "Share" BUTTON │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ useShareableTool │
│ handleShare() │
└──────────┬───────────┘
│
▼
┌──────────────────────┐
│ Compress State │
│ (base64 encode) │
└──────────┬───────────┘
│
├─→ const payload = { input, output, options }
│
├─→ const json = JSON.stringify(payload)
│
├─→ const encoded = btoa(json)
│
▼
┌──────────────────────┐
│ Build Share URL │
└──────────┬───────────┘
│
├─→ const url = `${origin}/tools/${slug}?share=${encoded}`
│
▼
┌──────────────────────┐
│ Copy to Clipboard │
└──────────┬───────────┘
│
├─→ navigator.clipboard.writeText(url)
│
▼
┌──────────────────────┐
│ Show Toast: │
│ "Link copied!" │
└──────────────────────┘
Load Shared State
┌─────────────────────────────────────────────────────────────┐
│ 1. USER CLICKS SHARE LINK │
│ /tools/json-formatter?share=eyJpbnB1dCI6... │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌──────────────────────┐
│ Tool Component │
│ useEffect() │
└──────────┬───────────┘
│
├─→ const params = new URLSearchParams(window.location.search)
│
├─→ const shareData = params.get("share")
│
▼
┌──────────────────────┐
│ Decode State │
└──────────┬───────────┘
│
├─→ const decoded = atob(shareData)
│
├─→ const payload = JSON.parse(decoded)
│
▼
┌──────────────────────┐
│ Restore State │
└──────────┬───────────┘
│
├─→ setInput(payload.input)
│
├─→ setOptions(payload.options)
│
▼
┌──────────────────────┐
│ Tool Auto-Processes │
│ (useMemo triggers) │
└──────────────────────┘
State Management
Client-Side State Hierarchy
┌─────────────────────────────────────────────────────────────┐
│ GLOBAL STATE │
│ │
│ 1. Clerk Auth Context │
│ - user: User | null │
│ - isSignedIn: boolean │
│ │
│ 2. Theme Context │
│ - theme: string │
│ - setTheme: (theme) => void │
│ │
│ 3. Preferences Context │
│ - compactMode: boolean │
│ - keyboardShortcuts: boolean │
└────────────────────┬────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ TOOL-LEVEL STATE │
│ │
│ 1. Input State (useUndoRedo) │
│ - state: string │
│ - setState: (newState) => void │
│ - undo: () => void │
│ - redo: () => void │
│ - canUndo: boolean │
│ - canRedo: boolean │
│ │
│ 2. Options State (useState) │
│ - Tool-specific configuration │
│ │
│ 3. UI State (useState) │
│ - copied: boolean │
│ - toastMessage: string │
│ - shareStatus: "idle" | "copying" | "copied" │
│ │
│ 4. Computed State (useMemo) │
│ - output: string (derived from input + options) │
│ - stats: object (word count, line count, etc.) │
└─────────────────────────────────────────────────────────────┘
State Persistence Layers
Layer 1: Memory (React State)
- Lifetime: Current session only
- Storage: Component state
- Speed: Instant
- Use case: UI interactions, temporary data
Layer 2: localStorage (Browser)
- Lifetime: Until cleared by user
- Storage: Browser localStorage (~10MB limit)
- Speed: < 1ms read/write
- Use case: History, preferences, cached data
- Keys:
user_preferences→ UserPreferencesclt-history-{toolSlug}→ HistoryEntry[]clt-theme→ string
Layer 3: Upstash Redis (Cloud)
- Lifetime: 90 days (TTL)
- Storage: Upstash Redis (unlimited, encrypted)
- Speed: ~50ms read/write
- Use case: Pro users, cross-device sync
- Keys:
user:{userId}:history:{toolSlug}→ Encrypted HistoryEntry[]user:{userId}:workflows→ WorkflowPreset[]api-key:{key}→ userId
Summary
Key Data Flows
| Flow | Type | Dependencies | Fallback |
|---|---|---|---|
| Tool Usage (Anonymous) | Client-only | None | N/A |
| Tool Usage (Pro) | Client + Cloud | Clerk, Upstash | localStorage |
| Authentication | OAuth | Clerk | Deny access |
| History Sync | API | Clerk, Upstash | localStorage |
| API Request | Edge Function | Upstash (rate limit) | Public limit |
| Payment | Webhook | Stripe, Clerk | Manual support |
| Workflow | Client-side | Tool processors | Partial results |
| Share Link | URL-based | None | N/A |
Performance Metrics
| Operation | Target Latency | Actual Latency | Notes |
|---|---|---|---|
| Tool processing (client) | < 100ms | ~20ms | Runs in useMemo |
| History save (localStorage) | < 10ms | ~2ms | Synchronous |
| History save (cloud) | < 200ms | ~80ms | Background |
| API request | < 500ms | ~250ms | Edge function + Redis |
| Authentication | < 1s | ~600ms | OAuth redirect |
| Payment processing | < 3s | ~2s | Stripe hosted |
Document Owner: Tyson K. Last Updated: January 5, 2026 Next Review: March 1, 2026