Session Checkpointing

Save and resume conversations with checkpoint and recovery

Session Checkpointing and Recovery

Qwen Code supports session checkpointing, allowing you to save conversation state and resume sessions later. This is essential for long-running tasks, recovery from errors, and maintaining context across multiple work sessions.

Overview

Checkpointing provides:

• Session Persistence: Save conversation history and state
• Resume Capability: Continue sessions after interruption
• Chat Compression: Compact long conversations while preserving context
• Token Optimization: Reduce token usage in resumed sessions
• UI Telemetry: Restore metrics and statistics on resume

Session Storage

Session Directory

Sessions are stored in:

~/.qwen/sessions/<session-id>/

Each session directory contains:

~/.qwen/sessions/<uuid>/
├── conversation.jsonl      # Line-delimited JSON conversation log
├── checkpoint-<tag>.json   # Named checkpoints
└── metadata.json          # Session metadata

Conversation Format

From packages/core/src/core/logger.ts, conversations are stored as JSONL:

{"role":"user","parts":[{"text":"Hello"}]}
{"role":"model","parts":[{"text":"Hi there!"}]}
{"role":"user","parts":[{"text":"Create a function"}]}

Checkpoint Operations

Creating Checkpoints

From packages/core/src/core/logger.ts:328:

async saveCheckpoint(conversation: Content[], tag: string): Promise<void> {
  if (!this.initialized) {
    this.debugLogger.error(
      'Logger not initialized or checkpoint file path not set. Cannot save a checkpoint.',
    );
    return;
  }
  // Always save with the new encoded path.
  const path = this._checkpointPath(tag);
  try {
    await fs.writeFile(path, JSON.stringify(conversation, null, 2), 'utf-8');
  } catch (error) {
    this.debugLogger.error('Error writing to checkpoint file:', error);
  }
}

Checkpoint Path Encoding (from logger.ts:286):

private _checkpointPath(tag: string): string {
  if (!tag.length) {
    throw new Error('No checkpoint tag specified.');
  }
  if (!this.qwenDir) {
    throw new Error('Checkpoint file path not set.');
  }
  // Encode the tag to handle all special characters safely.
  const encodedTag = encodeTagName(tag);
  return path.join(this.qwenDir, `checkpoint-${encodedTag}.json`);
}

Tags are URL-encoded to handle special characters:

# Tag: "pre-refactor"
~/.qwen/sessions/<uuid>/checkpoint-pre-refactor.json

# Tag: "before:api/changes"
~/.qwen/sessions/<uuid>/checkpoint-before%3Aapi%2Fchanges.json

Loading Checkpoints

From logger.ts:344:

async loadCheckpoint(tag: string): Promise<Content[]> {
  if (!this.initialized) {
    this.debugLogger.error(
      'Logger not initialized or checkpoint file path not set. Cannot load checkpoint.',
    );
    return [];
  }

  const path = await this._getCheckpointPath(tag);
  try {
    const fileContent = await fs.readFile(path, 'utf-8');
    const parsedContent = JSON.parse(fileContent);
    if (!Array.isArray(parsedContent)) {
      this.debugLogger.warn(
        `Checkpoint file at ${path} is not a valid JSON array. Returning empty checkpoint.`,
      );
      return [];
    }
    return parsedContent as Content[];
  } catch (error) {
    const nodeError = error as NodeJS.ErrnoException;
    if (nodeError.code === 'ENOENT') {
      // This is okay, it just means the checkpoint doesn't exist in either format.
      return [];
    }
    this.debugLogger.error(
      `Failed to read or parse checkpoint file ${path}:`,
      error,
    );
    return [];
  }
}

Deleting Checkpoints

From logger.ts:377:

async deleteCheckpoint(tag: string): Promise<boolean> {
  if (!this.initialized || !this.qwenDir) {
    this.debugLogger.error(
      'Logger not initialized or checkpoint file path not set. Cannot delete checkpoint.',
    );
    return false;
  }

  let deletedSomething = false;

  // 1. Attempt to delete the new encoded path.
  const newPath = this._checkpointPath(tag);
  try {
    await fs.unlink(newPath);
    deletedSomething = true;
  } catch (error) {
    const nodeError = error as NodeJS.ErrnoException;
    if (nodeError.code !== 'ENOENT') {
      this.debugLogger.error(
        `Failed to delete checkpoint file ${newPath}:`,
        error,
      );
      throw error; // Rethrow unexpected errors
    }
    // It's okay if it doesn't exist.
  }

  // 2. Attempt to delete the old raw path for backward compatibility.
  const oldPath = path.join(this.qwenDir!, `checkpoint-${tag}.json`);
  if (newPath !== oldPath) {
    try {
      await fs.unlink(oldPath);
      deletedSomething = true;
    } catch (error) {
      const nodeError = error as NodeJS.ErrnoException;
      if (nodeError.code !== 'ENOENT') {
        this.debugLogger.error(
          `Failed to delete checkpoint file ${oldPath}:`,
          error,
        );
        throw error; // Rethrow unexpected errors
      }
      // It's okay if it doesn't exist.
    }
  }

  return deletedSomething;
}

Checking Checkpoint Existence

From logger.ts:426:

async checkpointExists(tag: string): Promise<boolean> {
  if (!this.initialized) {
    throw new Error(
      'Logger not initialized. Cannot check for checkpoint existence.',
    );
  }
  
  const path = await this._getCheckpointPath(tag);
  try {
    await fs.access(path);
    return true;
  } catch {
    return false;
  }
}

Session Resume

Command Line Usage

# Start new session
qwen "Create a web server"
# Session ID: abc-123-def

# Resume later
qwen --resume abc-123-def

# Resume with new input
qwen --resume abc-123-def "Add authentication"

Resume Process

From packages/core/src/services/sessionService.ts:581:

/**
 * Builds the model-facing chat history (Content[]) from a reconstructed
 * conversation. This keeps UI history intact while applying chat compression
 * checkpoints for the API history used on resume.
 *
 * Strategy:
 * - Find the latest system/chat_compression record (if any).
 * - Use its compressedHistory snapshot as the base history.
 * - Append all messages after that checkpoint (skipping system records).
 * - If no checkpoint exists, return the linear message list (message field only).
 */
export function buildApiHistoryFromConversation(
  conversation: ConversationRecord,
  options: BuildApiHistoryOptions = {},
): Content[] {
  const { stripThoughtsFromHistory = true } = options;
  const { messages } = conversation;

  let lastCompressionIndex = -1;
  let compressedHistory: Content[] | undefined;

  messages.forEach((record, index) => {
    if (record.type === 'system' && record.subtype === 'chat_compression') {
      const payload = record.systemPayload as
        | ChatCompressionRecordPayload
        | undefined;
      if (payload?.compressedHistory) {
        lastCompressionIndex = index;
        compressedHistory = payload.compressedHistory;
      }
    }
  });

  if (compressedHistory && lastCompressionIndex >= 0) {
    const baseHistory: Content[] = structuredClone(compressedHistory);

    // Append everything after the compression record (newer turns)
    for (let i = lastCompressionIndex + 1; i < messages.length; i++) {
      const record = messages[i];
      if (record.type === 'system') continue;
      if (record.message) {
        baseHistory.push(structuredClone(record.message as Content));
      }
    }

    if (stripThoughtsFromHistory) {
      return baseHistory
        .map(stripThoughtsFromContent)
        .filter((content): content is Content => content !== null);
    }
    return baseHistory;
  }

  // Fallback: return linear messages as Content[]
  const result = messages
    .map((record) => record.message)
    .filter((message): message is Content => message !== undefined)
    .map((message) => structuredClone(message));

  if (stripThoughtsFromHistory) {
    return result
      .map(stripThoughtsFromContent)
      .filter((content): content is Content => content !== null);
  }
  return result;
}

Session Data Structure

From packages/core/src/services/chatRecordingService.ts:102:

/**
 * Stored payload for chat compression checkpoints. This allows us to rebuild the
 * effective chat history on resume while keeping the original UI-visible history.
 */
export interface ChatCompressionRecordPayload {
  /**
   * Compressed conversation history snapshot. Used as base when building API
   * resume reconstruction.
   */
  compressedHistory: Content[];
  
  /**
   * Original token count before compression (for tracking metrics).
   */
  oldTokenCount?: number;
  
  /**
   * New token count after compression.
   */
  newTokenCount?: number;
  
  /**
   * Copy of the raw Content[] array sent to the compression model/prompt for
   * the CLI (without IDs). Stored as plain objects for replay on resume.
   */
  rawInputHistory?: Content[];
}

Chat Compression

Compression Triggers

From packages/core/src/config/config.ts, chat compression activates based on:

export interface ChatCompressionSettings {
  contextPercentageThreshold?: number;  // Default: 70%
}

When prompt tokens exceed 70% of model’s context window, compression is triggered.

Compression Process

From packages/core/src/core/prompts.ts:356:

const COMPRESS_SYSTEM_PROMPT = `
You are a chat history compression agent. Your job is to distill a long conversation into a concise XML snapshot.

When the conversation history grows too large, you will be invoked to distill the entire history into a concise, structured XML snapshot. This snapshot is CRITICAL, as it will become the agent's *only* memory of the past. The agent will resume its work based solely on this snapshot. All crucial details, plans, errors, and user directives MUST be preserved.

Guidelines:
- Preserve all important context, decisions, and state
- Remove redundant explanations and verbose responses
- Keep technical details and error messages
- Maintain chronological order of significant events
- Include user preferences and directives

Output format:
<conversation_snapshot>
  <context>
    (Essential background and setup)
  </context>
  <progress>
    (Key accomplishments and current state)
  </progress>
  <pending>
    (Unfinished tasks and next steps)
  </pending>
  <errors>
    (Important errors and their resolutions)
  </errors>
</conversation_snapshot>
`;

Compression Benefits

1. Token Reduction: 50-80% reduction in prompt tokens
2. Cost Savings: Lower API costs for long sessions
3. Performance: Faster response times with smaller context
4. Context Retention: Important information preserved

Compression Example

Before compression (10,000 tokens):

[
  {"role": "user", "parts": [{"text": "Create a web server"}]},
  {"role": "model", "parts": [{"text": "I'll help you create..."}]},
  {"role": "user", "parts": [{"text": "Add routing"}]},
  {"role": "model", "parts": [{"text": "Let me add routing..."}]},
  // ... 50+ more turns ...
  {"role": "user", "parts": [{"text": "Fix the bug"}]},
  {"role": "model", "parts": [{"text": "I'll investigate..."}]}
]

After compression (2,500 tokens):

[
  {
    "role": "user",
    "parts": [{
      "text": "<conversation_snapshot>\n<context>\nCreating Express.js web server with routing, authentication, and database integration.\n</context>\n<progress>\n- Set up Express server on port 3000\n- Implemented JWT authentication\n- Connected to PostgreSQL database\n- Created 5 API endpoints\n</progress>\n<pending>\n- Fix CORS error in production\n- Add rate limiting\n- Write integration tests\n</pending>\n<errors>\n- Database connection timeout (resolved: increased pool size)\n- JWT verification failing (resolved: fixed secret key)\n</errors>\n</conversation_snapshot>"
    }]
  },
  // Recent uncompressed messages continue here
  {"role": "user", "parts": [{"text": "Fix the bug"}]},
  {"role": "model", "parts": [{"text": "I'll investigate..."}]}
]

UI Telemetry Persistence

Recording Telemetry

From packages/core/src/services/chatRecordingService.ts:414:

/**
 * Records a UI telemetry event for replaying metrics on resume.
 */
recordUiTelemetryEvent(uiEvent: UiTelemetryEvent): void {
  const record: MessageRecord = {
    uuid: randomUUID(),
    parentUuid: this.getCurrentParentUuid(),
    timestamp: Date.now(),
    type: 'system',
    subtype: 'ui_telemetry',
    role: 'system',
    systemPayload: {
      uiEvent
    }
  };
  this.appendMessage(record);
}

Replaying Telemetry

From sessionService.ts:648:

/**
 * Replays stored UI telemetry events to rebuild metrics when resuming a session.
 * Also restores the last prompt token count from the best available source.
 */
export function replayUiTelemetryFromConversation(
  conversation: ConversationRecord,
): void {
  uiTelemetryService.reset();

  for (const record of conversation.messages) {
    if (record.type !== 'system' || record.subtype !== 'ui_telemetry') {
      continue;
    }
    const payload = record.systemPayload as
      | UiTelemetryRecordPayload
      | undefined;
    const uiEvent = payload?.uiEvent;
    if (uiEvent) {
      uiTelemetryService.addEvent(uiEvent);
    }
  }

  const resumePromptTokens = getResumePromptTokenCount(conversation);
  if (resumePromptTokens !== undefined) {
    uiTelemetryService.setLastPromptTokenCount(resumePromptTokens);
  }
}

This restores:

• Token counts (prompt, cached, completion)
• Request counts
• Model usage statistics
• Timing information

Git Integration

Checkpointing can integrate with Git for version control.

From packages/core/src/services/gitService.ts:32:

/**
 * The Git repository is used to support checkpointing.
 */
async initializeRepo(): Promise<void> {
  try {
    // Check if Git is available
    if (!await this.isGitInstalled()) {
      throw new Error(
        'Checkpointing is enabled, but Git is not installed. Please install Git or disable checkpointing to continue.',
      );
    }
    
    // Initialize or verify repository
    await this.ensureGitRepo();
  } catch (error) {
    throw new Error(
      `Failed to initialize checkpointing: ${error instanceof Error ? error.message : 'Unknown error'}. Please check that Git is working properly or disable checkpointing.`,
    );
  }
}

From gitService.ts:110:

async createCheckpointSnapshot(message: string): Promise<void> {
  try {
    await this.gitCommit(message);
  } catch (error) {
    throw new Error(
      `Failed to create checkpoint snapshot: ${error instanceof Error ? error.message : 'Unknown error'}. Checkpointing may not be working properly.`,
    );
  }
}

Each checkpoint can create a Git commit:

# Automatic commits during session
git log --oneline
abc123 Checkpoint: All checks passed. This is a stable checkpoint.
def456 Checkpoint: API endpoints implemented
ghi789 Checkpoint: Database schema updated

SDK Integration

TypeScript SDK

From packages/sdk-typescript/src/types/types.ts:37:

/**
 * Equivalent to CLI's --resume flag.
 */
resume?: string;

/**
 * Session ID for the current query.
 * When resume is provided, this should match the resume ID.
 */
sessionId?: string;

Usage:

import { createQuery } from '@qwen-code/sdk-typescript';

// Start new session
const query1 = await createQuery({
  prompt: "Create a function",
  sessionId: "my-session-123"
});

// Resume later
const query2 = await createQuery({
  prompt: "Add error handling",
  resume: "my-session-123"  // Resumes previous session
});

From packages/sdk-typescript/src/query/createQuery.ts:46:

const sessionId = options.resume ?? options.sessionId ?? randomUUID();

Process Transport

From packages/sdk-typescript/src/transport/ProcessTransport.ts:263:

if (this.options.resume) {
  // Add --resume flag
  args.push('--resume', this.options.resume);
}

The SDK automatically passes --resume to the CLI process.

Best Practices

When to Create Checkpoints

1. Before Major Changes:

   qwen --checkpoint "before-refactor" "Refactor the API"
   

2. After Milestones:

   # After completing a feature
   qwen --checkpoint "feature-complete" "Tests passing"
   

3. Before Risky Operations:

   qwen --checkpoint "before-migration" "Migrate database"
   

Session Naming

Use descriptive session IDs:

# Good: Descriptive
qwen --session-id "api-refactor-2024-03" "Refactor API"
qwen --session-id "bug-fix-auth-issue" "Fix auth bug"

# Bad: Generic
qwen --session-id "session1" "Do stuff"
qwen --session-id "test" "Test things"

Checkpoint Tags

Use clear, meaningful tags:

// Good tags
"pre-refactor"
"all-tests-passing"
"stable-v1.0"
"before:database/migration"

// Bad tags
"checkpoint1"
"temp"
"test"

Troubleshooting

Session Not Found

Problem: Error: Session <id> not found

Solution:

# List available sessions
ls ~/.qwen/sessions/

# Verify session exists
ls ~/.qwen/sessions/<session-id>/

# Check for conversation file
cat ~/.qwen/sessions/<session-id>/conversation.jsonl

Checkpoint Load Failed

Problem: Checkpoint returns empty array

Causes:

1. Checkpoint file doesn’t exist
2. Invalid JSON format
3. File corruption

Solution:

# Check checkpoint file
ls -la ~/.qwen/sessions/<session-id>/checkpoint-*.json

# Validate JSON
jq . ~/.qwen/sessions/<session-id>/checkpoint-<tag>.json

# Restore from backup if corrupted
cp ~/.qwen/sessions/<session-id>/conversation.jsonl.bak \
   ~/.qwen/sessions/<session-id>/conversation.jsonl

Git Initialization Failed

Problem: Checkpointing fails with Git errors

Solution:

# Verify Git is installed
git --version

# Initialize Git in session directory
cd ~/.qwen/sessions/<session-id>/
git init

# Or disable Git integration
qwen config set checkpointing.useGit false

Token Count Mismatch

Problem: Resumed session shows incorrect token counts

Solution: Token counts are restored from compression checkpoints or last usage metadata. If incorrect:

// Telemetry is replayed on resume
// Check for compression checkpoint
const hasCompression = conversation.messages.some(
  r => r.type === 'system' && r.subtype === 'chat_compression'
);

if (!hasCompression) {
  // No compression checkpoint - using fallback
  // Token count from last assistant message
}

Advanced Topics

Custom Checkpoint Storage

import { Logger } from '@qwen-code/qwen-code-core';

const logger = new Logger('custom-path');

// Save to custom location
await logger.saveCheckpoint(conversation, 'milestone-1');

// Load from custom location  
const restored = await logger.loadCheckpoint('milestone-1');

Checkpoint Migration

# Migrate old checkpoint format to new
for file in ~/.qwen/sessions/*/checkpoint-*.json; do
  if [[ -f "$file" ]]; then
    # Validate and re-save with encoding
    jq . "$file" > "$file.tmp" && mv "$file.tmp" "$file"
  fi
done

Programmatic Resume

import { SessionService } from '@qwen-code/qwen-code-core';

const sessionService = new SessionService();

// Load session
const resumedData = await sessionService.loadSession(sessionId);

// Rebuild API history
const apiHistory = buildApiHistoryFromConversation(
  resumedData.conversation,
  { stripThoughtsFromHistory: true }
);

// Replay telemetry
replayUiTelemetryFromConversation(resumedData.conversation);

// Continue session
const response = await contentGenerator.generateContent({
  contents: [...apiHistory, newUserMessage]
});

Source Code References

• Logger (checkpoints): packages/core/src/core/logger.ts:286-430
• Session service: packages/core/src/services/sessionService.ts:581-680
• Chat recording: packages/core/src/services/chatRecordingService.ts:33,102,414
• Git integration: packages/core/src/services/gitService.ts:32,110
• SDK types: packages/sdk-typescript/src/types/types.ts:37-45
• Compression prompts: packages/core/src/core/prompts.ts:356