Patchouli

Patchouli is an imageboard dumper. It creates a local image store and local archive of posts and threads as they appear on an imageboard.

Currently supports:

• 4chan (original functionality)
• JSChan-based imageboards (zzzchan, etc.)

This is a conversion of the original Java Asagi project to Go, providing the same functionality with improved performance and memory efficiency.

AI has been enslaved to comment the code because i am too lazy to do it.

Features

• Multi-board support: Archive multiple 4chan boards simultaneously
• Database backends: MySQL and PostgreSQL support
• JSON configuration: Easy configuration management
• Concurrent processing: Thread-based processing for efficiency
• Graceful shutdown: Proper cleanup on termination
• Extensible architecture: Plugin-based design for different engines

Configuration

The application reads its configuration from patchouli.json:

{
  "settings": {
    "dumperEngine": "DumperJSON",
    "sourceEngine": "YotsubaJSON",
    "boardSettings": {
      "default": {
        "engine": "Pgsql",
        "database": "patchouli_archive",
        "host": "localhost",
        "username": "patchy",
        "password": "patchouli_password",
        "path": "/var/lib/patchouli/archive/",
        "thumbThreads": 5,
        "mediaThreads": 0,
        "refreshDelay": 15
      },
      "4chan_boards": {
        "type": "4chan",
        "boards": ["g", "jp"],
        "refreshDelay": 30,
        "skipHashVerification": false
      },
      "jschan_example": {
        "type": "jschan",
        "apiUrl": "https://fatchan.org",
        "boards": ["b"],
        "refreshDelay": 60,
        "skipHashVerification": true
      }
    }
  }
}

Configuration Options

• type: Imageboard type ("4chan" or "jschan")
• apiUrl: Base API URL for JSChan instances (e.g., "https://zzzchan.xyz")
• boards: Array of board URIs to archive
• engine: Database engine ("Pgsql" or "Mysql")
• database/host/username/password: Database connection settings
• path: Local archive storage path
• refreshDelay: Seconds between dump cycles
• skipHashVerification: Skip MD5 hash verification for downloaded media (default: false)

JSChan Sites

JSChan is a another imageboard software.

• zzzchan.xyz - Random/discussion (cool place)

Note: Many JSChan sites use 4chan-compatible APIs, so you may need to set "type": "4chan" even for JSChan sites.

To find active JSChan sites:

1. Google.
2. Test API endpoints: {site}/boards.json and {site}/{board}/catalog.json
3. Look for sites that return JSON, not HTML maintenance pages
4. Some sites may require specific board configurations

API Compatibility

The application supports two API formats:

• 4chan format: Standard JSON with threads array
• JSChan format: Extended JSON with additional metadata fields

If a site returns HTML instead of JSON, it's likely down for maintenance.

Testing JSChan Sites

To test if a JSChan site is working:

# Test if site has boards API
curl -s "https://example.com/boards.json" | head -c 100

# Test if board catalog works
curl -s "https://example.com/board/catalog.json" | head -c 100

# Should return JSON, not HTML

Troubleshooting JSChan

If you get HTML responses instead of JSON:

1. Site is down: Check if the site loads in your browser
2. Maintenance mode: Many JSChan sites frequently show maintenance pages
3. Wrong API format: Some sites use 4chan-compatible APIs - try "type": "4chan"
4. Board doesn't exist: Check available boards at {apiUrl}/boards.json
5. Rate limiting: JSChan sites may block rapid requests

• thumbThreads/mediaThreads: Concurrent download threads

Building and Running

Prerequisites

• Go 1.24.1 or later
• PostgreSQL database (recommended) or MySQL

Build

go mod tidy
go build -o patchouli

Run

./patchouli

Configuration

You can specify a custom config file:

./patchouli --config /path/to/custom/patchouli.json

Or read from stdin:

cat config.json | ./patchouli --config -

Database Setup

PostgreSQL (Recommended)

Create a database and user:

CREATE DATABASE patchouli_archive;
CREATE USER patchy WITH PASSWORD 'patchouli_password';
GRANT ALL PRIVILEGES ON DATABASE patchouli_archive TO patchy;

MySQL

Create a database and user:

CREATE DATABASE archive CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'patchy'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON archive.* TO 'patchouli'@'localhost';

What's with the name?

Patchouli is the librarian at the Scarlet Mansion in Touhou.