Decoupling Web Worker Instantiation

I was building a database library that needed to run SQLite in web browsers, and everything worked perfectly in development, but when I tried to build for production, everything fell apart. The web workers wouldn't load, bundlers threw cryptic errors, and I spent days pulling my hair out trying to figure out what was wrong.

If you've ever tried to use Web Workers in a JavaScript project that needs to work across different build tools, you've probably hit the same wall I did. Let me walk you through exactly what went wrong and how I finally solved it.

What are Web Workers?

Before we dive into the problem, let's first understand what web workers are. Web Workers are JavaScript's way of running code in the background, separate from your main UI thread.

┌─────────────────┐    ┌─────────────────┐
│   Main Thread   │    │   Web Worker    │
│                 │    │                 │
│  ┌───────────┐  │    │  ┌───────────┐  │
│  │    UI     │  │    │  │  Heavy    │  │
│  │ Updates   │  │    │  │Processing │  │
│  └───────────┘  │    │  └───────────┘  │
│                 │    │                 │
│ ┌─────────────┐ │◄──►│ ┌─────────────┐ │
│ │   Messages  │ │    │ │   Messages  │ │
│ └─────────────┘ │    │ └─────────────┘ │
└─────────────────┘    └─────────────────┘

Think of it like having a helper who works in a separate room. Your main thread (the UI) can keep running smoothly while the worker does heavy lifting like database operations, image processing, or complex calculations.

The Problem That Made Me Question Everything

Here's what I was trying to do, pretty standard stuff:

// This worked perfectly in development
const worker = new Worker(new URL("./worker.js", import.meta.url), {
  type: "module",
});

The import.meta.url is a fancy way of saying "get the current file's location and find worker.js relative to it." Super clean, very modern JavaScript.

But when I ran npm run build, everything exploded:

Module not found: Can't resolve '/assets/worker-xyz123.js'
import.meta.url resolution failed
Worker loading error: Failed to construct 'Worker'

Why this happens

Let me show you what actually happens during the build process:

                +-------------+
                | Source Code |
                +-------------+
                      |
                      v
             +--------------------+
             | Bundler Analysis   |
             +--------------------+
                      |
                      v
       +-----------------------------------+
       | Can Resolve import.meta.url?      |
       +-----------------------------------+
              /                 \
             /                   \
     Yes - Simple Case       No - Complex Case
           |                        |
           v                        v
   +----------------+        +-------------------+
   | Bundle Worker  |        |    Build Fails    |
   +----------------+        +-------------------+
           |                        |
           v                        v
   +------------------+     +------------------------+
   |  Hash Filename   |     |   Unresolved Import    |
   +------------------+     +------------------------+
           |                        |
           v                        v
   +------------------+     +------------------------+
   | Copy to Assets   |     |  Missing File Error    |
   +------------------+     +------------------------+
           |                        |
           v                        v
   +---------------------+   +------------------+
   | Update References   |   |  Broken Build    |
   +---------------------+   +------------------+
           |
           v
   +--------------------+
   |   Working Build    |
   +--------------------+

The issue is that import.meta.url depends on runtime context, but bundlers need to resolve everything at build time. It's like trying to give someone directions to a place that doesn't exist yet.

Different bundlers, different problems

Each bundler handles this differently, and that's where the chaos begins:

Vite (Development):

• Handles import.meta.url perfectly
• Serves files from filesystem directly
• Everything just works

Vite (Production Build):

• Sometimes fails to resolve worker paths
• Hash generation breaks references
• Assets might not copy correctly

Webpack:

• Requires special configuration
• import.meta.url support is inconsistent
• Often needs custom loaders

Next.js:

• Even more complex due to SSR
• Server-side rendering doesn't understand workers
• Build-time vs runtime path resolution conflicts

My first failed attempts:

Attempt #1: Manual File Copying

// Copy worker.js to public folder manually
const worker = new Worker("/worker.js");

Problems:

• Had to remember to copy files every time
• No automatic updates when worker code changed
• Breaks in subdirectories
• Not scalable for teams

Attempt #2: Webpack Configuration Hell

// webpack.config.js - 50 lines of configuration
module.exports = {
  module: {
    rules: [
      {
        test: /\.worker\.js$/,
        use: { loader: "worker-loader" },
      },
    ],
  },
};

Problems:

• Only worked with Webpack
• Broke with other bundlers
• Configuration was complex and fragile
• Different setups for different environments

Attempt #3: Dynamic Imports

// Try to load worker dynamically
const workerModule = await import("./worker.js?worker");
const worker = new Worker(workerModule.default);

Problems:

• Syntax not supported everywhere
• Still had bundler resolution issues
• Different behavior in dev vs production

Learning

After days of frustration, I started looking at how successful database libraries solve this problem. That's when I discovered the pattern that changed everything.

Instead of trying to make bundlers automatically resolve worker paths, successful libraries do something much smarter: they let the user create the Worker instance themselves.

Here's the pattern:

Old Approach (Automatic):
┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Library   │───►│   Creates   │───►│   Worker    │
│             │    │   Worker    │    │  (broken)   │
└─────────────┘    └─────────────┘    └─────────────┘

New Approach (Explicit):
┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│    User     │───►│   Creates   │───►│   Worker    │
│             │    │   Worker    │    │  (works)   │
└─────────────┘    └─────────────┘    └─────────────┘
                              │
                              ▼
                   ┌─────────────┐
                   │   Library   │
                   │  Uses It    │
                   └─────────────┘

The solution

Here's how I restructured everything:

Step 1: Create a Worker Class That Accepts Workers

export class DatabaseWorker {
  constructor(worker, options = {}) {
    this.worker = worker; // User provides this!
    this.options = options;
    this.setupCommunication();
  }

  setupCommunication() {
    this.worker.addEventListener("message", (event) => {
      this.handleMessage(event.data);
    });
  }

  async query(sql, params) {
    return new Promise((resolve, reject) => {
      const id = Math.random().toString(36);
      this.pending.set(id, { resolve, reject });

      this.worker.postMessage({
        type: "query",
        id,
        sql,
        params,
      });
    });
  }
}

Step 2: Create a Standalone Worker Entry Point

// worker-entry.js - This is what gets bundled
import { setupWorker } from "./worker-communication.js";

// Initialize the worker when loaded
setupWorker();

Step 3: Export Pattern for Users

// main.js - What users import
export { DatabaseWorker } from "./database-worker.js";

// Users create workers like this:
const worker = new Worker(
  new URL("your-library/worker-entry", import.meta.url),
  { type: "module" }
);

const db = new DatabaseWorker(worker, {
  databasePath: "myapp.db",
});

Why this approach works everywhere

The magic is in the separation of concerns:

   +------------+          +----------------+          +----------------------+          +------------------------+
   | User Code  | --------> | Creates Worker | -------> | Worker Entry Point   | -------> | Library Worker Logic   |
   +------------+          +----------------+          +----------------------+          +------------------------+
         |
         |
         |
         v
 +-------------------------+          +------------------------+          +----------------------+
 | Creates Library Instance | -------> | Passes Worker to Lib  | -------> | Library Uses Worker |
 +-------------------------+          +------------------------+          +----------------------+

• User controls worker creation - They handle the import.meta.url in their environment
• Library provides worker logic - We just export the worker code
• Bundlers see explicit imports - No dynamic resolution needed
• Works with any bundler - User's bundler handles their worker creation
• No configuration needed - Just standard ES modules

Real-World Implementation

Here's how users actually use this pattern:

// In any bundler environment
import { DatabaseWorker } from "my-database-lib";

// User creates worker (their bundler handles this)
const worker = new Worker(
  new URL("my-database-lib/worker-entry", import.meta.url),
  { type: "module" }
);

// Pass worker to library
const db = new DatabaseWorker(worker, {
  databasePath: "app.db",
});

// Use it normally
await db.query("SELECT * FROM users WHERE id = ?", [1]);

The Build Configuration

Here's how I set up the build to support this pattern:

// vite.config.js
export default defineConfig({
  build: {
    lib: {
      entry: {
        index: "src/index.ts",
        "worker-entry": "src/worker-entry.ts",
      },
      formats: ["es"],
    },
    rollupOptions: {
      external: ["@sqlite.org/sqlite-wasm"],
    },
  },
});

// package.json
{
  "exports": {
    ".": {
      "import": "./dist/index.js"
    },
    "./worker-entry": {
      "import": "./dist/worker-entry.js"
    }
  }
}

Benefits of this approach

• Universal Compatibility
• No Manual File Copying
• Type Safety
• Better Developer Experience

// Clear, explicit API
const worker = new Worker(workerUrl, { type: "module" });
const db = new DatabaseWorker(worker, options);

If we talk about performance, it's a win-win situation.

Before (Broken Builds):
Build Time: Failed
Bundle Size: N/A
Runtime Performance: N/A
Developer Experience: Terrible

After (Explicit Pattern):
Build Time: ~2s
Bundle Size: ~45KB (compressed)
Runtime Performance: Database operations don't block UI
Developer Experience: Just works

Conclusion

The key lesson from this journey is simple: explicit is better than magical when it comes to Web Workers. Instead of fighting bundler limitations, let users create their own workers and focus on making your worker logic robust.

This pattern works everywhere because it respects bundler boundaries and doesn't rely on dynamic path resolution. The result is a library that works with any bundler without special configuration.

Thank you 😊 for taking the time ⏰ to read this blog post 📖. I hope you found the information 📚 helpful and informative 🧠. If you have any questions ❓ or comments 💬, please feel free to leave them below ⬇️. Your feedback 📝 is always appreciated.

Portfolio GitHub LinkedIn X