> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/mercuryworkshop/scramjet/llms.txt
> Use this file to discover all available pages before exploring further.

# URL rewriting

> How Scramjet encodes, decodes, and rewrites URLs throughout the proxy

URL rewriting is fundamental to how Scramjet works. Every URL that passes through the proxy must be encoded to route through the Service Worker, and every URL in proxied content must be rewritten to maintain the proxy.

## URL encoding and decoding

### The codec system

Scramjet uses a configurable **codec** to encode and decode URLs. The codec is a pair of functions defined in your configuration:

```typescript theme={null}
const scramjet = new ScramjetController({
  prefix: "/scramjet/",
  codec: {
    encode: (url: string) => encodeURIComponent(url),
    decode: (url: string) => decodeURIComponent(url),
  },
});
```

<Info>
  The default codec uses `encodeURIComponent` and `decodeURIComponent`, but you can use any encoding scheme (base64, custom obfuscation, etc.).
</Info>

### How encoding works

When you navigate to a URL through Scramjet:

```typescript theme={null}
frame.go("https://example.com/page?query=value#hash");
```

The controller's `encodeUrl()` method transforms it:

<Steps>
  <Step title="Parse the URL">
    The URL is parsed into a URL object.
  </Step>

  <Step title="Extract and encode hash">
    The hash fragment is separated and encoded independently:

    ```typescript theme={null}
    const encodedHash = codecEncode(url.hash.slice(1));
    const realHash = encodedHash ? "#" + encodedHash : "";
    url.hash = ""; // Remove from URL before encoding
    ```
  </Step>

  <Step title="Encode the URL">
    The main URL (without hash) is encoded:

    ```typescript theme={null}
    return config.prefix + codecEncode(url.href) + realHash;
    ```
  </Step>

  <Step title="Result">
    ```
    https://example.com/page?query=value#hash
    ↓
    /scramjet/https%3A%2F%2Fexample.com%2Fpage%3Fquery%3Dvalue#hash
    ```

    Note that the hash is preserved separately so browser navigation works correctly.
  </Step>
</Steps>

### How decoding works

When the Service Worker intercepts a request, it decodes the URL:

```typescript theme={null}
const url = new URL(unrewriteUrl(requestUrl));
```

The `unrewriteUrl()` function in `src/shared/rewriters/url.ts` reverses the process:

<Steps>
  <Step title="Check for special protocols">
    Handle special URL types first:

    ```typescript theme={null}
    if (url.startsWith("javascript:")) return url;
    if (url.startsWith("mailto:")) return url;
    if (url.startsWith("about:")) return url;
    ```
  </Step>

  <Step title="Handle blob/data URLs">
    ```typescript theme={null}
    if (url.startsWith(prefixed + "blob:")) {
      return url.substring(prefixed.length);
    }
    if (url.startsWith(prefixed + "data:")) {
      return url.substring(prefixed.length);
    }
    ```
  </Step>

  <Step title="Decode the main URL">
    ```typescript theme={null}
    const realUrl = tryCanParseURL(url);
    const decodedHash = codecDecode(realUrl.hash.slice(1));
    const realHash = decodedHash ? "#" + decodedHash : "";
    realUrl.hash = "";

    return codecDecode(realUrl.href.slice(prefixed.length) + realHash);
    ```
  </Step>
</Steps>

## URL rewriting in content

Once content is fetched, all URLs within it must be rewritten to point through the proxy.

### The rewriteUrl function

The `rewriteUrl()` function is the core rewriter, defined in `src/shared/rewriters/url.ts`:

```typescript theme={null}
export function rewriteUrl(url: string | URL, meta: URLMeta): string
```

It takes:

* **url**: The URL to rewrite (absolute or relative)
* **meta**: Context about the current page (origin, base URL, frame names)

And returns the proxied URL.

### URLMeta context

The `URLMeta` object provides context for rewriting:

```typescript theme={null}
type URLMeta = {
  origin: URL;              // Real origin of the current page
  base: URL;                // Base URL for resolving relative URLs
  topFrameName?: string;    // Top Scramjet frame name
  parentFrameName?: string; // Parent frame name
};
```

<Note>
  The `base` URL can differ from `origin` if the page contains a `<base>` tag. This is updated dynamically during HTML rewriting.
</Note>

### Special URL handling

`rewriteUrl()` handles different URL schemes:

<Accordion title="javascript: URLs">
  JavaScript URLs are rewritten by rewriting the JavaScript code:

  ```typescript theme={null}
  if (url.startsWith("javascript:")) {
    return (
      "javascript:" +
      rewriteJs(url.slice("javascript:".length), "(javascript: url)", meta)
    );
  }
  ```
</Accordion>

<Accordion title="blob: URLs">
  Blob URLs are prefixed with the proxy origin:

  ```typescript theme={null}
  if (url.startsWith("blob:")) {
    return location.origin + config.prefix + url;
  }
  ```

  This routes blob fetches through the Service Worker so they can be rewritten.
</Accordion>

<Accordion title="data: URLs">
  Data URLs are also prefixed:

  ```typescript theme={null}
  if (url.startsWith("data:")) {
    return location.origin + config.prefix + url;
  }
  ```
</Accordion>

<Accordion title="mailto: and about: URLs">
  These are returned unchanged:

  ```typescript theme={null}
  if (url.startsWith("mailto:") || url.startsWith("about:")) {
    return url;
  }
  ```
</Accordion>

<Accordion title="HTTP(S) URLs">
  Regular URLs are resolved against the base, then encoded:

  ```typescript theme={null}
  let base = meta.base.href;
  if (base.startsWith("about:")) {
    base = unrewriteUrl(self.location.href);
  }

  const realUrl = tryCanParseURL(url, base);
  if (!realUrl) return url; // Invalid URL, return as-is

  const encodedHash = codecEncode(realUrl.hash.slice(1));
  const realHash = encodedHash ? "#" + encodedHash : "";
  realUrl.hash = "";

  return (
    location.origin + config.prefix + codecEncode(realUrl.href) + realHash
  );
  ```
</Accordion>

### Relative URL resolution

Relative URLs are resolved against `meta.base`:

```typescript theme={null}
const realUrl = tryCanParseURL(url, base);
```

This uses the browser's native URL parser:

```typescript theme={null}
function tryCanParseURL(url: string, origin?: string | URL): URL | null {
  try {
    return new URL(url, origin);
  } catch {
    return null;
  }
}
```

<Tip>
  If URL parsing fails (e.g., for invalid URLs), the original URL is returned unchanged. This prevents breaking pages with malformed URLs.
</Tip>

## Where URLs are rewritten

Scramjet rewrites URLs in multiple places throughout the stack:

### HTML rewriting

In `src/shared/rewriters/html.ts`, URLs in HTML attributes are rewritten:

```typescript theme={null}
for (const rule of htmlRules) {
  for (const attr in rule) {
    if (node.attribs[attr] !== undefined) {
      const value = node.attribs[attr];
      const v = rule.fn(value, meta, cookieStore);
      if (v === null) delete node.attribs[attr];
      else node.attribs[attr] = v;
    }
  }
}
```

The `htmlRules` array (from `src/shared/htmlRules.ts`) defines which attributes to rewrite:

```typescript theme={null}
{ 
  href: ["a", "link", "area", "base"],
  fn: (value, meta) => rewriteUrl(value, meta)
},
{
  src: ["script", "img", "iframe", "embed", "source", "track", "video", "audio"],
  fn: (value, meta) => rewriteUrl(value, meta)
},
{
  action: ["form"],
  fn: (value, meta) => rewriteUrl(value, meta)
}
// ... etc
```

### Special HTML cases

<CodeGroup>
  ```typescript srcset attribute theme={null}
  export function rewriteSrcset(srcset: string, meta: URLMeta) {
    const sources = srcset.split(/ .*,/).map((src) => src.trim());
    const rewrittenSources = sources.map((source) => {
      const [url, ...descriptors] = source.split(/\s+/);
      const rewrittenUrl = rewriteUrl(url.trim(), meta);
      return descriptors.length > 0
        ? `${rewrittenUrl} ${descriptors.join(" ")}`
        : rewrittenUrl;
    });
    return rewrittenSources.join(", ");
  }
  ```

  ```typescript Event handlers theme={null}
  for (const [attr, value] of Object.entries(node.attribs)) {
    if (eventAttributes.includes(attr)) {
      node.attribs[attr] = rewriteJs(
        value as string,
        `(inline ${attr} on element)`,
        meta
      );
    }
  }
  ```

  ```typescript Import maps theme={null}
  if (node.name === "script" && node.attribs.type === "importmap") {
    const map = JSON.parse(json);
    if (map.imports) {
      for (const key in map.imports) {
        let url = map.imports[key];
        if (typeof url === "string") {
          url = rewriteUrl(url, meta);
          map.imports[key] = url;
        }
      }
    }
    node.children[0].data = JSON.stringify(map);
  }
  ```
</CodeGroup>

### JavaScript rewriting

JavaScript rewriting is more complex. It uses an oxc-based WASM rewriter to:

1. Parse the JavaScript AST
2. Identify API calls that accept URLs
3. Wrap those calls with runtime functions that rewrite URLs

For example, this code:

```javascript theme={null}
fetch("https://example.com/api");
```

Is rewritten to:

```javascript theme={null}
fetch($scramjet$rewrite("https://example.com/api"));
```

The `$scramjet$rewrite` function calls `rewriteUrl()` at runtime with the current page's metadata.

### CSS rewriting

In `src/shared/rewriters/css.ts`, URLs in CSS are rewritten:

```typescript theme={null}
export function rewriteCss(css: string, meta: URLMeta): string {
  return css.replace(/url\(["']?([^"')]+)["']?\)/gi, (match, url) => {
    const rewritten = rewriteUrl(url.trim(), meta);
    return `url("${rewritten}")`;
  });
}
```

This handles:

* `background-image: url(...)`
* `@import url(...)`
* `@font-face { src: url(...) }`
* etc.

### Client-side interception

In the `ScramjetClient`, DOM APIs are intercepted to rewrite URLs at runtime:

```typescript theme={null}
client.Trap("HTMLAnchorElement.prototype.href", {
  get(ctx) {
    const href = ctx.get();
    return unrewriteUrl(href); // Show real URL
  },
  set(ctx, value) {
    const rewritten = rewriteUrl(value, client.meta);
    ctx.set(rewritten); // Set proxied URL
  },
});
```

This ensures that even programmatic URL manipulation is proxied.

## URL preservation

Scramjet preserves original URLs in HTML using `scramjet-attr-*` attributes:

```html theme={null}
<!-- Before rewriting -->
<a href="https://example.com">Link</a>

<!-- After rewriting -->
<a href="/scramjet/https%3A%2F%2Fexample.com" scramjet-attr-href="https://example.com">Link</a>
```

This allows:

* Debugging and inspection
* Restoring original URLs when needed
* Compatibility with scripts that read attributes directly

## Hash handling

Hash fragments require special handling because:

1. They're not sent to the server
2. They're used for client-side routing
3. They need to work with browser navigation APIs

Scramjet encodes hashes separately:

```
https://example.com/page#section
↓
/scramjet/https%3A%2F%2Fexample.com%2Fpage#section
```

The hash is encoded but kept as a real hash fragment so:

* `window.location.hash` works correctly
* Hash-based routers work
* The browser's back/forward buttons work

## Performance considerations

<Warning>
  URL rewriting can be expensive for large documents. The HTML rewriter uses `htmlparser2` for performance, and the JS rewriter is written in Rust/WASM.
</Warning>

<Tip>
  Use the `rewriterLogs` flag during development to see timing information:

  ```typescript theme={null}
  scramjet.modifyConfig({
    flags: { rewriterLogs: true }
  });
  ```
</Tip>

## Common pitfalls

<Warning>
  Always use `ScramjetFrame` for iframes. Direct `<iframe>` elements won't have proper frame tracking, breaking nested iframe URL resolution.
</Warning>

<Warning>
  Be careful with custom URL encoders. They must:

  * Be deterministic (same input = same output)
  * Not produce URLs with characters that need escaping in URLs
  * Be reversible (encode and decode must be inverses)
</Warning>

## Next steps

<CardGroup cols={2}>
  <Card title="Configuration" icon="sliders" href="/concepts/configuration">
    Learn about codec configuration and other options
  </Card>

  <Card title="Service Worker" icon="gear" href="/concepts/service-worker">
    See how URLs are decoded in the Service Worker
  </Card>
</CardGroup>