dlang-http/README.md

# dlang.http

An HTTP/1.1 server plugin for the **dlang** VM. It gives a dlang script a `http` global with
routing, static file serving, custom error pages, a small template engine (`view`) and
file uploads (`multipart/form-data`).

> [!IMPORTANT]
> **This project was generated with AI** (GitHub Copilot) and has not been reviewed by a
> human. Treat the code and this documentation as a starting point: read it, test it, and
> only use it in production once you understand and trust it. See
> [AI-generated code](#ai-generated-code) at the bottom.

> The dlang language and its VM live in their own repository. This repository only contains
> the HTTP plugin; the `dlang/` folder is an unmodified copy of the upstream VM headers and
> sources, kept here so the plugin can be built as a standalone shared library.

---

## Table of contents

- [Features](#features)
- [Requirements](#requirements)
- [Building](#building)
- [Installing the plugin](#installing-the-plugin)
- [Quick start](#quick-start)
- [API reference](#api-reference)
  - [`http.server()`](#httpserver)
  - [`server.listen(port)`](#serverlistenport)
  - [`server.route(path, methods, handler)`](#serverroutepath-methods-handler)
  - [Route handler arguments](#route-handler-arguments)
  - [`server.static(...)`](#serverstatic)
  - [`server.error(code, handler)`](#servererrorcode-handler)
  - [`server.maxBody(bytes)`](#servermaxbodybytes)
  - [File uploads](#file-uploads)
  - [`http.save` / `http.saveTo`](#httpsave--httpsaveto)
  - [`view.Render(path, vars)`](#viewrenderpath-vars)
- [Status codes](#status-codes)
- [Request and response flow](#request-and-response-flow)
- [Limitations](#limitations)
- [Project layout](#project-layout)
- [AI-generated code](#ai-generated-code)

---

## Features

| Feature | Summary |
| --- | --- |
| Routing | Exact-path routes with a per-route list of HTTP methods |
| Static files | URL prefix → folder, exact URL → file, MIME type detected by extension |
| Request parsing | Query string, url-encoded bodies, JSON bodies and `multipart/form-data` |
| File uploads | Uploaded files as tables (`filename`, `contentType`, `size`, raw `content`) plus `http.save` / `http.saveTo` helpers to write them to disk |
| Cookies | Parsed `Cookie:` header as a table |
| Error pages | Custom handlers or bodies per status code, with a built-in HTML fallback |
| Templates | `view.Render("page.html", { ... })` with recursive `@include` and `{{ key }}` interpolation |
| Body limit | Configurable request size limit (`server.maxBody`), oversized uploads get `413` |

---

## Requirements

- **Windows** or **Linux** (the plugin refuses to compile on other platforms).
- A **C++20** compiler: MSVC (Windows) or Clang/GCC (Linux).
- **CMake ≥ 3.10** and **Ninja** for the presets in `CMakePresets.json`.
- A **dlang host** (interpreter/runner) that supports plugins, since this repository builds
  a plugin, not a program.

---

## Building

The presets use Ninja with `cl.exe`, so run the commands from a *Developer Command Prompt*
(or build the folder from VS Code with the CMake Tools extension, which sets up the
compiler environment automatically).

```bash
# configure
cmake --preset x64-debug        # or x64-release, x86-debug, x86-release
                                # linux-debug on Linux, macos-debug on macOS

# build
cmake --build out/build/x64-debug
```

Available presets: `x64-debug`, `x64-release`, `x86-debug`, `x86-release` (Windows),
`linux-debug` (Linux) and `macos-debug` (macOS).

The build output is a shared library:

| Platform | Output |
| --- | --- |
| Windows | `out/build/<preset>/dlang.http/dlang.http.dll` |
| Linux | `out/build/<preset>/dlang.http/dlang.http.so` |

> The shared library compiles `entry.cpp` together with the VM sources in
> `dlang.http/dlang/src/`, so no external dlang SDK is needed. Windows additionally links
> `ws2_32`; Linux links `dl` and `pthread`.

---

## Installing the plugin

Plugins are looked up in a `plugins/` folder relative to the working directory of the dlang
host. The loader appends the platform extension itself (`.dll` on Windows, `.so` on Linux),
so the file must be named `dlang.http.dll` / `dlang.http.so`:

```text
my-app/
├─ plugins/
│  └─ dlang.http.dll      # copied from the build output
└─ app.dl
```

Load it from a script with `require` (the returned string is a status message such as
`"plugin loaded!"`):

```dlang
require("dlang.http");
```

Once loaded, the plugin registers two globals: `http` and `view`.

---

## Quick start

`app.dl`:

```dlang
require("dlang.http");

var server = http.server();

// Serve everything in ./public under /assets (e.g. /assets/css/site.css)
server.static("/assets", "./public");

// A plain text route
server.route("/", ["GET"], func(req, params, cookies, files)
    return "Hello world!";
end);

// Query-string parameters arrive in `params`
server.route("/hello", ["GET"], func(req, params, cookies, files)
    return "Hello " + params.name + "!";
end);

// Return a table as `body` and the plugin JSON-encodes it for you
server.route("/api/status", ["GET"], func(req, params, cookies, files)
    return {
        body: {
            ok: true,
            name: "dlang.http"
        }
    };
end);

// Uploads: `files` holds the multipart/form-data file fields
server.route("/upload", ["POST"], func(req, params, cookies, files)
    if (#files > 0) then
        var saved = http.saveTo(files.avatar, "./uploads");
        return { body: "Saved to " + saved, contentType: "text/plain" };
    end
    return { body: "No file uploaded", statusCode: 400 };
end);

// Custom error page
server.error(404, func(req)
    return {
        body: "<h1>Not found</h1><p>" + req.path + "</p>",
        contentType: "text/html"
    };
end);

// Blocks and serves requests until the process is stopped
server.listen(8080);
```

> `#files` is the size operator: it returns the number of entries in the table. Because a
> missing table field evaluates to `nil` (and logs a VM warning), checking `#files` first
> keeps handlers free of warnings.

---

## API reference

### `http.server()`

Creates a server table. Every other `server.*` function is called on that table, and the
table itself is not sent to the OS socket until `listen` is called.

### `server.listen(port)`

Binds `0.0.0.0:<port>`, accepts connections in a loop and serves them until the process
ends. Returns `false` when the socket cannot be created, bound or listened on; on success it
never returns.

### `server.route(path, methods, handler)`

Registers a route.

- `path` – the exact request path, e.g. `"/users"` (no wildcards or path parameters; read
  `req.path` if you need to match patterns yourself).
- `methods` – a list of method names, e.g. `["GET", "POST"]`. Comparison is
  case-sensitive, so use uppercase.
- `handler` – a function (see below).

When a path matches but none of its methods do, the server answers `405 Method Not
Allowed`.

### Route handler arguments

Handlers are called with four arguments:

```dlang
func(req, params, cookies, files)
```

| Argument | Contents |
| --- | --- |
| `req` | Table with `method`, `path`, `query` (only when the URL had one), `params`, `cookies`, `files` and `body` (only when the request body was not empty) |
| `params` | All request parameters merged into one table: query string + body |
| `cookies` | The parsed `Cookie:` header |
| `files` | Uploaded files from a `multipart/form-data` body, keyed by form field name |

`params` is filled from the body depending on the `Content-Type`:

| `Content-Type` | Body handling |
| --- | --- |
| `application/x-www-form-urlencoded` | `a=1&b=2` decoded into strings |
| anything containing `json` | Parsed as a JSON object via the VM's `table.json_decode`, keeping numbers, booleans and nested tables/lists; only the top-level fields are merged |
| `multipart/form-data` | Text fields are merged; file fields go to `files` |
| anything else | Treated as url-encoded form data |

A key that occurs more than once (for example `?tag=a&tag=b`) becomes a list.

The four-argument signature is backward compatible: older three-parameter handlers keep
working, they simply never see `files`.

**Return values.** A handler returns either a string (sent as the body, `text/html` by
default) or a table:

```dlang
return {
    body: "text or a table (a table is JSON encoded)",
    contentType: "text/plain",
    statusCode: 200
};
```

If a handler throws, the server answers `500` using the configured error page.

### `server.static(...)`

Serves files from disk. Four call shapes are supported:

```dlang
server.static("/assets", "./public");            // URL prefix  -> folder
server.static("/style.css", "./style.css");      // exact URL   -> single file
server.static({ assets: "./public" });           // map of keys -> paths
server.static([["/a.css", "./a.css"]]);          // list of [url, file] pairs
```

Details:

- Table keys must be dlang identifiers (dlang tables have no string keys); a leading `/` is
  added automatically, so the `{ assets: "./public" }` form serves `/assets/...`.
- Folder mounts answer `/` with `index.html`, pick the longest matching prefix and detect the
  MIME type from the file extension.
- Exact URL → file mappings take precedence over folder mounts.
- Static files are only served when no route matches the path.
- Paths are URL-decoded first; anything containing `..` or `\` is rejected, so a request
  cannot escape the mounted folder.

### `server.error(code, handler)`

Overrides the page for a status code. The handler may be a body string, a function, or a
table:

```dlang
server.error(404, "<h1>Not found</h1>");
server.error(500, { body: "Something broke", contentType: "text/plain" });

server.error(404, func(req)
    // req = { method, path, statusCode }
    return { body: "Missing: " + req.path, contentType: "text/html" };
end);
```

Status codes without an override (and handler failures) fall back to a built-in dark-themed
HTML page. A custom body of `""` also falls back to the built-in page.

### `server.maxBody(bytes)`

Changes the maximum request body the server buffers. The default is **16 MiB**. Requests
that announce a larger `Content-Length` are refused with `413 Payload Too Large` before the
body is read:

```dlang
server.maxBody(64 * 1024 * 1024);   // allow 64 MiB uploads
```

### File uploads

An upload (`<form enctype="multipart/form-data">`, or `FormData` in `fetch`/XHR) is split
into its parts. Text fields end up in `params` like any other form post, and every file
field becomes an entry in `files`:

```dlang
func(req, params, cookies, files)
    var avatar = files.avatar;      // one entry per field name
    // avatar.field        -> "avatar"
    // avatar.filename     -> "cat.png"
    // avatar.contentType  -> "image/png"
    // avatar.size         -> 1234
    // avatar.content      -> the raw bytes
end
```

If several files are uploaded under the same field name (for example a multi-select), that
entry becomes a list of file tables. `filename*` (RFC 5987), quoted filenames, CRLF and
LF-only framing are all handled; malformed bodies are ignored instead of raising an error.

The same table is available as `req.files`, so a handler can read `req.files.avatar` if it
prefers not to use the fourth parameter. To store an upload, hand it to
[`http.save` / `http.saveTo`](#httpsave--httpsaveto):

```dlang
var saved = http.saveTo(files.avatar, "./uploads");
```

Raw bodies are available too: `req.body` holds the undecoded request body, which is handy
for uploads sent as `application/octet-stream`.

### `http.save` / `http.saveTo`

Write an upload to disk:

```dlang
http.save(upload, "./uploads/cat.png");   // -> true on success, false on failure
http.saveTo(upload, "./uploads");         // -> the written path (e.g. "./uploads/cat.png"), or "" on failure
```

The helpers live on the `http` table rather than a separate global, because a route
handler's fourth parameter is conventionally called `files` and would shadow a global of
the same name.

- `save` takes an explicit path and creates missing parent directories.
- `saveTo` appends the uploaded `filename` to a folder.
- Both accept a plain string as the first argument, so a raw body can be written as well:
  `http.save(req.body, "./uploads/blob.bin")`.
- The filename is sanitized before use: any directory part (`C:\Users\me\evil.txt`,
  `../../etc/passwd`) is stripped and unsafe characters are replaced, so a client can never
  choose the target directory.

### `view.Render(path, vars)`

Renders a small template file and returns the result as a string. `view.render` is an alias.
Template paths are read relative to the process working directory; a missing template raises
an error, which a route handler turns into a `500`.

```dlang
server.route("/", ["GET"], func(req, params, cookies, files)
    return view.Render("templates/page.html", {
        title: "Home",
        user: { name: "Ada" },
        items: ["a", "b"]
    });
end);
```

Template syntax:

| Syntax | Behaviour |
| --- | --- |
| `@include("partial.html")` | Inlines another template. Recursive, resolved relative to the including file (with a fallback to the working directory); cycles are skipped and nesting is limited to 32 levels |
| `{{ key }}` | Replaced with the matching variable. Strings, numbers and booleans are printed as-is |
| `{{ a.b }}` | Walks into nested tables |
| `{{ list }}` / `{{ table }}` | Serialized to JSON (convenient inside `<script>`) |

Unknown keys are left in the output as literal `{{key}}` text, which makes typos easy to
spot. Missing or cyclic includes are replaced with an HTML comment.

---

## Status codes

Responses use these standard reason phrases — anything else falls back to `OK`, while the
numeric status code is always the one you asked for:

`200 OK`, `201 Created`, `204 No Content`, `301 Moved Permanently`, `302 Found`,
`400 Bad Request`, `401 Unauthorized`, `403 Forbidden`, `404 Not Found`,
`405 Method Not Allowed`, `413 Payload Too Large`, `500 Internal Server Error`,
`501 Not Implemented`.

Responses are always sent with `Content-Length`, `Content-Type` and `Connection: close`.

---

## Request and response flow

```mermaid
flowchart TD
    A[Accept connection] --> B[Read headers + Content-Length body]
    B --> C{Body within limit?}
    C -- no --> D[413]
    C -- yes --> E[Parse query, cookies and body<br/>urlencoded / JSON / multipart]
    E --> F{Route matches path?}
    F -- yes, method allowed --> G[Call handler with req, params, cookies, files]
    F -- yes, other method --> H[405]
    F -- no --> I{Static mount matches?}
    I -- yes --> J[Serve file with detected MIME type]
    I -- no --> K[404]
    G --> L[Send response: string, or table with body / contentType / statusCode]
    G -. handler threw .-> M[500 with error page]
```

---

## Limitations

- **One request at a time.** `listen` accepts and handles connections sequentially in a
  blocking loop; there is no thread pool or keep-alive.
- **Exact route paths only.** No `:parameters`, wildcards or regex routes; match on
  `req.path` yourself if you need them.
- **`Content-Length` only.** Chunked transfer encoding is not supported, and a request
  without `Content-Length` has no body.
- **In-memory bodies.** The whole body (up to `server.maxBody`) is buffered before a handler
  runs, and static files are read fully into memory before being sent.
- **HTTP/1.1 only**, no TLS. Put a reverse proxy in front if you need HTTPS.
- **Global server state.** Routes, static mounts, error pages and the socket are
  process-wide, so one `listen` call serves one server per process.

---

## Project layout

```text
dlang-http/
├─ CMakeLists.txt                 # top-level project
├─ CMakePresets.json              # Windows / Linux / macOS configure presets
└─ dlang.http/
   ├─ CMakeLists.txt              # builds the `dlang.http` shared library
   ├─ entry.cpp                   # the entire plugin: server, routing, static files,
   │                              # uploads, error pages and the view engine
   ├─ json.hpp                    # nlohmann/json (used by the VM's table library)
   └─ dlang/                      # upstream dlang VM copy — do not edit
      ├─ vm.hpp, types/, libs/, parser/
      └─ src/                     # compiler, lexer, parser, VM implementation
```

Everything plugin-specific lives in `dlang.http/entry.cpp`; `dlang.http/dlang/` is a
verbatim copy of the dlang VM so the shared library can be built and link the VM
implementation.

---

## AI-generated code

This repository and this README were produced with the help of an AI assistant
(GitHub Copilot). That means:

- The code was **generated, then verified against its own test harnesses** (request parsing,
  multipart uploads, socket-level request handling, `http.save`/`saveTo`), but it has not
  been reviewed by a human maintainer.
- Comments and documentation may describe behaviour that has not been audited in every
  edge case, and the API is still evolving.
- Treat it as a starting point: review the source in `dlang.http/entry.cpp`, test it with
  your own inputs, and add a license before publishing or distributing it.

There is no warranty of any kind. Use it at your own risk.