Merge branch 'master' of https://git.enfusionsolutions.com/damian/dlang-http

README.md

@@ -0,0 +1,470 @@
+# dlang.http
+
+An HTTP/1.1 server plugin for the **dlang** VM. It gives a dlang script a `http` global with
+routing, static file serving, custom error pages, a small template engine (`view`) and
+file uploads (`multipart/form-data`).
+
+> [!IMPORTANT]
+> **This project was generated with AI** (GitHub Copilot) and has not been reviewed by a
+> human. Treat the code and this documentation as a starting point: read it, test it, and
+> only use it in production once you understand and trust it. See
+> [AI-generated code](#ai-generated-code) at the bottom.
+
+> The dlang language and its VM live in their own repository. This repository only contains
+> the HTTP plugin; the `dlang/` folder is an unmodified copy of the upstream VM headers and
+> sources, kept here so the plugin can be built as a standalone shared library.
+
+---
+
+## Table of contents
+
+- [Features](#features)
+- [Requirements](#requirements)
+- [Building](#building)
+- [Installing the plugin](#installing-the-plugin)
+- [Quick start](#quick-start)
+- [API reference](#api-reference)
+  - [`http.server()`](#httpserver)
+  - [`server.listen(port)`](#serverlistenport)
+  - [`server.route(path, methods, handler)`](#serverroutepath-methods-handler)
+  - [Route handler arguments](#route-handler-arguments)
+  - [`server.static(...)`](#serverstatic)
+  - [`server.error(code, handler)`](#servererrorcode-handler)
+  - [`server.maxBody(bytes)`](#servermaxbodybytes)
+  - [File uploads](#file-uploads)
+  - [`http.save` / `http.saveTo`](#httpsave--httpsaveto)
+  - [`view.Render(path, vars)`](#viewrenderpath-vars)
+- [Status codes](#status-codes)
+- [Request and response flow](#request-and-response-flow)
+- [Limitations](#limitations)
+- [Project layout](#project-layout)
+- [AI-generated code](#ai-generated-code)
+
+---
+
+## Features
+
+| Feature | Summary |
+| --- | --- |
+| Routing | Exact-path routes with a per-route list of HTTP methods |
+| Static files | URL prefix → folder, exact URL → file, MIME type detected by extension |
+| Request parsing | Query string, url-encoded bodies, JSON bodies and `multipart/form-data` |
+| File uploads | Uploaded files as tables (`filename`, `contentType`, `size`, raw `content`) plus `http.save` / `http.saveTo` helpers to write them to disk |
+| Cookies | Parsed `Cookie:` header as a table |
+| Error pages | Custom handlers or bodies per status code, with a built-in HTML fallback |
+| Templates | `view.Render("page.html", { ... })` with recursive `@include` and `{{ key }}` interpolation |
+| Body limit | Configurable request size limit (`server.maxBody`), oversized uploads get `413` |
+
+---
+
+## Requirements
+
+- **Windows** or **Linux** (the plugin refuses to compile on other platforms).
+- A **C++20** compiler: MSVC (Windows) or Clang/GCC (Linux).
+- **CMake ≥ 3.10** and **Ninja** for the presets in `CMakePresets.json`.
+- A **dlang host** (interpreter/runner) that supports plugins, since this repository builds
+  a plugin, not a program.
+
+---
+
+## Building
+
+The presets use Ninja with `cl.exe`, so run the commands from a *Developer Command Prompt*
+(or build the folder from VS Code with the CMake Tools extension, which sets up the
+compiler environment automatically).
+
+```bash
+# configure
+cmake --preset x64-debug        # or x64-release, x86-debug, x86-release
+                                # linux-debug on Linux, macos-debug on macOS
+
+# build
+cmake --build out/build/x64-debug
+```
+
+Available presets: `x64-debug`, `x64-release`, `x86-debug`, `x86-release` (Windows),
+`linux-debug` (Linux) and `macos-debug` (macOS).
+
+The build output is a shared library:
+
+| Platform | Output |
+| --- | --- |
+| Windows | `out/build/<preset>/dlang.http/dlang.http.dll` |
+| Linux | `out/build/<preset>/dlang.http/dlang.http.so` |
+
+> The shared library compiles `entry.cpp` together with the VM sources in
+> `dlang.http/dlang/src/`, so no external dlang SDK is needed. Windows additionally links
+> `ws2_32`; Linux links `dl` and `pthread`.
+
+---
+
+## Installing the plugin
+
+Plugins are looked up in a `plugins/` folder relative to the working directory of the dlang
+host. The loader appends the platform extension itself (`.dll` on Windows, `.so` on Linux),
+so the file must be named `dlang.http.dll` / `dlang.http.so`:
+
+```text
+my-app/
+├─ plugins/
+│  └─ dlang.http.dll      # copied from the build output
+└─ app.dl
+```
+
+Load it from a script with `require` (the returned string is a status message such as
+`"plugin loaded!"`):
+
+```dlang
+require("dlang.http");
+```
+
+Once loaded, the plugin registers two globals: `http` and `view`.
+
+---
+
+## Quick start
+
+`app.dl`:
+
+```dlang
+require("dlang.http");
+
+var server = http.server();
+
+// Serve everything in ./public under /assets (e.g. /assets/css/site.css)
+server.static("/assets", "./public");
+
+// A plain text route
+server.route("/", ["GET"], func(req, params, cookies, files)
+    return "Hello world!";
+end);
+
+// Query-string parameters arrive in `params`
+server.route("/hello", ["GET"], func(req, params, cookies, files)
+    return "Hello " + params.name + "!";
+end);
+
+// Return a table as `body` and the plugin JSON-encodes it for you
+server.route("/api/status", ["GET"], func(req, params, cookies, files)
+    return {
+        body: {
+            ok: true,
+            name: "dlang.http"
+        }
+    };
+end);
+
+// Uploads: `files` holds the multipart/form-data file fields
+server.route("/upload", ["POST"], func(req, params, cookies, files)
+    if (#files > 0) then
+        var saved = http.saveTo(files.avatar, "./uploads");
+        return { body: "Saved to " + saved, contentType: "text/plain" };
+    end
+    return { body: "No file uploaded", statusCode: 400 };
+end);
+
+// Custom error page
+server.error(404, func(req)
+    return {
+        body: "<h1>Not found</h1><p>" + req.path + "</p>",
+        contentType: "text/html"
+    };
+end);
+
+// Blocks and serves requests until the process is stopped
+server.listen(8080);
+```
+
+> `#files` is the size operator: it returns the number of entries in the table. Because a
+> missing table field evaluates to `nil` (and logs a VM warning), checking `#files` first
+> keeps handlers free of warnings.
+
+---
+
+## API reference
+
+### `http.server()`
+
+Creates a server table. Every other `server.*` function is called on that table, and the
+table itself is not sent to the OS socket until `listen` is called.
+
+### `server.listen(port)`
+
+Binds `0.0.0.0:<port>`, accepts connections in a loop and serves them until the process
+ends. Returns `false` when the socket cannot be created, bound or listened on; on success it
+never returns.
+
+### `server.route(path, methods, handler)`
+
+Registers a route.
+
+- `path` – the exact request path, e.g. `"/users"` (no wildcards or path parameters; read
+  `req.path` if you need to match patterns yourself).
+- `methods` – a list of method names, e.g. `["GET", "POST"]`. Comparison is
+  case-sensitive, so use uppercase.
+- `handler` – a function (see below).
+
+When a path matches but none of its methods do, the server answers `405 Method Not
+Allowed`.
+
+### Route handler arguments
+
+Handlers are called with four arguments:
+
+```dlang
+func(req, params, cookies, files)
+```
+
+| Argument | Contents |
+| --- | --- |
+| `req` | Table with `method`, `path`, `query` (only when the URL had one), `params`, `cookies`, `files` and `body` (only when the request body was not empty) |
+| `params` | All request parameters merged into one table: query string + body |
+| `cookies` | The parsed `Cookie:` header |
+| `files` | Uploaded files from a `multipart/form-data` body, keyed by form field name |
+
+`params` is filled from the body depending on the `Content-Type`:
+
+| `Content-Type` | Body handling |
+| --- | --- |
+| `application/x-www-form-urlencoded` | `a=1&b=2` decoded into strings |
+| anything containing `json` | Parsed as a JSON object via the VM's `table.json_decode`, keeping numbers, booleans and nested tables/lists; only the top-level fields are merged |
+| `multipart/form-data` | Text fields are merged; file fields go to `files` |
+| anything else | Treated as url-encoded form data |
+
+A key that occurs more than once (for example `?tag=a&tag=b`) becomes a list.
+
+The four-argument signature is backward compatible: older three-parameter handlers keep
+working, they simply never see `files`.
+
+**Return values.** A handler returns either a string (sent as the body, `text/html` by
+default) or a table:
+
+```dlang
+return {
+    body: "text or a table (a table is JSON encoded)",
+    contentType: "text/plain",
+    statusCode: 200
+};
+```
+
+If a handler throws, the server answers `500` using the configured error page.
+
+### `server.static(...)`
+
+Serves files from disk. Four call shapes are supported:
+
+```dlang
+server.static("/assets", "./public");            // URL prefix  -> folder
+server.static("/style.css", "./style.css");      // exact URL   -> single file
+server.static({ assets: "./public" });           // map of keys -> paths
+server.static([["/a.css", "./a.css"]]);          // list of [url, file] pairs
+```
+
+Details:
+
+- Table keys must be dlang identifiers (dlang tables have no string keys); a leading `/` is
+  added automatically, so the `{ assets: "./public" }` form serves `/assets/...`.
+- Folder mounts answer `/` with `index.html`, pick the longest matching prefix and detect the
+  MIME type from the file extension.
+- Exact URL → file mappings take precedence over folder mounts.
+- Static files are only served when no route matches the path.
+- Paths are URL-decoded first; anything containing `..` or `\` is rejected, so a request
+  cannot escape the mounted folder.
+
+### `server.error(code, handler)`
+
+Overrides the page for a status code. The handler may be a body string, a function, or a
+table:
+
+```dlang
+server.error(404, "<h1>Not found</h1>");
+server.error(500, { body: "Something broke", contentType: "text/plain" });
+
+server.error(404, func(req)
+    // req = { method, path, statusCode }
+    return { body: "Missing: " + req.path, contentType: "text/html" };
+end);
+```
+
+Status codes without an override (and handler failures) fall back to a built-in dark-themed
+HTML page. A custom body of `""` also falls back to the built-in page.
+
+### `server.maxBody(bytes)`
+
+Changes the maximum request body the server buffers. The default is **16 MiB**. Requests
+that announce a larger `Content-Length` are refused with `413 Payload Too Large` before the
+body is read:
+
+```dlang
+server.maxBody(64 * 1024 * 1024);   // allow 64 MiB uploads
+```
+
+### File uploads
+
+An upload (`<form enctype="multipart/form-data">`, or `FormData` in `fetch`/XHR) is split
+into its parts. Text fields end up in `params` like any other form post, and every file
+field becomes an entry in `files`:
+
+```dlang
+func(req, params, cookies, files)
+    var avatar = files.avatar;      // one entry per field name
+    // avatar.field        -> "avatar"
+    // avatar.filename     -> "cat.png"
+    // avatar.contentType  -> "image/png"
+    // avatar.size         -> 1234
+    // avatar.content      -> the raw bytes
+end
+```
+
+If several files are uploaded under the same field name (for example a multi-select), that
+entry becomes a list of file tables. `filename*` (RFC 5987), quoted filenames, CRLF and
+LF-only framing are all handled; malformed bodies are ignored instead of raising an error.
+
+The same table is available as `req.files`, so a handler can read `req.files.avatar` if it
+prefers not to use the fourth parameter. To store an upload, hand it to
+[`http.save` / `http.saveTo`](#httpsave--httpsaveto):
+
+```dlang
+var saved = http.saveTo(files.avatar, "./uploads");
+```
+
+Raw bodies are available too: `req.body` holds the undecoded request body, which is handy
+for uploads sent as `application/octet-stream`.
+
+### `http.save` / `http.saveTo`
+
+Write an upload to disk:
+
+```dlang
+http.save(upload, "./uploads/cat.png");   // -> true on success, false on failure
+http.saveTo(upload, "./uploads");         // -> the written path (e.g. "./uploads/cat.png"), or "" on failure
+```
+
+The helpers live on the `http` table rather than a separate global, because a route
+handler's fourth parameter is conventionally called `files` and would shadow a global of
+the same name.
+
+- `save` takes an explicit path and creates missing parent directories.
+- `saveTo` appends the uploaded `filename` to a folder.
+- Both accept a plain string as the first argument, so a raw body can be written as well:
+  `http.save(req.body, "./uploads/blob.bin")`.
+- The filename is sanitized before use: any directory part (`C:\Users\me\evil.txt`,
+  `../../etc/passwd`) is stripped and unsafe characters are replaced, so a client can never
+  choose the target directory.
+
+### `view.Render(path, vars)`
+
+Renders a small template file and returns the result as a string. `view.render` is an alias.
+Template paths are read relative to the process working directory; a missing template raises
+an error, which a route handler turns into a `500`.
+
+```dlang
+server.route("/", ["GET"], func(req, params, cookies, files)
+    return view.Render("templates/page.html", {
+        title: "Home",
+        user: { name: "Ada" },
+        items: ["a", "b"]
+    });
+end);
+```
+
+Template syntax:
+
+| Syntax | Behaviour |
+| --- | --- |
+| `@include("partial.html")` | Inlines another template. Recursive, resolved relative to the including file (with a fallback to the working directory); cycles are skipped and nesting is limited to 32 levels |
+| `{{ key }}` | Replaced with the matching variable. Strings, numbers and booleans are printed as-is |
+| `{{ a.b }}` | Walks into nested tables |
+| `{{ list }}` / `{{ table }}` | Serialized to JSON (convenient inside `<script>`) |
+
+Unknown keys are left in the output as literal `{{key}}` text, which makes typos easy to
+spot. Missing or cyclic includes are replaced with an HTML comment.
+
+---
+
+## Status codes
+
+Responses use these standard reason phrases — anything else falls back to `OK`, while the
+numeric status code is always the one you asked for:
+
+`200 OK`, `201 Created`, `204 No Content`, `301 Moved Permanently`, `302 Found`,
+`400 Bad Request`, `401 Unauthorized`, `403 Forbidden`, `404 Not Found`,
+`405 Method Not Allowed`, `413 Payload Too Large`, `500 Internal Server Error`,
+`501 Not Implemented`.
+
+Responses are always sent with `Content-Length`, `Content-Type` and `Connection: close`.
+
+---
+
+## Request and response flow
+
+```mermaid
+flowchart TD
+    A[Accept connection] --> B[Read headers + Content-Length body]
+    B --> C{Body within limit?}
+    C -- no --> D[413]
+    C -- yes --> E[Parse query, cookies and body<br/>urlencoded / JSON / multipart]
+    E --> F{Route matches path?}
+    F -- yes, method allowed --> G[Call handler with req, params, cookies, files]
+    F -- yes, other method --> H[405]
+    F -- no --> I{Static mount matches?}
+    I -- yes --> J[Serve file with detected MIME type]
+    I -- no --> K[404]
+    G --> L[Send response: string, or table with body / contentType / statusCode]
+    G -. handler threw .-> M[500 with error page]
+```
+
+---
+
+## Limitations
+
+- **One request at a time.** `listen` accepts and handles connections sequentially in a
+  blocking loop; there is no thread pool or keep-alive.
+- **Exact route paths only.** No `:parameters`, wildcards or regex routes; match on
+  `req.path` yourself if you need them.
+- **`Content-Length` only.** Chunked transfer encoding is not supported, and a request
+  without `Content-Length` has no body.
+- **In-memory bodies.** The whole body (up to `server.maxBody`) is buffered before a handler
+  runs, and static files are read fully into memory before being sent.
+- **HTTP/1.1 only**, no TLS. Put a reverse proxy in front if you need HTTPS.
+- **Global server state.** Routes, static mounts, error pages and the socket are
+  process-wide, so one `listen` call serves one server per process.
+
+---
+
+## Project layout
+
+```text
+dlang-http/
+├─ CMakeLists.txt                 # top-level project
+├─ CMakePresets.json              # Windows / Linux / macOS configure presets
+└─ dlang.http/
+   ├─ CMakeLists.txt              # builds the `dlang.http` shared library
+   ├─ entry.cpp                   # the entire plugin: server, routing, static files,
+   │                              # uploads, error pages and the view engine
+   ├─ json.hpp                    # nlohmann/json (used by the VM's table library)
+   └─ dlang/                      # upstream dlang VM copy — do not edit
+      ├─ vm.hpp, types/, libs/, parser/
+      └─ src/                     # compiler, lexer, parser, VM implementation
+```
+
+Everything plugin-specific lives in `dlang.http/entry.cpp`; `dlang.http/dlang/` is a
+verbatim copy of the dlang VM so the shared library can be built and link the VM
+implementation.
+
+---
+
+## AI-generated code
+
+This repository and this README were produced with the help of an AI assistant
+(GitHub Copilot). That means:
+
+- The code was **generated, then verified against its own test harnesses** (request parsing,
+  multipart uploads, socket-level request handling, `http.save`/`saveTo`), but it has not
+  been reviewed by a human maintainer.
+- Comments and documentation may describe behaviour that has not been audited in every
+  edge case, and the API is still evolving.
+- Treat it as a starting point: review the source in `dlang.http/entry.cpp`, test it with
+  your own inputs, and add a license before publishing or distributing it.
+
+There is no warranty of any kind. Use it at your own risk.