Compare commits

..
9 Commits
5 changed files with 2846 additions and 82 deletions

No files matched your search

+4
View File
@@ -12,3 +12,7 @@ CTestTestfile.cmake
_deps _deps
CMakeUserPresets.json CMakeUserPresets.json
out/ out/
# Auth user store (holds password hashes) and its temporary file
users.json
users.json.tmp
+578
View File
@@ -0,0 +1,578 @@
# dlang.http
An HTTP/1.1 server plugin for the **dlang** VM. It gives a dlang script a `http` global with
routing, static file serving, custom error pages, a small template engine (`view`) and
file uploads (`multipart/form-data`).
> [!IMPORTANT]
> **This project was generated with AI** and has been reviewed by a
> human. Treat the code and this documentation as a starting point: read it, test it, and
> only use it in production once you understand and trust it. See
> [AI-generated code](#ai-generated-code) at the bottom.
> The dlang language and its VM live in their own repository. This repository only contains
> the HTTP plugin; the `dlang/` folder is an unmodified copy of the upstream VM headers and
> sources, kept here so the plugin can be built as a standalone shared library.
---
## Table of contents
- [Features](#features)
- [Requirements](#requirements)
- [Building](#building)
- [Installing the plugin](#installing-the-plugin)
- [Quick start](#quick-start)
- [API reference](#api-reference)
- [`http.server()`](#httpserver)
- [`server.listen(port)`](#serverlistenport)
- [`server.route(path, methods, handler)`](#serverroutepath-methods-handler)
- [Route handler arguments](#route-handler-arguments)
- [`server.static(...)`](#serverstatic)
- [`server.error(code, handler)`](#servererrorcode-handler)
- [`server.maxBody(bytes)`](#servermaxbodybytes)
- [File uploads](#file-uploads)
- [`http.save` / `http.saveTo`](#httpsave--httpsaveto)
- [Authentication (`auth.*`)](#authentication-auth)
- [`view.Render(path, vars)`](#viewrenderpath-vars)
- [Status codes](#status-codes)
- [Request and response flow](#request-and-response-flow)
- [Limitations](#limitations)
- [Project layout](#project-layout)
- [AI-generated code](#ai-generated-code)
---
## Features
| Feature | Summary |
| --- | --- |
| Routing | Exact-path routes with a per-route list of HTTP methods |
| Static files | URL prefix → folder, exact URL → file, MIME type detected by extension |
| Request parsing | Query string, url-encoded bodies, JSON bodies and `multipart/form-data` |
| File uploads | Uploaded files as tables (`filename`, `contentType`, `size`, raw `content`) plus `http.save` / `http.saveTo` helpers to write them to disk |
| Cookies | Parsed `Cookie:` header as a table |
| Authentication | HTTP Basic auth for the paths you choose, with PBKDF2-HMAC-SHA256 hashed passwords in `users.json` |
| Error pages | Custom handlers or bodies per status code, with a built-in HTML fallback |
| Templates | `view.Render("page.html", { ... })` with recursive `@include` and `{{ key }}` interpolation |
| Body limit | Configurable request size limit (`server.maxBody`), oversized uploads get `413` |
---
## Requirements
- **Windows** or **Linux** (the plugin refuses to compile on other platforms).
- A **C++20** compiler: MSVC (Windows) or Clang/GCC (Linux).
- **CMake ≥ 3.10** and **Ninja** for the presets in `CMakePresets.json`.
- A **dlang host** (interpreter/runner) that supports plugins, since this repository builds
a plugin, not a program.
---
## Building
The presets use Ninja with `cl.exe`, so run the commands from a *Developer Command Prompt*
(or build the folder from VS Code with the CMake Tools extension, which sets up the
compiler environment automatically).
```bash
# configure
cmake --preset x64-debug # or x64-release, x86-debug, x86-release
# linux-debug on Linux, macos-debug on macOS
# build
cmake --build out/build/x64-debug
```
Available presets: `x64-debug`, `x64-release`, `x86-debug`, `x86-release` (Windows),
`linux-debug` (Linux) and `macos-debug` (macOS).
The build output is a shared library:
| Platform | Output |
| --- | --- |
| Windows | `out/build/<preset>/dlang.http/dlang.http.dll` |
| Linux | `out/build/<preset>/dlang.http/dlang.http.so` |
> The shared library compiles `entry.cpp` together with the VM sources in
> `dlang.http/dlang/src/`, so no external dlang SDK is needed. Windows additionally links
> `ws2_32`; Linux links `dl` and `pthread`.
---
## Installing the plugin
Plugins are looked up in a `plugins/` folder relative to the working directory of the dlang
host. The loader appends the platform extension itself (`.dll` on Windows, `.so` on Linux),
so the file must be named `dlang.http.dll` / `dlang.http.so`:
```text
my-app/
├─ plugins/
│ └─ dlang.http.dll # copied from the build output
└─ app.dl
```
Load it from a script with `require` (the returned string is a status message such as
`"plugin loaded!"`):
```dlang
require("dlang.http");
```
Once loaded, the plugin registers three globals: `http`, `view` and `auth`.
---
## Quick start
`app.dl`:
```dlang
require("dlang.http");
// Basic auth: hash the password into ./users.json and gate /admin/**
auth.addUser("admin", "geheim");
auth.protectPrefix("/admin");
var server = http.server();
// Serve everything in ./public under /assets (e.g. /assets/css/site.css)
server.static("/assets", "./public");
// A plain text route
server.route("/", ["GET"], func(req, params, cookies, files)
return "Hello world!";
end);
// Query-string parameters arrive in `params`
server.route("/hello", ["GET"], func(req, params, cookies, files)
return "Hello " + params.name + "!";
end);
// Return a table as `body` and the plugin JSON-encodes it for you
server.route("/api/status", ["GET"], func(req, params, cookies, files)
return {
body: {
ok: true,
name: "dlang.http"
}
};
end);
// Uploads: `files` holds the multipart/form-data file fields
server.route("/upload", ["POST"], func(req, params, cookies, files)
if (#files > 0) then
var saved = http.saveTo(files.avatar, "./uploads");
return { body: "Saved to " + saved, contentType: "text/plain" };
end
return { body: "No file uploaded", statusCode: 400 };
end);
// Custom error page
server.error(404, func(req)
return {
body: "<h1>Not found</h1><p>" + req.path + "</p>",
contentType: "text/html"
};
end);
// Blocks and serves requests until the process is stopped
server.listen(8080);
```
> `#files` is the size operator: it returns the number of entries in the table. Because a
> missing table field evaluates to `nil` (and logs a VM warning), checking `#files` first
> keeps handlers free of warnings.
---
## API reference
### `http.server()`
Creates a server table. Every other `server.*` function is called on that table, and the
table itself is not sent to the OS socket until `listen` is called.
### `server.listen(port)`
Binds `0.0.0.0:<port>`, accepts connections in a loop and serves them until the process
ends. Returns `false` when the socket cannot be created, bound or listened on; on success it
never returns.
### `server.route(path, methods, handler)`
Registers a route.
- `path` – the exact request path, e.g. `"/users"` (no wildcards or path parameters; read
`req.path` if you need to match patterns yourself).
- `methods` – a list of method names, e.g. `["GET", "POST"]`. Comparison is
case-sensitive, so use uppercase.
- `handler` – a function (see below).
When a path matches but none of its methods do, the server answers `405 Method Not
Allowed`.
### Route handler arguments
Handlers are called with four arguments:
```dlang
func(req, params, cookies, files)
```
| Argument | Contents |
| --- | --- |
| `req` | Table with `method`, `path`, `query` (only when the URL had one), `headers`, `params`, `cookies`, `files`, `body` (only when the request body was not empty) and `user` (only when the request passed an [`auth`](#authentication-auth) gate) |
| `params` | All request parameters merged into one table: query string + body |
| `cookies` | The parsed `Cookie:` header |
| `files` | Uploaded files from a `multipart/form-data` body, keyed by form field name |
`params` is filled from the body depending on the `Content-Type`:
| `Content-Type` | Body handling |
| --- | --- |
| `application/x-www-form-urlencoded` | `a=1&b=2` decoded into strings |
| anything containing `json` | Parsed as a JSON object via the VM's `table.json_decode`, keeping numbers, booleans and nested tables/lists; only the top-level fields are merged |
| `multipart/form-data` | Text fields are merged; file fields go to `files` |
| anything else | Treated as url-encoded form data |
A key that occurs more than once (for example `?tag=a&tag=b`) becomes a list.
`req.headers` holds every request header with its name lower-cased, so
`req.headers.authorization` or `req.headers.content-type` can be read directly; a header
that occurs more than once becomes a list too.
The four-argument signature is backward compatible: older three-parameter handlers keep
working, they simply never see `files`.
**Return values.** A handler returns either a string (sent as the body, `text/html` by
default) or a table:
```dlang
return {
body: "text or a table (a table is JSON encoded)",
contentType: "text/plain",
statusCode: 200
};
```
If a handler throws, the server answers `500` using the configured error page.
### `server.static(...)`
Serves files from disk. Four call shapes are supported:
```dlang
server.static("/assets", "./public"); // URL prefix -> folder
server.static("/style.css", "./style.css"); // exact URL -> single file
server.static({ assets: "./public" }); // map of keys -> paths
server.static([["/a.css", "./a.css"]]); // list of [url, file] pairs
```
Details:
- Table keys must be dlang identifiers (dlang tables have no string keys); a leading `/` is
added automatically, so the `{ assets: "./public" }` form serves `/assets/...`.
- Folder mounts answer `/` with `index.html`, pick the longest matching prefix and detect the
MIME type from the file extension.
- Exact URL → file mappings take precedence over folder mounts.
- Static files are only served when no route matches the path.
- Paths are URL-decoded first; anything containing `..` or `\` is rejected, so a request
cannot escape the mounted folder.
### `server.error(code, handler)`
Overrides the page for a status code. The handler may be a body string, a function, or a
table:
```dlang
server.error(404, "<h1>Not found</h1>");
server.error(500, { body: "Something broke", contentType: "text/plain" });
server.error(404, func(req)
// req = { method, path, statusCode }
return { body: "Missing: " + req.path, contentType: "text/html" };
end);
```
Status codes without an override (and handler failures) fall back to a built-in dark-themed
HTML page. A custom body of `""` also falls back to the built-in page.
### `server.maxBody(bytes)`
Changes the maximum request body the server buffers. The default is **16 MiB**. Requests
that announce a larger `Content-Length` are refused with `413 Payload Too Large` before the
body is read:
```dlang
server.maxBody(64 * 1024 * 1024); // allow 64 MiB uploads
```
### File uploads
An upload (`<form enctype="multipart/form-data">`, or `FormData` in `fetch`/XHR) is split
into its parts. Text fields end up in `params` like any other form post, and every file
field becomes an entry in `files`:
```dlang
func(req, params, cookies, files)
var avatar = files.avatar; // one entry per field name
// avatar.field -> "avatar"
// avatar.filename -> "cat.png"
// avatar.contentType -> "image/png"
// avatar.size -> 1234
// avatar.content -> the raw bytes
end
```
If several files are uploaded under the same field name (for example a multi-select), that
entry becomes a list of file tables. `filename*` (RFC 5987), quoted filenames, CRLF and
LF-only framing are all handled; malformed bodies are ignored instead of raising an error.
The same table is available as `req.files`, so a handler can read `req.files.avatar` if it
prefers not to use the fourth parameter. To store an upload, hand it to
[`http.save` / `http.saveTo`](#httpsave--httpsaveto):
```dlang
var saved = http.saveTo(files.avatar, "./uploads");
```
Raw bodies are available too: `req.body` holds the undecoded request body, which is handy
for uploads sent as `application/octet-stream`.
### `http.save` / `http.saveTo`
Write an upload to disk:
```dlang
http.save(upload, "./uploads/cat.png"); // -> true on success, false on failure
http.saveTo(upload, "./uploads"); // -> the written path (e.g. "./uploads/cat.png"), or "" on failure
```
The helpers live on the `http` table rather than a separate global, because a route
handler's fourth parameter is conventionally called `files` and would shadow a global of
the same name.
- `save` takes an explicit path and creates missing parent directories.
- `saveTo` appends the uploaded `filename` to a folder.
- Both accept a plain string as the first argument, so a raw body can be written as well:
`http.save(req.body, "./uploads/blob.bin")`.
- The filename is sanitized before use: any directory part (`C:\Users\me\evil.txt`,
`../../etc/passwd`) is stripped and unsafe characters are replaced, so a client can never
choose the target directory.
### Authentication (`auth.*`)
Optional HTTP Basic auth. You choose which paths are protected; everything else stays open.
Passwords are **hashed, not encrypted**: the plugin generates a random salt per account and
stores a PBKDF2-HMAC-SHA256 hash of the password, so the file never holds anything that can
be turned back into the original password.
```dlang
require("dlang.http");
auth.addUser("admin", "geheim"); // hashes the password and writes ./users.json
auth.realm("My site"); // the text shown in the browser's login popup
auth.protectPrefix("/admin"); // /admin and everything below need credentials
var server = http.server();
server.route("/admin", ["GET"], func(req, params, cookies, files)
return "<h1>Welcome back, " + req.user + "</h1>";
end);
server.listen(8080);
```
The first request for a protected path is answered with `401 Unauthorized` plus a
`WWW-Authenticate: Basic realm="..."` header. A browser reacts to that header by showing its
own username/password dialog; once the visitor fills it in, the request is repeated with an
`Authorization: Basic ...` header and the page is served. Inside the handler `req.user` holds
the authenticated name.
The user file lives in the working directory (`users.json` by default) — add it to your
`.gitignore`, since it holds password hashes.
#### The user store
| Call | Description |
| --- | --- |
| `auth.file(path)` | Use another user file; returns the path in use. Default: `users.json`. |
| `auth.reload()` | Re-read the file from disk (after editing it by hand). `true` on success. |
| `auth.addUser(name, password)` | Create the account or replace its password. Writes the file and returns `true`, or `false` when the name/password is rejected or the file cannot be written. |
| `auth.removeUser(name)` | Delete the account. `true` when it existed and the file was written. |
| `auth.hasUser(name)` | `true` when the account exists. |
| `auth.users()` | List of user names. |
| `auth.verify(name, password)` | `true` when the password matches — to check credentials inside your own handler. |
| `auth.login(req)` | The name from the request's `Authorization` header, or `""`. |
| `auth.iterations(count)` | PBKDF2 rounds for new or updated passwords (default `100000`, clamped to `1000`–`1000000`); returns the value in use. |
| `auth.realm(name)` | The realm in the login popup (default `Restricted`); returns the value in use. |
A name may not be empty or contain `:`, whitespace or control characters (it is sent as
`name:password`), and the password may not be empty. `addUser` never overwrites a
`users.json` it cannot parse — it reports the problem on `stderr` instead, so a hand-edited
file with a typo cannot silently wipe your accounts.
The file looks like this (`salt` and `hash` are hex; `iterations` is stored per account so
the cost can be raised later without invalidating existing passwords):
```json
{
"algorithm": "pbkdf2-sha256",
"users": [
{ "username": "admin", "salt": "ca63c036f511d4006c93ad68353677d7",
"hash": "d4b7284ab9668ecbb548f8fd8048b44200368b258cd2b87d00807737c9aec472",
"iterations": 100000 }
]
}
```
Accounts are read lazily, so `auth.addUser(...)` can simply run just before
`server.listen(...)`. A password that verified once is remembered in memory (as a hash of the
password, never the password itself), so the deliberately slow PBKDF2 work is paid for the
first request of a session instead of every request. Editing the file by hand? Call
`auth.reload()`.
#### Protecting paths
| Call | Description |
| --- | --- |
| `auth.protect(path)` | Require credentials for exactly this path. |
| `auth.protectPrefix(path)` | Require credentials for this path and everything below it. |
| `auth.unprotect(path)` | Remove an exact-path protection again. |
| `auth.unprotectPrefix(path)` | Remove a prefix protection again. |
Paths are normalised (`admin/panel/` becomes `/admin/panel`) and `auth.protectPrefix("/")`
gates the whole site. The check happens before routing, static files and error pages, so a
protected path never leaks whether it exists: `/secret/nope` answers `401` without
credentials and `404` with them. The `401` body comes from
[`server.error(401, ...)`](#servererrorcode-handler) like any other status code; only the
`WWW-Authenticate` header is added by the plugin.
### `view.Render(path, vars)`
Renders a small template file and returns the result as a string. `view.render` is an alias.
Template paths are read relative to the process working directory; a missing template raises
an error, which a route handler turns into a `500`.
```dlang
server.route("/", ["GET"], func(req, params, cookies, files)
return view.Render("templates/page.html", {
title: "Home",
user: { name: "Ada" },
items: ["a", "b"]
});
end);
```
Template syntax:
| Syntax | Behaviour |
| --- | --- |
| `@include("partial.html")` | Inlines another template. Recursive, resolved relative to the including file (with a fallback to the working directory); cycles are skipped and nesting is limited to 32 levels |
| `{{ key }}` | Replaced with the matching variable. Strings, numbers and booleans are printed as-is |
| `{{ a.b }}` | Walks into nested tables |
| `{{ list }}` / `{{ table }}` | Serialized to JSON (convenient inside `<script>`) |
Unknown keys are left in the output as literal `{{key}}` text, which makes typos easy to
spot. Missing or cyclic includes are replaced with an HTML comment.
---
## Status codes
Responses use these standard reason phrases — anything else falls back to `OK`, while the
numeric status code is always the one you asked for:
`200 OK`, `201 Created`, `204 No Content`, `301 Moved Permanently`, `302 Found`,
`400 Bad Request`, `401 Unauthorized`, `403 Forbidden`, `404 Not Found`,
`405 Method Not Allowed`, `413 Payload Too Large`, `500 Internal Server Error`,
`501 Not Implemented`.
Responses are always sent with `Content-Length`, `Content-Type` and `Connection: close`.
---
## Request and response flow
```mermaid
flowchart TD
A[Accept connection] --> B[Read headers + Content-Length body]
B --> C{Body within limit?}
C -- no --> D[413]
C -- yes --> N{Protected path?}
N -- "wrong or missing credentials" --> O[401 + WWW-Authenticate]
N -- "not protected, or valid credentials" --> E[Parse query, headers, cookies and body<br/>urlencoded / JSON / multipart]
E --> F{Route matches path?}
F -- yes, method allowed --> G[Call handler with req, params, cookies, files]
F -- yes, other method --> H[405]
F -- no --> I{Static mount matches?}
I -- yes --> J[Serve file with detected MIME type]
I -- no --> K[404]
G --> L[Send response: string, or table with body / contentType / statusCode]
G -. handler threw .-> M[500 with error page]
```
---
## Limitations
- **One request at a time.** `listen` accepts and handles connections sequentially in a
blocking loop; there is no thread pool or keep-alive.
- **Exact route paths only.** No `:parameters`, wildcards or regex routes; match on
`req.path` yourself if you need them.
- **`Content-Length` only.** Chunked transfer encoding is not supported, and a request
without `Content-Length` has no body.
- **In-memory bodies.** The whole body (up to `server.maxBody`) is buffered before a handler
runs, and static files are read fully into memory before being sent.
- **HTTP/1.1 only**, no TLS. Put a reverse proxy in front if you need HTTPS.
- **Basic auth is only as safe as the transport.** The browser resends the password with
every request (base64 encoded, not encrypted), so use it behind HTTPS — or keep it on a
trusted network.
- **A wrong password costs a full PBKDF2 run.** `auth.iterations` defaults to 100 000, and
the server handles one request at a time, so an attacker can keep it busy with bad
credentials. Lower the iteration count or add rate limiting if that matters to you.
- **Global server state.** Routes, static mounts, error pages, the protected paths and the
loaded accounts are process-wide, so one `listen` call serves one server per process.
---
## Project layout
```text
dlang-http/
├─ CMakeLists.txt # top-level project
├─ CMakePresets.json # Windows / Linux / macOS configure presets
└─ dlang.http/
├─ CMakeLists.txt # builds the `dlang.http` shared library
├─ entry.cpp # the entire plugin: server, routing, static files,
│ # uploads, error pages, Basic auth and the view engine
├─ json.hpp # nlohmann/json (used by the VM's table library)
└─ dlang/ # upstream dlang VM copy — do not edit
├─ vm.hpp, types/, libs/, parser/
└─ src/ # compiler, lexer, parser, VM implementation
```
Everything plugin-specific lives in `dlang.http/entry.cpp`; `dlang.http/dlang/` is a
verbatim copy of the dlang VM so the shared library can be built and link the VM
implementation.
---
## AI-generated code
This repository and this README were produced with the help of an AI assistant
(GitHub Copilot). That means:
- The code was **generated, then verified against its own test harnesses** (request parsing,
multipart uploads, socket-level request handling, `http.save`/`saveTo`, and the auth
module including its SHA-256/HMAC/PBKDF2 test vectors), but it has not been reviewed by a
human maintainer.
- Comments and documentation may describe behaviour that has not been audited in every
edge case, and the API is still evolving.
- Treat it as a starting point: review the source in `dlang.http/entry.cpp`, test it with
your own inputs, and add a license before publishing or distributing it.
There is no warranty of any kind. Use it at your own risk.
+20 -6
View File
@@ -200,13 +200,25 @@ void DlangVM::Execute(ByteCodeArray bytecode)
auto b = stack.pop(); auto b = stack.pop();
auto a = stack.pop(); auto a = stack.pop();
if (a.type != StackValueType::Int || b.type != StackValueType::Int) if (a.type == StackValueType::Int && b.type == StackValueType::Int)
throw std::runtime_error("LESS_THAN Operation error, not all variables are integer!"); {
stack.push(StackValue(StackValueType::Bool,
std::get<double>(a.data) ==
std::get<double>(b.data)
));
continue;
}
stack.push(StackValue(StackValueType::Bool, if (a.type == StackValueType::String && b.type == StackValueType::String)
std::get<double>(a.data) == {
std::get<double>(b.data) stack.push(StackValue(StackValueType::Bool,
)); std::get<std::string>(a.data) ==
std::get<std::string>(b.data)
));
continue;
}
throw std::runtime_error("COMPARE Operation error, in compatible variables!");
} break; } break;
case Opcodes::GTE: { case Opcodes::GTE: {
@@ -284,6 +296,8 @@ void DlangVM::Execute(ByteCodeArray bytecode)
if (a.type == StackValueType::Int && b.type == StackValueType::Int) { if (a.type == StackValueType::Int && b.type == StackValueType::Int) {
result.data = (std::get<double>(a.data) != std::get<double>(b.data)); result.data = (std::get<double>(a.data) != std::get<double>(b.data));
} else if(a.type == StackValueType::String && b.type == StackValueType::String) {
result.data = (std::get<std::string>(a.data) != std::get<std::string>(b.data));
} }
stack.push(result); stack.push(result);
+1 -1
View File
@@ -12,7 +12,7 @@
inline ModuleManager* pluginManager = nullptr; inline ModuleManager* pluginManager = nullptr;
#define DEBUG_MODE 0 #define DEBUG_MODE 0
#define VERSION "v1.0.36" #define VERSION "v1.0.39"
using ByteCodeArray = std::vector<double>; using ByteCodeArray = std::vector<double>;
+2243 -75
View File
File diff suppressed because it is too large. Load diff