Configuration

The goal of the Configuration module is to let you declaratively define how your application pulls in values from the system and structure them in an easy accessible format.

no magic — you tell it how you want things done.
always available — as much as possible, configuration should parse and and use fallback values so it is as easy as possible to run the app.
have precident — cli flags > environment variables > configuration files > fallbacks, in that order
always valid — once declared, your value should always match that structure so there are no unexpected errors

Recommended reading 12 fractured apps - it really inspired the design of configuration

Things you might want to configure:

How much logging to do
The databases to connect to
Which features to turn on or off
Tokens for thirdy-party APIs
Who to send emails from

You define your configuration like this:

import { getConfiguration } from "gruber";

const config = getConfiguration();

const struct = config.object({
  env: config.string({ variable: "NODE_ENV", fallback: "development" }),

  server: config.object({
    port: config.number({
      variable: "APP_PORT",
      flag: "--port",
      fallback: 3000,
    }),
    url: config.url({
      variable: "SELF_URL",
      fallback: "http://localhost:3000",
    }),
  }),
});

You can start to see the shape of your configuration, at the top-level there is env which pulls from the NODE_ENV environment variable and falls back to "development" if not set.

There is also a server object with a port number and url field, each pulling different variables or flags and providing their own fallbacks. Fallbacks are important so you don't have to specify every single argument when you set up a fresh environment.

Another Pattern I like to use is to include a meta object like below. It allows you to have a single source of truth for the name and version of the app. Which is often useful in a meta endpoint that returns this information, or if the app needs to construct a User-Agent to represent itself.

import pkg from "./package.json" with { type: "json" };

const config = getConfiguration();

const struct = config.object({
  // ...
  meta: config.object({
    name: config.string({ variable: "APP_NAME", fallback: pkg.name }),
    version: config.string({ variable: "APP_VERSION", fallback: pkg.version }),
  }),
});

Once you have that structure, you can use it to parse your configuration:

// Load the configuration and parse it
export function loadConfiguration(path) {
  return config.load(path, struct);
}

// The configutation for use in the application
export const appConfig = await loadConfiguration(
  new URL("./config.json", import.meta.url),
);

You use config.load to process the configuration against given file and the environment itself.

A nice pattern to use is to also export the appConfig so it is easy to access everywhere and strongly-typed.

You can do more with configuration, its useful to see how your app is currently configured and what options are available:

// A method to generate usage documentation
export function getConfigurationUsage() {
  return config.getUsage(struct, appConfig);
}

Which when called, will output something like this:

Usage:

| name         | type   | flag   | variable    | fallback               |
| ------------ | ------ | -------| ----------- | ---------------------- |
| env          | string | ~      | NODE_ENV    | development            |
| server.port  | number | --port | PORT        | 3000                   |
| server.url   | url    | ~      | SELF_URL    | http://localhost:3000  |
| meta.name    | string | ~      | APP_NAME    | gruber-app             |
| meta.version | string | ~      | APP_VERSION | 1.2.3                  |

Default:
{
  "env": "development",
  "meta": {
    "name": "gruber-app",
    "version": "1.2.3"
  },
  "server": {
    "port":
  },
}

Running this from the CLI is very useful to see what is going on, if you pass the second parameter to getConfigurationUsage it will show you how it is configured too. The default value is also useful for initially creating your configuration.

From this you can see that you can set the port by either specifying the PORT environment variable, using the --port CLI flag or setting server.port field in the configuration file.

Considerations

You should consider the security for your default values, e.g. if you app runs differently under NODE_ENV=production and you forget to set it, what is the implication?

If you use something like dotenv, ensure it has already loaded before creating the configuration.

You can add extra checks to loadConfiguration to ensure things are correct in production, this can be done like so:

export function loadConfiguration(path) {
  const appConfig = config.loadJsonSync(path, struct);

  // Only run these checks when running in production
  if (appConfig.env !== "development") {
    if (appConfig.database.url.href.includes("top_secret")) {
      throw new Error("database.url has not been configured");
    }
    // more checks ...
  }

  return appConfig;
}

This checks the default value for database.url is not used when in production mode.

A Pattern I follow is to assume the app is in development mode, then when I build the app as a container set the NODE_ENV variable to production. When running as a container it assumes it is in production and runs the extra checks.

Configuration

ConfigurationOptions `type`

Options for creating a platform-sepcific Configuration object, different methods provide abstractions over the filesystem & parsing capabilities of the Configuration.

For instance, you could create one that loads remote files over S3 and parses them as YAML, or just a simple one that loads JSON files from the filesystem

Configuration

Configuration is both an abstraction around processing config files, environment variables & CLI flags from the platform and also a tool for users to declaratively define how their configuration is.

Each platform specifies a default options to load JSON files, but you can also construct your own if you want to customise how it works.

With an instance, you can then define how an app's config can be specified as either configuration files, CLI flag, environment variables or a combination of any of them.

const config = new Configuration({
	readTextFile(url) {},
	getEnvironmentVariable(key) {},
	getCommandArgument() {},
	stringify(value) {},
	parse(value) {},
})

array `unstable`

Create an ordered list of another type

config.array(
	Structure.string()
)

boolean

Define a boolean value with options to load from the config-file, an environment variable or a CLI flag. The only required field is fallback

There are extra coercions for boolean-like strings

1, true & yes coerce to true
0, false & no coerce to false

config.boolean({
	variable: "USE_SSL",
	flag: "--ssl",
	fallback: false
})

external `unstable`

Load another configuration file or use value in the original configuration

config.external(
	new URL("./api-keys.json", import.meta.url),
	config.object({
		keys: Structure.array(Structure.string())
	})
)

Which will attempt to load "api-keys.json" and parse that, and if that doesn't exist it will also try the value in the original configuration.

getJSONSchema `unstable`

Given a structure defined using configuration, generate a JSON Schema to validate it. This could be useful to write to a file then use a IDE-based validator using something like

{
	"$schema": "./app-config.schema.json",
}

getUsage

Given a structure defined using Configuration, generate human-readable usage information. The usage includes a table of all configuration options and what the default value would be if no other soruces are used.

Optionally, output the current value of the configuration too.

load

Load configuration with a base file, also pulling in environment variables and CLI flags using ConfigurationOptions

const struct = config.object({
	env: config.string({ variable: "NODE_ENV", fallback: "development" })
})

config.load(
	new URL("./app-config.json", import.meta.url),
	struct
)

It will asynchronously load the configuration, validate it and return the coerced value. If it fails it will output a friendly string listing what is wrong and throw the Structure.Error

number

Define a numeric value with options to load from the config-file, an environment variable or a CLI flag. The only required field is fallback

It will also coerce floating point numbers from strings

config.number({
	variable: "PORT",
	flag: "--port",
	fallback: "1234"
})

object

Group or nest configuration in an object.

config.object({
	name: config.string({ fallback: "Geoff Testington" }),
	age: config.number({ fallback: 42 }),
})

string

Define a string-based value with options to load from the config-file, an environment variable or a CLI flag. The only required field is fallback

config.string({
	variable: "HOSTNAME",
	flag: "--host",
	fallback: "localhost"
})

url

Define a URL based value, the value is validated and converted into a URL.

config.url({
	variable: "SELF_URL",
	flag: "--url",
	fallback: "http://localhost:1234"
})

Structure

Structure.Error

An error produced from processing a value for a Structure

const error = new Structure.Error("Expected something", ["some", "path"])

It takes a message, path & children in the constructor.

You can also iterate over a Structure.Error to walk the tree of errors.

for (const error2 of error) {
  console.log(error2.getOneLiner())
}

getOneLiner

Get a single-line variant, describing the error

error.getOneLiner()

which outputs something like:

some.path — expected a number

getStandardSchemaIssues

Convert the error to a StandardSchema issue to be used with that ecosystem.

toFriendlyString

Generate a human-friendly string describing the error and all nested errors

error.toFriendlyString()

which outputs something like:

Object does not match schema
  name — expected a string
  age — expected a number

_StructError.chain `internal`

Create a new error with the context added to it

const nested = Structure.Error.chain(
	new Error("Something went wrong"),
	["some", "path"]
)

Structure

Structure is a composable primative for processing values to make sure they are what you expect them to be, optionally coercing the value into something else. It's also strongly-typed so values that are validated have the correct TypeScript type too.

The Structure class also supports StandardSchema v1 so you can use it anywhere that supports that standard.

getFullSchema

Get a JSON schema from the structure, where equivalent fields are available.

process

Execute the structure by passing it a value and getting back the result if it is successful, otherwise a Structure.Error is thrown

Structure.any

Define a Structure that lets any value through

Structure.any()

Structure.array

Define a list of values that each match the same structure.

// An array of strings
Structure.array(
	Structure.string()
)

// An array of objects
Structure.array(
	Structure.object({
		name: Structure.string(),
		age: Structure.number()
	})
)

Structure.boolean

Define a boolean value with an optional fallback.

Structure.boolean()
Structure.boolean(false)

Structure.date

Creates a Structure that validates dates or values that can be turned into dates through the Date constructor.

Structure.date()

Structure.literal

Define a specific value that must be exactly equal.

Structure.literal("click_event")
Structure.literal(42)
Structure.literal(true)

Structure.null

Define a Structure to validate the value is null

Structure.null()

Structure.number

Define a number-based value with an optional fallback, it will also try to parse floating-point values from strings.

Structure.number()
Structure.number("Geoff Testington")

Structure.object

Define a group of structures under an object. Each field needs to matched their respective Structures and no additionaly fields are allowed.

Structure.object({
	name: Structure.string(),
	age: Structure.number(),
})

Structure.partial

Create a Structure that validates an object where some or none of the fields match their respective Structures. Only fields specified may be set, nothing additional.

Structure.partial({
	name: Structure.string(),
	age: Structure.number()
})

Structure.string

Define a string-based value with an optional fallback.

Structure.string()
Structure.string("Geoff Testington")

Structure.union

Define a Structure that must match one of a set of Structures

Structure.union([
	Structure.object({
		type: Structure.literal("click"),
		element: Structure.string()
	}),
	Structure.object({
		type: Structure.literal("login"),
	}),
])

Structure.url

Define a URL value with an optional fallback, that will be coerced into a URL.

Structure.url()
Structure.url("http://example.com")
Structure.url(new URL("http://example.com"))

debug

{
  "ConfigurationOptions": {
    "entrypoint": "config/mod.ts",
    "id": "ConfigurationOptions",
    "name": "ConfigurationOptions",
    "content": "Options for creating a platform-sepcific [Configuration](#configuration) object,\ndifferent methods provide abstractions over the filesystem & parsing capabilities of the Configuration.\n\nFor instance, you could create one that loads remote files over S3 and parses them as YAML,\nor just a simple one that loads JSON files from the filesystem",
    "tags": {
      "group": "Configuration",
      "type": "true"
    },
    "children": {}
  },
  "Configuration": {
    "entrypoint": "config/mod.ts",
    "id": "Configuration",
    "name": "Configuration",
    "content": "**Configuration** is both an abstraction around processing config files,\nenvironment variables & CLI flags from the platform\nand also a tool for users to declaratively define how their configuration is.\n\nEach platform specifies a default `options` to load JSON files,\nbut you can also construct your own if you want to customise how it works.\n\nWith an instance, you can then define how an app's config can be specified as either configuration files,\nCLI flag, environment variables or a combination of any of them.\n\n```js\nconst config = new Configuration({\n\treadTextFile(url) {},\n\tgetEnvironmentVariable(key) {},\n\tgetCommandArgument() {},\n\tstringify(value) {},\n\tparse(value) {},\n})\n```",
    "tags": {
      "group": "Configuration"
    },
    "children": {
      "object": {
        "id": "Configuration#object",
        "name": "object",
        "content": "Group or nest configuration in an object.\n\n```js\nconfig.object({\n\tname: config.string({ fallback: \"Geoff Testington\" }),\n\tage: config.number({ fallback: 42 }),\n})\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "array": {
        "id": "Configuration#array",
        "name": "array",
        "content": "Create an ordered list of another type\n\n```js\nconfig.array(\n\tStructure.string()\n)\n```",
        "tags": {
          "unstable": "true",
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "external": {
        "id": "Configuration#external",
        "name": "external",
        "content": "Load another configuration file or use value in the original configuration\n\n```js\nconfig.external(\n\tnew URL(\"./api-keys.json\", import.meta.url),\n\tconfig.object({\n\t\tkeys: Structure.array(Structure.string())\n\t})\n)\n```\n\nWhich will attempt to load \"api-keys.json\" and parse that,\nand if that doesn't exist it will also try the value in the original configuration.",
        "tags": {
          "unstable": "true",
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "string": {
        "id": "Configuration#string",
        "name": "string",
        "content": "Define a string-based value with options to load from the config-file,\nan environment variable or a CLI flag.\nThe only required field is **fallback**\n\n```js\nconfig.string({\n\tvariable: \"HOSTNAME\",\n\tflag: \"--host\",\n\tfallback: \"localhost\"\n})\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "number": {
        "id": "Configuration#number",
        "name": "number",
        "content": "Define a numeric value with options to load from the config-file,\nan environment variable or a CLI flag.\nThe only required field is **fallback**\n\nIt will also coerce floating point numbers from strings\n\n```js\nconfig.number({\n\tvariable: \"PORT\",\n\tflag: \"--port\",\n\tfallback: \"1234\"\n})\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "boolean": {
        "id": "Configuration#boolean",
        "name": "boolean",
        "content": "Define a boolean value with options to load from the config-file,\nan environment variable or a CLI flag.\nThe only required field is **fallback**\n\nThere are extra coercions for boolean-like strings\n\n- `1`, `true` & `yes` coerce to true\n- `0`, `false` & `no` coerce to false\n\n```js\nconfig.boolean({\n\tvariable: \"USE_SSL\",\n\tflag: \"--ssl\",\n\tfallback: false\n})\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "url": {
        "id": "Configuration#url",
        "name": "url",
        "content": "Define a URL based value, the value is validated and converted into a [URL](https://developer.mozilla.org/en-US/docs/Web/API/URL).\n\n```js\nconfig.url({\n\tvariable: \"SELF_URL\",\n\tflag: \"--url\",\n\tfallback: \"http://localhost:1234\"\n})\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "load": {
        "id": "Configuration#load",
        "name": "load",
        "content": "Load configuration with a base file, also pulling in environment variables and CLI flags using [ConfigurationOptions](#configurationoptions)\n\n```js\nconst struct = config.object({\n\tenv: config.string({ variable: \"NODE_ENV\", fallback: \"development\" })\n})\n\nconfig.load(\n\tnew URL(\"./app-config.json\", import.meta.url),\n\tstruct\n)\n```\n\nIt will asynchronously load the configuration, validate it and return the coerced value.\nIf it fails it will output a friendly string listing what is wrong and throw the Structure.Error",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "getUsage": {
        "id": "Configuration#getUsage",
        "name": "getUsage",
        "content": "Given a structure defined using Configuration, generate human-readable usage information.\nThe usage includes a table of all configuration options and what the default value would be if no other soruces are used.\n\nOptionally, output the current value of the configuration too.",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "getJSONSchema": {
        "id": "Configuration#getJSONSchema",
        "name": "getJSONSchema",
        "content": "Given a structure defined using configuration, generate a JSON Schema to validate it. This could be useful to write to a file then use a IDE-based validator using something like\n\n```json\n{\n\t\"$schema\": \"./app-config.schema.json\",\n}\n```",
        "tags": {
          "unstable": "true",
          "group": "Miscellaneous"
        },
        "children": {}
      }
    }
  },
  "Structure.Error": {
    "entrypoint": "config/mod.ts",
    "id": "_StructError",
    "name": "Structure.Error",
    "content": "An error produced from processing a value for a [Structure](#structure)\n\n```js\nconst error = new Structure.Error(\"Expected something\", [\"some\", \"path\"])\n```\n\nIt takes a `message`, `path` & `children` in the constructor.\n\nYou can also iterate over a Structure.Error to walk the tree of errors.\n\n```js\nfor (const error2 of error) {\n  console.log(error2.getOneLiner())\n}\n```",
    "tags": {
      "name": "Structure.Error",
      "group": "Structure"
    },
    "children": {
      "getOneLiner": {
        "id": "_StructError#getOneLiner",
        "name": "getOneLiner",
        "content": "Get a single-line variant, describing the error\n\n```js\nerror.getOneLiner()\n```\n\nwhich outputs something like:\n\n```\nsome.path — expected a number\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "toFriendlyString": {
        "id": "_StructError#toFriendlyString",
        "name": "toFriendlyString",
        "content": "Generate a human-friendly string describing the error and all nested errors\n\n```js\nerror.toFriendlyString()\n```\n\nwhich outputs something like:\n\n```\nObject does not match schema\n  name — expected a string\n  age — expected a number\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "getStandardSchemaIssues": {
        "id": "_StructError#getStandardSchemaIssues",
        "name": "getStandardSchemaIssues",
        "content": "Convert the error to a [StandardSchema](https://standardschema.dev/) issue to be used with that ecosystem.",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "_StructError.chain": {
        "id": "_StructError.chain",
        "name": "chain",
        "content": "Create a new error with the context added to it\n\n```js\nconst nested = Structure.Error.chain(\n\tnew Error(\"Something went wrong\"),\n\t[\"some\", \"path\"]\n)\n```",
        "tags": {
          "internal": "true",
          "group": "Miscellaneous"
        },
        "children": {}
      }
    }
  },
  "Structure": {
    "entrypoint": "config/mod.ts",
    "id": "Structure",
    "name": "Structure",
    "content": "**Structure** is a composable primative for processing values to make sure they are what you expect them to be, optionally coercing the value into something else. It's also strongly-typed so values that are validated have the correct TypeScript type too.\n\nThe Structure class also supports [StandardSchema v1](https://standardschema.dev) so you can use it anywhere that supports that standard.",
    "tags": {
      "group": "Structure"
    },
    "children": {
      "process": {
        "id": "Structure#process",
        "name": "process",
        "content": "Execute the structure by passing it a value and getting back the result if it is successful, otherwise a [Structure.Error](#structure-error) is thrown",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "getFullSchema": {
        "id": "Structure#getFullSchema",
        "name": "getFullSchema",
        "content": "Get a JSON schema from the structure, where equivalent fields are available.",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.string": {
        "id": "Structure.string",
        "name": "string",
        "content": "Define a string-based value with an optional `fallback`.\n\n```js\nStructure.string()\nStructure.string(\"Geoff Testington\")\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.number": {
        "id": "Structure.number",
        "name": "number",
        "content": "Define a number-based value with an optional `fallback`,\nit will also try to parse floating-point values from strings.\n\n```js\nStructure.number()\nStructure.number(\"Geoff Testington\")\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.boolean": {
        "id": "Structure.boolean",
        "name": "boolean",
        "content": "Define a boolean value with an optional `fallback`.\n\n```js\nStructure.boolean()\nStructure.boolean(false)\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.url": {
        "id": "Structure.url",
        "name": "url",
        "content": "Define a URL value with an optional `fallback`,\nthat will be coerced into a `URL`.\n\n```js\nStructure.url()\nStructure.url(\"http://example.com\")\nStructure.url(new URL(\"http://example.com\"))\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.object": {
        "id": "Structure.object",
        "name": "object",
        "content": "Define a group of structures under an object.\nEach field needs to matched their respective Structures and no additionaly fields are allowed.\n\n```js\nStructure.object({\n\tname: Structure.string(),\n\tage: Structure.number(),\n})\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.array": {
        "id": "Structure.array",
        "name": "array",
        "content": "Define a list of values that each match the same structure.\n\n```js\n// An array of strings\nStructure.array(\n\tStructure.string()\n)\n\n// An array of objects\nStructure.array(\n\tStructure.object({\n\t\tname: Structure.string(),\n\t\tage: Structure.number()\n\t})\n)\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.literal": {
        "id": "Structure.literal",
        "name": "literal",
        "content": "Define a specific value that must be exactly equal.\n\n\n```js\nStructure.literal(\"click_event\")\nStructure.literal(42)\nStructure.literal(true)\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.union": {
        "id": "Structure.union",
        "name": "union",
        "content": "Define a Structure that must match one of a set of Structures\n\n```js\nStructure.union([\n\tStructure.object({\n\t\ttype: Structure.literal(\"click\"),\n\t\telement: Structure.string()\n\t}),\n\tStructure.object({\n\t\ttype: Structure.literal(\"login\"),\n\t}),\n])\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.null": {
        "id": "Structure.null",
        "name": "null",
        "content": "Define a Structure to validate the value is `null`\n\n```js\nStructure.null()\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.any": {
        "id": "Structure.any",
        "name": "any",
        "content": "Define a Structure that lets any value through\n\n```js\nStructure.any()\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.partial": {
        "id": "Structure.partial",
        "name": "partial",
        "content": "Create a Structure that validates an object where some or none of the fields match their respective Structures.\nOnly fields specified may be set, nothing additional.\n\n```js\nStructure.partial({\n\tname: Structure.string(),\n\tage: Structure.number()\n})\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      },
      "Structure.date": {
        "id": "Structure.date",
        "name": "date",
        "content": "Creates a Structure that validates dates or values that can be turned into dates\nthrough the [Date constructor](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/Date).\n\n```js\nStructure.date()\n```",
        "tags": {
          "group": "Miscellaneous"
        },
        "children": {}
      }
    }
  }
}

Configuration

Considerations

Configuration

ConfigurationOptions type

Configuration

array unstable

boolean

external unstable

getJSONSchema unstable

getUsage

load

number

object

string

url

Structure

Structure.Error

getOneLiner

getStandardSchemaIssues

toFriendlyString

_StructError.chain internal

Structure

getFullSchema

process

Structure.any

Structure.array

Structure.boolean

Structure.date

Structure.literal

Structure.null

Structure.number

Structure.object

Structure.partial

Structure.string

Structure.union

Structure.url

ConfigurationOptions `type`

array `unstable`

external `unstable`

getJSONSchema `unstable`

_StructError.chain `internal`