Host.KV (Beamlet v0.1.0)

Copy Markdown View Source

Durable key/value storage for small state under string keys.

A cursor, a counter, a last-run time, a preference. Values are JSON: nil, booleans, numbers, strings, lists, and maps with string keys, and what you put is what you get back. Anything else raises on put/2, so store an atom as a string ("active" rather than :active), a map with string keys (%{"count" => 1} rather than %{count: 1}) and a time as ISO 8601 (DateTime.to_iso8601(now)).

Keys are one shared namespace across every agent and module on your beamlet, so prefix yours with your domain, e.g. "poller:last_id". The store lives in the agent database beside your own tables, so a write here inside a Host.Repo.transaction lands together with your rows:

def create(conn, %{"id" => id} = event) do
  Host.Repo.transaction(fn ->
    Host.Repo.insert!(Events.Event.changeset(%Events.Event{}, event))
    Host.KV.put("events:last_id", id)
  end)

  json(conn, %{ok: true})
end

A value worked out from the one before it, a counter or a list of recent ids, changes through update/3, since a get/2 then a put/2 loses updates under concurrent requests. Anything you would filter, sort or join on belongs in a table of its own, through a migration (Host.Migrator) and an Ecto.Schema. Reads return the value or a default; fetch/1 is the one that tells a stored nil from a missing key.

Summary

Types

A value the store holds: JSON's shapes, with maps keyed by strings.

Functions

Returns every entry whose key starts with prefix as a map of key to value, loaded in one query, e.g. all("poller:").

Removes key.

Removes every key starting with prefix in one query, e.g. delete_all("poller:").

Returns {:ok, value} for the value stored under key, or :error when there is none.

Returns the value stored under key, or default when there is none, e.g. get("poller:last_id", 0).

Returns every key starting with prefix, sorted, without loading the values, e.g. keys("poller:").

Stores value under key, replacing any existing value, e.g. put("poller:last_id", 42).

Updates the value under key with fun and returns the new value, storing default as it is when the key is missing, e.g. update("api:hits", 1, &(&1 + 1)).

Types

value()

@type value() ::
  nil
  | boolean()
  | number()
  | String.t()
  | [value()]
  | %{optional(String.t()) => value()}

A value the store holds: JSON's shapes, with maps keyed by strings.

Functions

all(prefix \\ "")

@spec all(String.t()) :: %{required(String.t()) => value()}

Returns every entry whose key starts with prefix as a map of key to value, loaded in one query, e.g. all("poller:").

Use this rather than get/2 in a loop; keys/1 lists the keys alone when the values are not needed. An empty prefix returns everything.

delete(key)

@spec delete(String.t()) :: :ok

Removes key.

Removing a key that is not there is a no-op.

delete_all(prefix)

@spec delete_all(String.t()) :: :ok

Removes every key starting with prefix in one query, e.g. delete_all("poller:").

An empty prefix removes every key.

fetch(key)

@spec fetch(String.t()) :: {:ok, value()} | :error

Returns {:ok, value} for the value stored under key, or :error when there is none.

The only read that distinguishes a stored nil from a missing key.

get(key, default \\ nil)

@spec get(String.t(), term()) :: value() | term()

Returns the value stored under key, or default when there is none, e.g. get("poller:last_id", 0).

keys(prefix \\ "")

@spec keys(String.t()) :: [String.t()]

Returns every key starting with prefix, sorted, without loading the values, e.g. keys("poller:").

all/1 returns keys and values together. An empty prefix lists every key.

put(key, value)

@spec put(String.t(), value()) :: :ok

Stores value under key, replacing any existing value, e.g. put("poller:last_id", 42).

A value that is not JSON-shaped (value/0) raises ArgumentError naming where in the value the problem sits, and nothing is written.

update(key, default, fun)

@spec update(String.t(), value(), (value() -> value())) :: value()

Updates the value under key with fun and returns the new value, storing default as it is when the key is missing, e.g. update("api:hits", 1, &(&1 + 1)).

No update is lost under concurrent requests: if another write lands on key while fun runs, fun runs again on the newer value. So keep fun fast and pure, computing the new value from the old one alone, and do I/O and other side effects before or after the call.

Host.KV.update("webhooks:recent", [id], fn ids -> Enum.take([id | ids], 20) end)

A value that is not JSON-shaped raises as put/2 does. If key is still changing under fun after a second of retries, the update raises: a sign that fun is too slow for how often key is written.