Zapatos:
Zero-Abstraction Postgres for TypeScript

Postgres and TypeScript are each, individually, fabulous.

Zapatos aims to make them work beautifully together. No abstractions, no distractions: just your database, with type safety.

What does it do?

To achieve this aim, Zapatos does these five things:

• Typescript schema   A command-line tool speaks to your Postgres database and writes up a detailed TypeScript schema for every table. This is just a means to an end: it enables the next three things in this list. Show me »

• Arbitrary SQL   Simple building blocks help you write arbitrary SQL using tagged templates, and manually apply the right types to what goes in and what comes back. Show me »

• Everyday CRUD   Shortcut functions produce everyday CRUD queries with no fuss and no surprises, fully and automatically typed. Show me »

• JOINs as nested JSON   Nested shortcut calls generate LATERAL JOIN queries, resulting in arbitrarily complex nested JSON structures, still fully and automatically typed. Show me »

• Transactions   Transaction helper functions assist in managing and retrying transactions. Show me »

How does that look?

Typescript schema

A command-line tool speaks to your Postgres database and writes up a detailed TypeScript schema for every table.

Take this ultra-simple SQL schema for a single table, authors:

CREATE TABLE "authors" 
( "id" SERIAL PRIMARY KEY
, "name" TEXT NOT NULL
, "isLiving" BOOLEAN );

We run npx zapatos to generate a file named schema.d.ts, including table definitions like this one:

export namespace authors {
  export type Table = 'authors';
  export interface Selectable {
    id: number;
    name: string;
    isLiving: boolean | null;
  }
  export interface Whereable {
    id?: number | db.Parameter<number> | db.SQLFragment /* | ... etc ... */;
    name?: string | db.Parameter<string> | db.SQLFragment /* | ... etc ... */;
    isLiving?: boolean | db.Parameter<boolean> | db.SQLFragment /* | ... etc ... */;
  }
  export interface Insertable {
    id?: number | db.Parameter<number> | db.DefaultType | db.SQLFragment;
    name: string | db.Parameter<string> | db.SQLFragment;
    isLiving?: boolean | db.Parameter<boolean> | null | db.DefaultType | db.SQLFragment;
  }
  export interface Updatable {
    id?: number | db.Parameter<number> | db.DefaultType | db.SQLFragment /* | ... etc ... */;
    name?: string | db.Parameter<string> | db.SQLFragment /* | ... etc ... */;
    isLiving?: boolean | db.Parameter<boolean> | null | db.DefaultType | db.SQLFragment /* | ... etc ... */;
  }
  /* ... etc ... */
}

The type names are, I hope, reasonably self-explanatory. authors.Selectable is what I’ll get back from a SELECT query on this table. authors.Whereable is what I can use in a WHERE condition: everything’s optional, and I can include arbitrary SQL. authors.Insertable is what I can INSERT: it’s similar to the Selectable, but any fields that are NULLable and/or have DEFAULT values are allowed to be missing, NULL or DEFAULT. authors.Updatable is what I can UPDATE the table with: like what I can INSERT, but all columns are optional: it’s (roughly) a Partial<authors.Insertable>.

schema.d.ts includes some other types that get used internally, including handy type mappings like this one:

export type SelectableForTable<T extends Table> = {
  authors: authors.Selectable;
  books: books.Selectable;
  tags: tags.Selectable;
  /* ... */
}[T];

Zapatos supports tables, foreign tables, views and materialized views. It understands enumerated types: CREATE TYPE "size" AS ENUM ('big', 'small'); comes to TypeScript as 'big' | 'small'. And it lets you define the TypeScript treatment of domain types and user-defined types too.

Tell me more about the command line tool »

Arbitrary SQL

Simple building blocks help you write arbitrary SQL using tagged templates, and manually apply the right types to what goes in and what comes back.

Let’s insert something into that authors table for which we just generated the types. We’ll write the SQL query ourselves, to show how that works (though we’ll see an easier way in the next section):

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const
  author: s.authors.Insertable = {
    name: 'Gabriel Garcia Marquez',
    isLiving: false,
  },
  [insertedAuthor] = await db.sql<s.authors.SQL, s.authors.Selectable[]>`
      INSERT INTO ${"authors"} (${db.cols(author)})
      VALUES (${db.vals(author)}) RETURNING *`
    .run(pool);

INSERT INTO "authors" ("isLiving", "name")
  VALUES ($1, $2)
RETURNING *

[false, "Gabriel Garcia Marquez"]

[
  {
    "id": 1,
    "name": "Gabriel Garcia Marquez",
    "isLiving": false
  }
]

We apply the appropriate type to the object we’re trying to insert (s.authors.Insertable), giving us type-checking and autocompletion on that object. And we specify both which types are allowed as interpolated values in the template string (s.authors.SQL) and what type is going to be returned (s.authors.Selectable[]) when the query runs.

We also use the cols and vals helper functions. These compile, respectively, to the object’s keys (which are the column names) and query placeholders ($1, $2, …) for the corresponding values.

You can click ‘Explore types’ above to open the code in an embedded Monaco (VS Code) editor, so you can check those typings for yourself.

Tell me more about writing arbitrary SQL »

Everyday CRUD

Shortcut functions produce everyday CRUD queries with no fuss and no surprises, fully and automatically typed.

So — writing SQL with Zapatos is nicer than constructing a query and all its input and output types from scratch. But for a totally bog-standard CRUD query like the INSERT above, it still involves quite a lot of boilerplate.

To eliminate the boilerplate, Zapatos supplies some simple functions to generate these sorts of queries, fully and automatically typed.

Let’s use one of them — insert — to add two more authors:

import * as db from 'zapatos/db';
import pool from './pgPool';const [doug, janey] = await db.insert('authors', [
  { name: 'Douglas Adams', isLiving: false },
  { name: 'Jane Austen', isLiving: false },
]).run(pool);

INSERT INTO "authors" ("isLiving", "name")
  VALUES ($1, $2), ($3, $4)
RETURNING to_jsonb ("authors".*) AS result

[false, "Douglas Adams", false, "Jane Austen"]

[
  {
    "id": 2,
    "name": "Douglas Adams",
    "isLiving": false
  },
  {
    "id": 3,
    "name": "Jane Austen",
    "isLiving": false
  }
]

The insert shortcut accepts a single Insertable or an Insertable[] array, and correspondingly returns a single JSONSelectable or a JSONSelectable[] array. Since we specified 'authors' as the first argument here, and an array as the second, input and output will be checked and auto-completed as authors.Insertable[] and authors.JSONSelectable[] respectively.

Again, click ‘Explore types’ to play around and check those typings.

In addition to insert, there are shortcuts for select (plus selectOne, selectExactlyOne, and simple aggregates such as count and sum), and for update, upsert, delete and truncate.

Tell me more about the shortcut functions »

JOINs as nested JSON

Nested shortcut calls generate LATERAL JOIN queries, resulting in arbitrarily complex nested JSON structures, still fully and automatically typed.

CRUD is our bread and butter, but the power of SQL is in the JOINs. Postgres has powerful JSON features than can deliver sensibly-structured JOIN results with minimal post-processing: json_agg, json_build_object, and so on. Zapatos builds on these.

To demonstrate, let’s say that authors have books and books have tags, adding two new tables to our simple schema:

CREATE TABLE "books" 
( "id" SERIAL PRIMARY KEY
, "authorId" INTEGER NOT NULL REFERENCES "authors"("id")
, "title" TEXT
, "createdAt" TIMESTAMPTZ NOT NULL DEFAULT now() );

CREATE TABLE "tags"
( "tag" TEXT NOT NULL
, "bookId" INTEGER NOT NULL REFERENCES "books"("id") ON DELETE CASCADE );

CREATE UNIQUE INDEX "tagsUniqueIdx" ON "tags"("tag", "bookId");

And let’s say I want to show a list of books, each with its (one) author and (many) associated tags. We could knock up a manual query for this, of course, but it gets quite hairy. The select shortcut has an option called lateral that can nest other select queries and do it for us.

Let’s try it:

import * as db from 'zapatos/db';
import pool from './pgPool';const bookAuthorTags = await db.select('books', db.all, {
  lateral: {
    author: db.selectExactlyOne('authors', { id: db.parent('authorId') }),
    tags: db.select('tags', { bookId: db.parent('id') }),
  }
}).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("books".*) || jsonb_build_object($1::text, "lateral_author".result, $2::text, "lateral_tags".result) AS result
  FROM "books"
  LEFT JOIN LATERAL (
    SELECT to_jsonb ("authors".*) AS result
    FROM "authors"
    WHERE ("id" = "books"."authorId")
  LIMIT $3) AS "lateral_author" ON true
  LEFT JOIN LATERAL (
    SELECT coalesce(jsonb_agg(result), '[]') AS result
    FROM (
      SELECT to_jsonb ("tags".*) AS result
      FROM "tags"
      WHERE ("bookId" = "books"."id")) AS "sq_tags") AS "lateral_tags" ON true) AS "sq_books"

["author", "tags", 1]

[
  {
    "id": 1000,
    "tags": [
      {
        "tag": "His Dark Materials",
        "bookId": 1000
      },
      {
        "tag": "1/3",
        "bookId": 1000
      }
    ],
    "title": "Northern Lights",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    },
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603082+01:00"
  },
  {
    "id": 1001,
    "tags": [
      {
        "tag": "His Dark Materials",
        "bookId": 1001
      },
      {
        "tag": "2/3",
        "bookId": 1001
      }
    ],
    "title": "The Subtle Knife",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    },
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603558+01:00"
  },
  {
    "id": 1002,
    "tags": [
      {
        "tag": "His Dark Materials",
        "bookId": 1002
      },
      {
        "tag": "3/3",
        "bookId": 1002
      }
    ],
    "title": "The Amber Spyglass",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    },
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603732+01:00"
  },
  {
    "id": 1003,
    "tags": [
      {
        "tag": "mystery",
        "bookId": 1003
      }
    ],
    "title": "The Curious Incident of the Dog in the Night-Time",
    "author": {
      "id": 1001,
      "name": "Mark Haddon",
      "isLiving": true
    },
    "authorId": 1001,
    "createdAt": "2024-06-23T15:22:52.605245+01:00"
  },
  {
    "id": 1004,
    "tags": [
      {
        "tag": "adventure",
        "bookId": 1004
      }
    ],
    "title": "Holes",
    "author": {
      "id": 1002,
      "name": "Louis Sachar",
      "isLiving": true
    },
    "authorId": 1002,
    "createdAt": "2024-06-23T15:22:52.605559+01:00"
  }
]

This generates an efficient three-table LATERAL JOIN that returns a nested JSON structure directly from the database. Every nested element is again fully and automatically typed.

Again, you can click ‘Explore types’ above to open the code in an embedded Monaco (VS Code) editor, so you can check those typings for yourself.

We can of course extend this to deeper nesting (e.g. query each author, with their books, with their tags); to self-joins (of a table with itself, e.g. employees to their managers in the same employees table); and to joins on relationships other than foreign keys (e.g. joining the nearest N somethings using the PostGIS <-> distance operator).

Tell me more about nested select queries »

Transactions

Transaction helper functions assist in managing and retrying transactions.

Transactions are where I’ve found traditional ORMs like TypeORM and Sequelize most footgun-prone. Zapatos is always explicit about what client or pool is running your query — hence that pool argument in all our examples so far.

Zapatos also offers simple transaction helpers that handle issuing a SQL ROLLBACK on error, releasing the database client in a finally clause, and automatically retrying queries in case of serialization failures. There’s one for each isolation level (SERIALIZABLE, REPEATABLE READ, and so on), and they look like this:

import * as db from 'zapatos/db';
import pool from './pgPool';const result = await db.serializable(pool, async txnClient => {
  /* queries here use txnClient instead of pool */
});

For instance, take this bankAccounts table:

CREATE TABLE "bankAccounts" 
( "id" SERIAL PRIMARY KEY
, "balance" INTEGER NOT NULL DEFAULT 0 CHECK ("balance" >= 0) );

We can use the transaction helpers like so:

import * as db from 'zapatos/db';
import pool from './pgPool';const [accountA, accountB] = await db.insert('bankAccounts', 
  [{ balance: 50 }, { balance: 50 }]).run(pool);

const transferMoney = (sendingAccountId: number, receivingAccountId: number, amount: number) =>
  db.serializable(pool, txnClient => Promise.all([
    db.update('bankAccounts',
      { balance: db.sql`${db.self} - ${db.param(amount)}` },
      { id: sendingAccountId }).run(txnClient),
    db.update('bankAccounts',
      { balance: db.sql`${db.self} + ${db.param(amount)}` },
      { id: receivingAccountId }).run(txnClient),
  ]));

try {
  const [[updatedAccountA], [updatedAccountB]] = await transferMoney(accountA.id, accountB.id, 60);
} catch(err: any) {
  console.log(err.message, '/', err.detail);
}

INSERT INTO "bankAccounts" ("balance")
  VALUES ($1), ($2)
RETURNING to_jsonb ("bankAccounts".*) AS result

[50, 50]

[
  {
    "id": 1,
    "balance": 50
  },
  {
    "id": 2,
    "balance": 50
  }
]

Transaction 0

START TRANSACTION ISOLATION LEVEL SERIALIZABLE

Transaction 0

UPDATE
  "bankAccounts"
SET ("balance") = ROW ("balance" - $1)
WHERE ("id" = $2)
RETURNING to_jsonb ("bankAccounts".*) AS result

[60, 1]

Transaction 0

UPDATE
  "bankAccounts"
SET ("balance") = ROW ("balance" + $1)
WHERE ("id" = $2)
RETURNING to_jsonb ("bankAccounts".*) AS result

[60, 2]

Transaction 0

ROLLBACK

new row for relation "bankAccounts" violates check constraint "bankAccounts_balance_check" / Failing row contains (1, -10).

Finally, Zapatos provides a set of hierarchical isolation types so that, for example, if you type a txnClient argument to a function as TxnClientForRepeatableRead, you can call it with IsolationLevel.Serializable or IsolationLevel.RepeatableRead but not IsolationLevel.ReadCommitted.

Tell me more about the transaction functions »

Why does it do those things?

It is a truth universally acknowledged that ORMs aren’t very good. JavaScript and TypeScript ORMs are perhaps even worse than the average. One Zapatos user described a popular TypeScript ORM as “full of broken magic under the hood”, which nicely captures what originally motivated me to write this library.

I like SQL, and Postgres especially. In my experience, abstractions that obscure the underlying SQL, or that prioritise ease of switching to another database tomorrow over effective use of this database today, are a source of misery.

I’ve also come to love strongly typed languages, and TypeScript in particular. VS Code’s type checking and autocomplete speed development, prevent bugs, and simplify refactoring. Especially when they just happen, they bring joy. But, traditionally, talking to the database is a place where they really don’t just happen.

Zapatos aims to fix that.

If it interests you, there’s a whole other repository about how Zapatos came about.

What doesn’t it do?

Zapatos doesn’t handle schema migrations. Other tools can help you with this: check out dbmate, for instance.

It also doesn’t manage the connection pool for you, as some ORMs do — mainly because the pg module makes this so easy. For example, my pgPool.ts looks something like this:

import pg from 'pg';

const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL });
pool.on('error', err => console.error(err));  // don't let a pg restart kill your app

export default pool;

Finally, it won’t tell you how to structure your code: Zapatos doesn’t deal in the ‘model’ classes beloved of traditional ORMs, just (fully-typed) POJOs.

How do I get it?

Install it

First: check your tsconfig.json. You need "strictNullChecks": true or "strict": true (which implies "strictNullChecks": true). Without strictNullChecks, some things just won’t work — namely, the lateral, extras, returning and columns options to the shortcut functions.

Since TypeScript 4.4, it’s also a good idea to set "exactOptionalPropertyTypes": true.

Then install Zapatos with npm:

npm install --save zapatos

Configure it

Add a top-level file zapatosconfig.json to your project. Here’s an example:

{
  "db": {
    "connectionString": "postgresql://localhost/example_db"
  },
  "outDir": "./src"
}

These are available top-level keys, all of which are optional:

• "db" gives Postgres connection details. You can provide anything that you’d pass to new pg.Pool(/* ... */) here.

• "outDir" defines where your zapatos folder will be created, relative to the project root. If not specified, it defaults to the project root, i.e. ".".

• "outExt" defines the file extension for all generated type files. It defaults to ".d.ts", but for certain use cases you may wish to set it to ".ts".

• "progressListener" is a boolean that determines how chatty the tool is. If true, it enumerates its progress in generating the schema. It defaults to false. If you generate your schema programmatically, you can alternatively provide your own listener function.

• "warningListener" is a boolean that determines whether or not the tool logs a warning when a new user-defined type or domain is encountered and given its own type file in zapatos/custom. If true, which is the default, it does. Again, if you generate your schema programmatically, you can alternatively provide your own listener function.

• "customTypesTransform" is a string that determines how user-defined Postgres type names are mapped to TypeScript type names. Your options are "my_type", "PgMyType" or "PgMy_type", each representing how a Postgres type named my_type will be transformed. The default (for reasons of backward-compatibility rather than superiority) is "PgMy_type". If you generate your schema programmatically, you can alternatively define your own transformation function.

• "schemas" is an object that lets you define the schemas, and the tables and views within schemas, for which types will be generated. Each key is a schema name, and each value is an object with keys "include" and "exclude". Those keys can take the value "*" (for all tables in the schema) or an array of table names. The "exclude" list takes precedence over the "include" list. Thanks to generous sponsorship by Seam, schemas are properly supported (via namespacing of types) as of version 6.

If not specified, the default value for "schemas" includes all tables in the public schema, i.e.:

"schemas": {
  "public": {
    "include": "*",
    "exclude": []
  }
}

If you use PostGIS, you’ll likely want to exclude its system tables:

"schemas": {
  "public": {
    "include": "*",
    "exclude": [
      "geography_columns", 
      "geometry_columns", 
      "raster_columns", 
      "raster_overviews", 
      "spatial_ref_sys"
    ]
  }
}

• "unprefixedSchema" determines which schema’s objects don’t need to be prefixed with their schema name (so that you can specify table myTable rather than public.myTable, for example). It should be set to the first schema listed in your Postgres search_path that actually exists in the database. Usually, that’s "public", which is the option’s default value. "unprefixedSchema" can also be set to null, in which case all objects will be prefixed. That’s necessary if any schema shares its name with any table in the public schema.

• "columnOptions" is an object mapping options to named columns of named (or all) tables. Currently, you can use it to manually exclude column keys from the Insertable and Updatable types, using the options "insert": "excluded" and "update": "excluded", or to force column keys to be optional in Insertable types, using the option "insert": "optional". This supports use cases where columns are set using triggers.

For example, say you have a BEFORE INSERT trigger on your customers table that can guess a default value for the gender column based on the value of the title column (though note: don’t do that). In this case, the gender column is actually optional on insert, even if it’s NOT NULL with no default, because the trigger provides a default value. You can tell Zapatos about that like so:

"columnOptions": {
  "customers": {
    "gender": {
      "insert": "optional"
    }
  }
}

Note that tables outside the public schema (or whichever schema you set for "unprefixedSchema") should be schema-prefixed here, as usual — e.g. "columnOptions": { "someSchema.someTable": /* ... */ } }.

You can also use "*" as a wildcard to match all tables in all schemas. For example, perhaps you’ve set up the appropriate triggers to keep updatedAt columns up to date throughout your database. Then you might choose to exclude all your updatedAt columns from the Insertable and Updatable types for all tables as follows:

"columnOptions": {
  "*": {
    "updatedAt": {
      "insert": "excluded",
      "update": "excluded"
    }
  }
}

Wildcard table options have lower precedence than named table options. The default values, should you want to restore them for named tables, are "insert": "auto" and "update": "auto". Note that "*" is only supported as the whole key — you can’t use a * to match parts of schema or table names — and isn’t supported for column names.

• "schemaJSDoc" is a boolean that turns JSDoc comments for each column in the generated schema on (the default) or off. JSDoc comments enable per-column VS Code pop-ups giving details of Postgres data type, default value and so on. They also make the schema file longer and less readable.

• "customJSONParsingForLargeNumbers" is a boolean that changes the types for bigint/int8 and numeric/decimal values to reflect the use of custom JSON parsing to maintain precision.

In summary, the expected structure is defined like so:

export interface OptionalConfig {
  db: pg.ClientConfig;
  outDir: string;
  outExt: string;
  schemas: SchemaRules;
  unprefixedSchema: string | null;
  progressListener: boolean | ((s: string) => void);
  warningListener: boolean | ((s: string) => void);
  customTypesTransform: 'PgMy_type' | 'my_type' | 'PgMyType' | ((s: string) => string);
  columnOptions: ColumnOptions;
  schemaJSDoc: boolean;
  customJSONParsingForLargeNumbers: boolean;
}

interface SchemaRules {
  [schema: string]: {
    include: '*' | string[];
    exclude: '*' | string[];
  };
}

interface ColumnOptions {
  [k: string]: {  // table name or '*'
    [k: string]: {  // column name
      insert?: 'auto' | 'excluded' | 'optional';
      update?: 'auto' | 'excluded';
    };
  };
}

Environment variables

All values in zapatosconfig.json can have environment variables (Node’s process.env.SOMETHING) interpolated via handlebars-style doubly-curly-brackets {{variables}}.

This is likely most useful for the database connection details. For example, on Heroku you might configure your database as:

"db": {
  "connectionString": "{{DATABASE_URL}}"
}

ESLint / tslint

A general configuration suggestion: set up ESLint with the rules @typescript-eslint/await-thenable and @typescript-eslint/no-floating-promises (or the now-deprecated tslint with no-floating-promises and await-promise) to avoid various Promise-related pitfalls.

Generate your schema

Zapatos provides a command line tool. With everything configured, run it like so:

npx zapatos

This generates the TypeScript schema for your database as zapatos/schema.d.ts inside your configured outDir. Any user-defined or domain types encountered get defined within zapatos/custom in their own .d.ts files, which you can subsequently customise.

These files must be included in your TypeScript compilation. That may happen for you automatically, but you may need to check the "include" or "files" keys in tsconfig.json. If you use ts-node or node -r ts-node/register, you may need to change it to ts-node --files or set TS_NODE_FILES=true.

Programmatic generation

As an alternative to the command line tool, it’s also possible to generate the schema programmatically by importing from zapatos/generate. For example:

import * as zg from 'zapatos/generate';

const zapCfg: zg.Config = { db: { connectionString: 'postgres://localhost/mydb' } };
await zg.generate(zapCfg);

Call the generate method with an object structured exactly the same as zapatosconfig.json, documented above, with the following two exceptions:

• The "progressListener" and "warningListener" keys can each take true or false (as in the JSON case), or alternatively a function with the signature (s: string) => void, which you can use to implement your own logging.

• The "customTypesTransform" key can take any of the string values allowed in the JSON case, or otherwise a function with the signature (s: string) => string, with which you can define your own type name transformation.

Custom types and domains

As mentioned previously, any user-defined or domain types encountered during schema generation get defined in their own .d.ts files under zapatos/custom, which you can subsequently customise.

You can use domain types in order to specify custom types on the TypeScript side for certain Postgres columns. Say, for example, that you have a Postgres jsonb column on which you want to impose a particular structure. You could do the following:

CREATE DOMAIN "mySpecialJsonb" AS "jsonb";

Since you’ve done nothing else with this domain, it’s effectively just a simple alias to jsonb on the Postgres side. Now you can use that in place of jsonb in your table definition:

ALTER TABLE "myTable" ALTER COLUMN "myExistingJsonbColumn" TYPE "mySpecialJsonb";

When you next regenerate the TypeScript schema, you’ll find a custom type for PgMySpecialJsonb in zapatos/custom/PgMySpecialJsonb.d.ts, defined like so:

export type PgMySpecialJsonb = db.JSONValue;

You can of course replace this definition with whatever TypeScript type or interface you choose. The file will not be overwritten on future schema generations. For example, perhaps this column holds blog article data:

export interface PgMySpecialJsonb {
  title: string;
  text: string;
  tags: string[];
  version: number;
};

Import it

In your code, get the core library like so:

import * as db from 'zapatos/db';

ESM wrappers are provided, so the import should work the same whether your project is set to use the CommonJS or ESM module specs.

To import your ordinary schema types (myTable.Selectable, myOtherTable.Insertable, etc.):

import type * as s from 'zapatos/schema';

Be sure to import type for this, not plain import, or you’ll upset ts-jest and maybe others.

To import any user-defined or domain types:

import type * as c from 'zapatos/custom';

The paths zapatos/db and zapatos/generate point to real folders in node_modules. Although they look like file paths, zapatos/schema and zapatos/custom are actually the names of ambient modules declared in the generated files in your source tree: zapatos/schema.d.ts and zapatos/custom/*.d.ts.

User guide

sql tagged template strings

Arbitrary queries are written using the tagged template function sql, which returns SQLFragment class instances.

The sql function is generic, having two type variables. For example:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const authors = await db.sql<s.authors.SQL, s.authors.Selectable[]>`
  SELECT * FROM ${"authors"}`.run(pool);

SELECT *
FROM "authors"

[
  {
    "id": 1000,
    "name": "Philip Pullman",
    "isLiving": true
  },
  {
    "id": 1001,
    "name": "Mark Haddon",
    "isLiving": true
  },
  {
    "id": 1002,
    "name": "Louis Sachar",
    "isLiving": true
  },
  {
    "id": 1,
    "name": "Gabriel Garcia Marquez",
    "isLiving": false
  },
  {
    "id": 2,
    "name": "Douglas Adams",
    "isLiving": false
  },
  {
    "id": 3,
    "name": "Jane Austen",
    "isLiving": false
  }
]

The first type variable, Interpolations (above: s.authors.SQL), defines allowable interpolation values. If not specified, it defaults to db.SQL: this is the union of all the per-table SQL types, and thus allows all table and column names present in the database as string interpolations (some of which would throw runtime errors in this case).

As another example, imagine we were joining the authors and books tables. Then we could specify s.authors.SQL | s.books.SQL for Interpolations here.

The second type variable, RunResult (above: s.authors.Selectable[]), describes what will be returned if we call run() on the query (after any transformations performed in runResultTransform()), or if we embed it within the extras or lateral query options. Its default value if not specified is any[].

Take another example of these type variables:

import * as db from 'zapatos/db';
import pool from './pgPool';const [{ random }] = await db.sql<never, [{ random: number }]>`
  SELECT random()`.run(pool);

console.log(random);

SELECT random()

[
  {
    "random": 0.621588923150834
  }
]

0.621588923150834

Interpolations is never because nothing needs to be interpolated in this query, and the RunResult type says that the query will return one row comprising one numeric column, named random. The random TypeScript variable we initialize will of course be typed as a number.

If you’re happy to have your types tied down a little less tightly, it also works to wholly omit the type variables in this particular query, falling back on their defaults:

import * as db from 'zapatos/db';
import pool from './pgPool';const [{ random }] = await db.sql`SELECT random()`.run(pool);

In this case, the random variable is of course still a number, but it is typed as any.

sql template interpolation types

Strings

The strings that can be directly interpolated into a sql template string are defined by its Interpolations type variable, as noted above. Typically, this will limit them to the names of tables and columns.

Interpolated strings are passed through to the raw SQL query double-quoted, to preserve capitalisation and neutralise SQL keywords. For example, myTable becomes "myTable", and mySchema.myTable becomes "mySchema"."myTable".

It’s highly preferable to use interpolated string literals for table and column names rather than just writing those values in the query itself, in order to benefit from auto-completion and (ongoing) type-checking.

So, for example, do write:

import * as db from 'zapatos/db';
import pool from './pgPool';const title = await db.sql`
  SELECT ${"title"} FROM ${"books"} LIMIT 1`.run(pool);

But don’t write

import * as db from 'zapatos/db';
import pool from './pgPool';const title = await db.sql`
  SELECT "title" FROM "books" LIMIT 1`.run(pool);  // no, don't do this

— even if the two produce the same result right now.

More critically, never never never explicitly override type-checking so as to write:

import * as db from 'zapatos/db';
import pool from './pgPool';const 
  nameSubmittedByUser = 'books"; DROP TABLE "authors"; --',
  title = await db.sql<any>`
    SELECT * FROM ${nameSubmittedByUser} LIMIT 1`.run(pool);  // NEVER do this!

SELECT *
FROM "books";

DROP TABLE "authors";

--" LIMIT 1

error: cannot drop table authors because other objects depend on it

If you override type-checking to pass untrusted data to Zapatos in unexpected places, such as the above use of any, you can expect successful SQL injection attacks.

(It is safe to pass untrusted data as values in Whereable, Insertable, and Updatable objects, manually by using param, and in certain other places. If you’re in any doubt, double-check that the generated SQL is using $1, $2, … parameters for all potentially untrusted data).

cols() and vals()

The cols and vals wrapper functions (which return ColumnNames and ColumnValues class instances respectively) are intended to help with certain INSERT and SELECT queries.

In the INSERT context, pass them each the same Insertable object: cols is compiled to a comma-separated list of the object’s keys, which are the column names, and vals is compiled to a comma-separated list of SQL placeholders ($1, $2, …) associated with the corresponding values, in matching order. To return to (approximately) an earlier example:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const
  author: s.authors.Insertable = {
    name: 'Joseph Conrad',
    isLiving: false,
  },
  [insertedAuthor] = await db.sql<s.authors.SQL, s.authors.Selectable[]>`
    INSERT INTO ${"authors"} (${db.cols(author)})
    VALUES (${db.vals(author)}) RETURNING *`.run(pool);

INSERT INTO "authors" ("isLiving", "name")
  VALUES ($1, $2)
RETURNING *

[false, "Joseph Conrad"]

[
  {
    "id": 4,
    "name": "Joseph Conrad",
    "isLiving": false
  }
]

The cols and vals wrappers can also each take an array instead of an object.

For the cols function, this can help us select only a subset of columns, in conjunction with the OnlyCols type. Pass an array of column names to cols to have them compiled appropriately, as seen in this example:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';// the <const> prevents generalization to string[]
const bookCols = <const>['id', 'title'];
type BookDatum = s.books.OnlyCols<typeof bookCols>;

const
  bookData = await db.sql<s.books.SQL, BookDatum[]>`
    SELECT ${db.cols(bookCols)} FROM ${"books"}`.run(pool);

SELECT "id", "title"
FROM "books"

[
  {
    "id": 1000,
    "title": "Northern Lights"
  },
  {
    "id": 1001,
    "title": "The Subtle Knife"
  },
  {
    "id": 1002,
    "title": "The Amber Spyglass"
  },
  {
    "id": 1003,
    "title": "The Curious Incident of the Dog in the Night-Time"
  },
  {
    "id": 1004,
    "title": "Holes"
  }
]

For the vals function, this can help with IN (...) queries, such as the following:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const 
  authorIds = [1, 2, 123],
  authors = await db.sql<s.authors.SQL, s.authors.Selectable[]>` 
    SELECT * FROM ${"authors"} WHERE ${"id"} IN (${db.vals(authorIds)})`.run(pool);

SELECT *
FROM "authors"
WHERE "id" IN ($1, $2, $3)

[1, 2, 123]

[
  {
    "id": 1,
    "name": "Gabriel Garcia Marquez",
    "isLiving": false
  },
  {
    "id": 2,
    "name": "Douglas Adams",
    "isLiving": false
  }
]

Whereable

Any plain JavaScript object interpolated into a sql template string is type-checked as a Whereable, and compiled into one or more conditions joined with AND (but, for flexibility, no WHERE). The object’s keys represent column names, and the corresponding values are automatically compiled as (injection-safe) Parameter instances.

For example:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const 
  title = 'Northern Lights',
  books = await db.sql<s.books.SQL, s.books.Selectable[]>`
    SELECT * FROM ${"books"} WHERE ${{ title }}`.run(pool);

SELECT *
FROM "books"
WHERE ("title" = $1)

["Northern Lights"]

[
  {
    "id": 1000,
    "authorId": 1000,
    "title": "Northern Lights",
    "createdAt": "2024-06-23T14:22:52.603Z"
  }
]

(If you need to specify a CAST of a parameter to a specific SQL type, you can also manually wrap Whereable values using param — this is useful primarily when using the shortcut functions).

A Whereable’s values can alternatively be SQLFragments, and this makes them extremely flexible. In a SQLFragment inside a Whereable, the special symbol self can be used to refer to the column name. This arrangement enables us to use any operator or function we want — not just =.

For example:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const 
  titleLike = 'Northern%',
  books = await db.sql<s.books.SQL, s.books.Selectable[]>`
    SELECT * FROM ${"books"} WHERE ${{ 
      title: db.sql`${db.self} LIKE ${db.param(titleLike)}`,
      createdAt: db.sql`${db.self} > now() - INTERVAL '7 days'`,
    }}`.run(pool);

SELECT *
FROM "books"
WHERE (("createdAt" > now() - INTERVAL '7 days')
  AND ("title" LIKE $1))

["Northern%"]

[
  {
    "id": 1000,
    "authorId": 1000,
    "title": "Northern Lights",
    "createdAt": "2024-06-23T14:22:52.603Z"
  }
]

Finally, there’s a set of helper functions you can use to create appropriate SQLFragments like these for use as Whereable values. The advantages are: (1) there’s slighly less to type, and (2) you get type-checking on their arguments (so you’re not tempted to compare incomparable things).

They’re exported under conditions on the main object, and the full set can be seen in conditions.ts. Using some of these, we could rewrite the above example as:

import * as db from 'zapatos/db';
import { conditions as dc } from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const 
  titleLike = 'Northern%',
  books = await db.sql<s.books.SQL, s.books.Selectable[]>`
    SELECT * FROM ${"books"} WHERE ${{ 
      title: dc.like(titleLike),
      createdAt: dc.after(dc.fromNow(-7, 'days')),
    }}`.run(pool);

SELECT *
FROM "books"
WHERE (("createdAt" > now() + $1)
  AND ("title" LIKE $2))

["-7 days", "Northern%"]

[
  {
    "id": 1000,
    "authorId": 1000,
    "title": "Northern Lights",
    "createdAt": "2024-06-23T14:22:52.603Z"
  }
]

self

The use of the self symbol is explained in the section on Whereables.

param(value: any, cast?: boolean | string): Parameter

In general, Zapatos’ type-checking won’t let us pass user-supplied data unsafely into a query by accident. The param wrapper function exists to enable the safe passing of user-supplied data into a query using numbered query parameters ($1, $2, …).

For example:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const 
  title = 'Pride and Prejudice',
  books = await db.sql<s.books.SQL, s.books.Selectable[]>`
    SELECT * FROM ${"books"} WHERE ${"title"} = ${db.param(title)}`.run(pool);

SELECT *
FROM "books"
WHERE "title" = $1

["Pride and Prejudice"]

This same mechanism is applied automatically when we use a Whereable object (and in this example, using a Whereable would be more readable and more concise). It’s also applied when we use the vals function to create a ColumnValues wrapper object.

The optional second argument to param, cast, allows us to specify a SQL CAST type for the wrapped value. If cast is a string, it’s interpreted as a Postgres type, so param(someValue, 'text') comes out in the compiled query as as CAST($1 TO "text"). If cast is true, the parameter value will be JSON stringified and cast to json, and if cast is false, the parameter will not be JSON stringified or cast to json (regardless, in both cases, of the castArrayParamsToJson and castObjectParamsToJson configuration options).

Default

The Default symbol simply compiles to the SQL DEFAULT keyword. This may be useful in INSERT and UPDATE queries where no value is supplied for one or more of the affected columns.

sql template strings

sql template strings (resulting in SQLFragments) can be interpolated within other sql template strings (SQLFragments). This provides flexibility in building queries programmatically.

For example, the select shortcut makes extensive use of nested sql templates to build its queries:

const
  rowsQuery = sql<SQL, any>`
    SELECT ${allColsSQL} AS result 
    FROM ${table}${tableAliasSQL}
    ${lateralSQL}${whereSQL}${orderSQL}${limitSQL}${offsetSQL}`,

  // we need the aggregate function, if one's needed, to sit in an outer 
  // query, to keep ORDER and LIMIT working normally in the main query
  query = mode !== SelectResultMode.Many ? rowsQuery :
    sql<SQL, any>`
      SELECT coalesce(jsonb_agg(result), '[]') AS result 
      FROM (${rowsQuery}) AS ${raw(`"sq_${aliasedTable}"`)}`;

Arrays

Items in an interpolated array are treated just the same as if they had been interpolated directly. This, again, can be useful for building queries programmatically.

To take the select shortcut as our example again, an interpolated array is used to generate LATERAL JOIN query elements from the lateral option, like so:

const
  lateralOpt = allOptions.lateral,
  lateralSQL = lateralOpt === undefined ? [] :
    Object.keys(lateralOpt).map(k => {
      const subQ = lateralOpt[k];
      subQ.parentTable = aliasedTable;  // enables `parent()` in subquery's Whereables
      return sql<SQL>` LEFT JOIN LATERAL (${subQ}) AS ${raw(`"cj_${k}"`)} ON true`;
    });

The lateralSQL variable — a SQLFragment[] — is subsequently interpolated into the final query (some additional SQL using jsonb_build_object() is interpolated earlier in that query, to return the result of the lateral subquery alongside the main query columns).

Note that a useful idiom also seen here is the use of the empty array ([]) to conditionally interpolate nothing at all.

raw(value: string): DangerousRawString

The raw function returns DangerousRawString wrapper instances. This represents an escape hatch, enabling us to interpolate arbitrary strings into queries in contexts where the param wrapper is unsuitable (such as when we’re interpolating basic SQL syntax elements). If you pass user-controlled data to this function you will open yourself up to SQL injection attacks.

parent(columnName?: string): ParentColumn

Within queries passed as subqueries to the lateral option of select and related queries, the parent() wrapper can be used to refer to a column of the table that’s the subject of the immediately containing query (the ‘parent’ table).

To refer to a column of the parent table by name, pass a string argument. If the column of the parent table has the same name as the column with which it’s being joined, no argument is required.

For usage details, see the documentation for the lateral option.

SQLFragment

SQLFragment<RunResult> class instances are what is returned by the sql tagged template function — you’re unlikely ever to contruct them directly with new. They take on the RunResult type variable from the sql template function that constructs them.

You can interpolate them into other sql tagged template strings, or call/access the following properties on them:

prepared(name: string): this

The prepared function causes a name property to be added to the compiled SQL query object that’s passed to pg, and this instructs Postgres to treat it as a prepared statement. You can specify a prepared statement name as the function’s argument, or let it default to "_zapatos_prepared_N" (where N is a sequence number). This name appears in the Postgres logs.

async run(queryable: Queryable, force = false): Promise<RunResult>

The run function compiles, executes, and returns the transformed result of the query represented by this SQLFragment. The awaited return value is typed according to the SQLFragment’s RunResult type variable.

Taking that one step at a time:

1. First, the compile function is called, recursively compiling this SQLFragment and its interpolated values into a { text: '', values: [] } query that can be passed straight to the pg module. If a queryListener function has been configured, it is called with the query as its argument now.

2. Next, the compiled SQL query is executed against the supplied Queryable, which is defined as a pg.Pool or pg.ClientBase (this definition also covers the TxnClient provided by the transaction helper function).

3. Finally, the result returned from pg is fed through this SQLFragment’s runResultTransform() function, whose default implementation simply returns the rows property of the result. If a resultListener function has been configured, it is called with the transformed result as its argument now.

Examples of the run function are scattered throughout this documentation.

The force parameter is relevant only if this SQLFragment has been marked as a no-op: at present, Zapatos does this automatically if you pass an empty array to insert or upsert. By default, the database will not be disturbed in such cases, but you can force a no-op query to actually be run against the database — perhaps for logging or triggering reasons — by setting force to true.

compile(): SQLQuery

The compile function recursively transforms this SQLFragment and its interpolated values into a SQLQuery object ({ text: string; values: any[]; }) that can be passed straight to the pg module. It is called without arguments (the arguments it can take are for internal use).

For example:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';const 
  authorId = 12,  // from some untrusted source
  query = db.sql<s.books.SQL, s.books.Selectable[]>`
    SELECT * FROM ${"books"} WHERE ${{authorId}}`,
  compiled = query.compile();

console.log(compiled);

{
  text: '\n    SELECT * FROM "books" WHERE ("authorId" = $1)',
  values: [ 12 ]
}

You may never need this function. Use it if and when you want to see the SQL that would be executed by the run function, without in fact executing it.

runResultTransform: (qr: pg.QueryResult) => any

When you call run, the function stored in this property is applied to the QueryResult object returned by pg, in order to produce the result that the run function ultimately returns.

By default, the QueryResult’s rows property (which is an array) is returned: that is, the default implementation is just qr => qr.rows. However, the shortcut functions supply their own runResultTransform implementations in order to match their declared RunResult types.

Generally you will not need to call this function directly, but there may be cases where you want to assign a new function to replace the default implementation.

For example, imagine we wanted to create a function returning a query that, when run, returns the current database timestamp directly as a Date. We could do so like this:

import * as db from 'zapatos/db';
import pool from './pgPool';function dbNowQuery() {
  const query = db.sql<never, Date>`SELECT now()`;
  query.runResultTransform = qr => qr.rows[0].now;
  return query;
}

const dbNow = await dbNowQuery().run(pool);
// dbNow is a Date: the result you can toggle below has come via JSON.stringify

SELECT now()

"2024-06-23T14:24:09.784Z"

Note that the RunResult type variable on the sql template function (in this case, Date) must reflect the type of the transformed result, not what comes straight back from pg (which in this case is roughly { rows: [{ now: Date }] }).

If a SQLFragment does not have run called on it directly — for example, if it is instead interpolated into another SQLFragment, or given as the value of the lateral option to the select shortcut — then the runResultTransform function is never applied.

Manual joins using Postgres’ JSON features

We can make use of Postgres’ excellent JSON support to achieve a variety of JOIN queries. That’s not unique to Zapatos, of course, but it may be helpful to consider a few example queries in this context.

Take this example, retrieving each book with its (single) author:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';type bookAuthorSQL = s.books.SQL | s.authors.SQL | "author";
type bookAuthorSelectable = s.books.Selectable & { author: s.authors.Selectable };

const query = db.sql<bookAuthorSQL, bookAuthorSelectable[]>`
  SELECT ${"books"}.*, to_jsonb(${"authors"}.*) as ${"author"}
  FROM ${"books"} JOIN ${"authors"} 
  ON ${"books"}.${"authorId"} = ${"authors"}.${"id"}`;

const bookAuthors = await query.run(pool);

SELECT "books".*, to_jsonb ("authors".*) as "author"
FROM "books"
  JOIN "authors" ON "books"."authorId" = "authors"."id"

[
  {
    "id": 1000,
    "authorId": 1000,
    "title": "Northern Lights",
    "createdAt": "2024-06-23T14:22:52.603Z",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    }
  },
  {
    "id": 1001,
    "authorId": 1000,
    "title": "The Subtle Knife",
    "createdAt": "2024-06-23T14:22:52.603Z",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    }
  },
  {
    "id": 1002,
    "authorId": 1000,
    "title": "The Amber Spyglass",
    "createdAt": "2024-06-23T14:22:52.603Z",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    }
  },
  {
    "id": 1003,
    "authorId": 1001,
    "title": "The Curious Incident of the Dog in the Night-Time",
    "createdAt": "2024-06-23T14:22:52.605Z",
    "author": {
      "id": 1001,
      "name": "Mark Haddon",
      "isLiving": true
    }
  },
  {
    "id": 1004,
    "authorId": 1002,
    "title": "Holes",
    "createdAt": "2024-06-23T14:22:52.605Z",
    "author": {
      "id": 1002,
      "name": "Louis Sachar",
      "isLiving": true
    }
  }
]

Of course, we might also want the converse query, retrieving each author with their (many) books. This is also easy enough to arrange:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';type authorBooksSQL = s.authors.SQL | s.books.SQL;
type authorBooksSelectable = s.authors.Selectable & { books: s.books.Selectable[] };

const query = db.sql<authorBooksSQL, authorBooksSelectable[]>`
  SELECT ${"authors"}.*, jsonb_agg(${"books"}.*) AS ${"books"}
  FROM ${"authors"} JOIN ${"books"} 
  ON ${"authors"}.${"id"} = ${"books"}.${"authorId"}
  GROUP BY ${"authors"}.${"id"}`;

const authorBooks = await query.run(pool);

SELECT "authors".*, jsonb_agg("books".*) AS "books"
FROM "authors"
  JOIN "books" ON "authors"."id" = "books"."authorId"
GROUP BY "authors"."id"

[
  {
    "id": 1000,
    "name": "Philip Pullman",
    "isLiving": true,
    "books": [
      {
        "id": 1000,
        "title": "Northern Lights",
        "authorId": 1000,
        "createdAt": "2024-06-23T15:22:52.603082+01:00"
      },
      {
        "id": 1001,
        "title": "The Subtle Knife",
        "authorId": 1000,
        "createdAt": "2024-06-23T15:22:52.603558+01:00"
      },
      {
        "id": 1002,
        "title": "The Amber Spyglass",
        "authorId": 1000,
        "createdAt": "2024-06-23T15:22:52.603732+01:00"
      }
    ]
  },
  {
    "id": 1001,
    "name": "Mark Haddon",
    "isLiving": true,
    "books": [
      {
        "id": 1003,
        "title": "The Curious Incident of the Dog in the Night-Time",
        "authorId": 1001,
        "createdAt": "2024-06-23T15:22:52.605245+01:00"
      }
    ]
  },
  {
    "id": 1002,
    "name": "Louis Sachar",
    "isLiving": true,
    "books": [
      {
        "id": 1004,
        "title": "Holes",
        "authorId": 1002,
        "createdAt": "2024-06-23T15:22:52.605559+01:00"
      }
    ]
  }
]

Note that if you want to include authors with no books, you need a LEFT JOIN in this query, and then you’ll also want to fix the annoying [null] array results jsonb_agg will return for those authors.

Rather than do it that way, though, we can achieve the same result using a LATERAL JOIN instead:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';type authorBooksSQL = s.authors.SQL | s.books.SQL;
type authorBooksSelectable = s.authors.Selectable & { books: s.books.Selectable[] };

const query = db.sql<authorBooksSQL, authorBooksSelectable[]>`
  SELECT ${"authors"}.*, bq.* 
  FROM ${"authors"} LEFT JOIN LATERAL (
    SELECT coalesce(json_agg(${"books"}.*), '[]') AS ${"books"}
    FROM ${"books"}
    WHERE ${"books"}.${"authorId"} = ${"authors"}.${"id"}
  ) bq ON true`;

const authorBooks = await query.run(pool);

SELECT "authors".*, bq.*
FROM "authors"
  LEFT JOIN LATERAL (
    SELECT coalesce(json_agg("books".*), '[]') AS "books"
    FROM "books"
    WHERE "books"."authorId" = "authors"."id") bq ON true

[
  {
    "id": 1000,
    "name": "Philip Pullman",
    "isLiving": true,
    "books": [
      {
        "id": 1000,
        "authorId": 1000,
        "title": "Northern Lights",
        "createdAt": "2024-06-23T15:22:52.603082+01:00"
      },
      {
        "id": 1001,
        "authorId": 1000,
        "title": "The Subtle Knife",
        "createdAt": "2024-06-23T15:22:52.603558+01:00"
      },
      {
        "id": 1002,
        "authorId": 1000,
        "title": "The Amber Spyglass",
        "createdAt": "2024-06-23T15:22:52.603732+01:00"
      }
    ]
  },
  {
    "id": 1001,
    "name": "Mark Haddon",
    "isLiving": true,
    "books": [
      {
        "id": 1003,
        "authorId": 1001,
        "title": "The Curious Incident of the Dog in the Night-Time",
        "createdAt": "2024-06-23T15:22:52.605245+01:00"
      }
    ]
  },
  {
    "id": 1002,
    "name": "Louis Sachar",
    "isLiving": true,
    "books": [
      {
        "id": 1004,
        "authorId": 1002,
        "title": "Holes",
        "createdAt": "2024-06-23T15:22:52.605559+01:00"
      }
    ]
  },
  {
    "id": 1,
    "name": "Gabriel Garcia Marquez",
    "isLiving": false,
    "books": []
  },
  {
    "id": 2,
    "name": "Douglas Adams",
    "isLiving": false,
    "books": []
  },
  {
    "id": 3,
    "name": "Jane Austen",
    "isLiving": false,
    "books": []
  },
  {
    "id": 4,
    "name": "Joseph Conrad",
    "isLiving": false,
    "books": []
  }
]

Lateral joins of this sort are very flexible, and can be nested multiple levels deep — but can quickly become quite hairy in that case. The select shortcut function and its lateral option can make this much less painful.

Shortcut functions and lateral joins

A key contribution of Zapatos is a set of simple shortcut functions that make everyday CRUD queries extremely easy to work with. Furthermore, the select shortcut can be nested in order to generate LATERAL JOIN queries, resulting in arbitrarily complex nested JSON structures with inputs and outputs that are still fully and automatically typed.

The shortcut functions make heavy use of Postgres’ JSON support, and their return values are thus JSONSelectables rather than the plain Selectables you’d get back from a manual query.

insert

The insert shortcut inserts one or more rows in a table, and returns them with any DEFAULT or generated values filled in. It takes a Table name and the corresponding Insertable or Insertable[], and returns the corresponding JSONSelectable or JSONSelectable[] (subject to the options described below).

The optional options argument has two keys.

• returning takes an array of column names, and narrows down the returned values accordingly. This may be useful if you are inserting large objects which you prefer don’t take an inefficient return trip over the wire and through the JSON parser.

• extras takes a map of string keys to column names and/or sql template strings (i.e. SQLFragments), allowing you to alias certain columns and/or compute and return other quantities alongside them. The RunResult type variable matters in the case of template strings, as it is passed through to the result type.

(Note that type inference can only do the right thing with returning and extras when strictNullChecks are enabled).

For example:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const 
  // insert one
  steve = await db.insert('authors', { 
    name: 'Steven Hawking', 
    isLiving: false,
  }).run(pool),

  // insert many
  [time, me] = await db.insert('books', [{ 
    authorId: steve.id, 
    title: 'A Brief History of Time',
    createdAt: db.sql`now()`,
  }, { 
    authorId: steve.id, 
    title: 'My Brief History',
    createdAt: db.sql`now()`,
  }]).run(pool),

  tags = await db.insert('tags', [
    { bookId: time.id, tag: 'physics' },
    { bookId: me.id, tag: 'physicist' },
    { bookId: me.id, tag: 'autobiography' },
  ]).run(pool),

  // insert with custom return values
  nutshell = await db.insert('books', { 
    authorId: steve.id, 
    title: 'The Universe in a Nutshell',
    createdAt: db.sql`now()`,
  }, {
    returning: ['id'],
    extras: { 
      aliasedTitle: "title",
      upperTitle: db.sql<s.books.SQL, string | null>`upper(${"title"})`,
    },
  }).run(pool);

INSERT INTO "authors" ("isLiving", "name")
  VALUES ($1, $2)
RETURNING to_jsonb ("authors".*) AS result

[false, "Steven Hawking"]

{
  "id": 5,
  "name": "Steven Hawking",
  "isLiving": false
}

INSERT INTO "books" ("authorId", "createdAt", "title")
  VALUES ($1, now(), $2), ($3, now(), $4)
RETURNING to_jsonb ("books".*) AS result

[5, "A Brief History of Time", 5, "My Brief History"]

[
  {
    "id": 1,
    "title": "A Brief History of Time",
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.463945+01:00"
  },
  {
    "id": 2,
    "title": "My Brief History",
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.463945+01:00"
  }
]

INSERT INTO "tags" ("bookId", "tag")
  VALUES ($1, $2), ($3, $4), ($5, $6)
RETURNING to_jsonb ("tags".*) AS result

[1, "physics", 2, "physicist", 2, "autobiography"]

[
  {
    "tag": "physics",
    "bookId": 1
  },
  {
    "tag": "physicist",
    "bookId": 2
  },
  {
    "tag": "autobiography",
    "bookId": 2
  }
]

INSERT INTO "books" ("authorId", "createdAt", "title")
  VALUES ($1, now(), $2)
RETURNING jsonb_build_object($3::text, "id") || jsonb_build_object($4::text, "title", $5::text, upper("title")) AS result

[5, "The Universe in a Nutshell", "id", "aliasedTitle", "upperTitle"]

{
  "id": 3,
  "upperTitle": "THE UNIVERSE IN A NUTSHELL",
  "aliasedTitle": "The Universe in a Nutshell"
}

You’ll note that Insertables can take SQLFragment values (from the sql tagged template function) as well as direct values (strings, numbers, and so on).

Postgres can accept up to 65,536 parameters per query (since an Int16 is used to convey the number of parameters in the Bind message of the wire protocol). If there’s a risk that a multiple-row INSERT could have more inserted values than that, you’ll need a mechanism to batch them up into separate calls.

If you provide an empty array to insert, this is identified as a no-op, and the database will not actually be queried unless you set the force option on run to true.

import * as db from 'zapatos/db';
import pool from './pgPool';await db.insert("authors", []).run(pool);  // never reaches DB
await db.insert("authors", []).run(pool, true);  // does reach DB, for same result

/* marked no-op: won't hit DB unless forced -> */
INSERT INTO "authors"
SELECT null
WHERE false

[]

/* marked no-op: won't hit DB unless forced -> */
INSERT INTO "authors"
SELECT null
WHERE false

[]

update

The update shortcut updates rows in the database. It takes a Table name and a corresponding Updatable and Whereable in that order, matching their order in the raw SQL query.

It returns a JSONSelectable[], listing every column of every row affected (or a subset or superset of those columns, if you use the returning and/or extras options, which work just as described above for insert).

For example, when we discover with that we’ve mis-spelled a famous physicist’s name, we can do this:

import * as db from 'zapatos/db';
import pool from './pgPool';await db.update('authors', 
  { name: 'Stephen Hawking' },
  { name: 'Steven Hawking' }
).run(pool);

UPDATE
  "authors"
SET ("name") = ROW ($1)
WHERE ("name" = $2)
RETURNING to_jsonb ("authors".*) AS result

["Stephen Hawking", "Steven Hawking"]

[
  {
    "id": 5,
    "name": "Stephen Hawking",
    "isLiving": false
  }
]

Like Insertable values, Updatable values can also be SQLFragments. For instance, take a table such as the following:

CREATE TABLE "emailAuthentication" 
( "email" citext PRIMARY KEY
, "consecutiveFailedLogins" INTEGER NOT NULL DEFAULT 0
, "lastFailedLogin" TIMESTAMPTZ );

To atomically increment the consecutiveFailedLogins value, we can do something like this:

import * as db from 'zapatos/db';
import { conditions as dc } from 'zapatos/db';
import pool from './pgPool';await db.update("emailAuthentication", { 
  consecutiveFailedLogins: db.sql`${db.self} + 1`,  
  // or equivalently: consecutiveFailedLogins: dc.add(1),
  lastFailedLogin: db.sql`now()`,
  // or equivalently: lastFailedLogin: dc.now,
}, { email: 'me@privacy.net' }).run(pool);

UPDATE
  "emailAuthentication"
SET ("consecutiveFailedLogins", "lastFailedLogin") = ROW ("consecutiveFailedLogins" + 1, now())
WHERE ("email" = $1)
RETURNING to_jsonb ("emailAuthentication".*) AS result

["me@privacy.net"]

[
  {
    "email": "me@privacy.net",
    "lastFailedLogin": "2024-06-23T15:24:11.210042+01:00",
    "consecutiveFailedLogins": 1
  }
]

upsert

The upsert shortcut issues an INSERT ... ON CONFLICT ... query. Like insert, it takes a Table name and a corresponding Insertable or Insertable[].

It then takes, in addition, a column name (or an array thereof) or an appropriate unique index as the conflict target: the ‘arbiter index(es)’ on which a conflict is to be detected.

It returns an UpsertReturnable or UpsertReturnable[]. An UpsertReturnable is the same as a JSONSelectable except that it includes one additional property, $action, taking the string 'INSERT' or 'UPDATE' so as to indicate which eventuality occurred for each row.

Let’s say we have a table of app subscription transactions:

CREATE TABLE "appleTransactions" 
( "environment" "appleEnvironment" NOT NULL  -- enum: 'PROD' or 'Sandbox'
, "originalTransactionId" TEXT NOT NULL
, "accountId" INTEGER REFERENCES "accounts"("id") NOT NULL
, "latestReceiptData" TEXT );

ALTER TABLE "appleTransactions" ADD CONSTRAINT "appleTransactionsPrimaryKey" 
  PRIMARY KEY ("environment", "originalTransactionId");

When we receive a purchase receipt, we need to either store a new record or update an existing record for each distinct (environment, originalTransactionId) it contains.

We can map the transaction data in the receipt into an appleTransactions.Insertable[], and do what’s needed with a single upsert call. In this example, though, we hard-code the Insertable[] for ease of exposition:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const 
  newTransactions: s.appleTransactions.Insertable[] = [{
    environment: 'PROD',
    originalTransactionId: '123456',
    accountId: 123,
    latestReceiptData: 'TWFuIGlzIGRpc3Rp',
  }, {
    environment: 'PROD',
    originalTransactionId: '234567',
    accountId: 234,
    latestReceiptData: 'bmd1aXNoZWQsIG5v',
  }],
  result = await db.upsert('appleTransactions', newTransactions, 
    ['environment', 'originalTransactionId']).run(pool);

INSERT INTO "appleTransactions" ("accountId", "environment", "latestReceiptData", "originalTransactionId")
  VALUES ($1, $2, $3, $4), ($5, $6, $7, $8)
ON CONFLICT ("environment", "originalTransactionId")
  DO UPDATE SET
    ("environment", "originalTransactionId", "accountId", "latestReceiptData") = ROW (EXCLUDED. "environment", EXCLUDED. "originalTransactionId", EXCLUDED. "accountId", EXCLUDED. "latestReceiptData")
  RETURNING to_jsonb ("appleTransactions".*) || jsonb_build_object('$action', CASE xmax
      WHEN 0 THEN
        'INSERT'
      ELSE
        'UPDATE'
      END) AS result

[123, "PROD", "TWFuIGlzIGRpc3Rp", "123456", 234, "PROD", "bmd1aXNoZWQsIG5v", "234567"]

[
  {
    "$action": "UPDATE",
    "accountId": 123,
    "environment": "PROD",
    "latestReceiptData": "TWFuIGlzIGRpc3Rp",
    "originalTransactionId": "123456"
  },
  {
    "$action": "INSERT",
    "accountId": 234,
    "environment": "PROD",
    "latestReceiptData": "bmd1aXNoZWQsIG5v",
    "originalTransactionId": "234567"
  }
]

And it’s wholly equivalent here to use the unique index name instead of the column names for the conflict target, by using the constraint wrapper function:

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';const 
  anotherNewTransaction: s.appleTransactions.Insertable = {
    environment: 'PROD',
    originalTransactionId: '345678',
    accountId: 345,
    latestReceiptData: 'lALvEleO4Ehwk3T5',
  },
  result = await db.upsert('appleTransactions', anotherNewTransaction, 
    db.constraint('appleTransactionsPrimaryKey')).run(pool);

INSERT INTO "appleTransactions" ("accountId", "environment", "latestReceiptData", "originalTransactionId")
  VALUES ($1, $2, $3, $4)
ON CONFLICT ON CONSTRAINT "appleTransactionsPrimaryKey"
  DO UPDATE SET
    ("environment", "originalTransactionId", "accountId", "latestReceiptData") = ROW (EXCLUDED. "environment", EXCLUDED. "originalTransactionId", EXCLUDED. "accountId", EXCLUDED. "latestReceiptData")
  RETURNING to_jsonb ("appleTransactions".*) || jsonb_build_object('$action', CASE xmax
      WHEN 0 THEN
        'INSERT'
      ELSE
        'UPDATE'
      END) AS result

[345, "PROD", "lALvEleO4Ehwk3T5", "345678"]

{
  "$action": "INSERT",
  "accountId": 345,
  "environment": "PROD",
  "latestReceiptData": "lALvEleO4Ehwk3T5",
  "originalTransactionId": "345678"
}

The same as for insert, an empty array provided to upsert is identified as a no-op, and the database will not actually be queried unless you set the force option on run to true.

upsert options

The optional fourth argument to upsert is an options object. The available options are returning and extras (see the documentation for insert for details) plus updateColumns, noNullUpdateColumns, updateValues and reportAction.

• The updateColumns option allows us to specify a subset of columns (as either one name or an array of names) that are to be updated on conflict. For example, you might want to include all columns except createdAt in this list.

• The noNullUpdateColumns option takes a column name or array of column names which are not to be overwritten with NULL in the case that the UPDATE branch is taken. It can also take the special value db.all to indicate that no column should ever be overwritten with NULL.

• The updateValues option allows us to specify alternative column values to be used in the UPDATE query branch: see below.

• The reportAction: 'suppress' option causes the $action result key to be omitted, so the query returns plain JSONSelectable instead of UpsertReturnable results.

INSERT ... ON CONFLICT ... DO NOTHING

A special case arises if you pass the empty array [] to the updateColumns option of upsert.

Since no columns are then to be updated in case of a conflict, an ON CONFLICT ... DO NOTHING query is generated instead of an ON CONFLICT ... DO UPDATE ... query. For better self-documenting code, an alias for the empty array is provided for this case: doNothing.

Since nothing is returned by Postgres for any DO NOTHING cases, a query with updateColumns: [] or updateColumns: db.doNothing may return fewer rows than were passed in. If you pass in an array, you could get back an empty array if all rows conflict with existing rows. If you pass in values of a single row, you’ll get back undefined if a conflict occurs (and the return type will automatically reflect this).

For example:

CREATE TABLE "usedVoucherCodes" 
( "code" text PRIMARY KEY
, "redeemedAt" timestamptz NOT NULL DEFAULT now()
);

import * as db from 'zapatos/db';
import pool from './pgPool';// unused code: returns the inserted row
const a = await db.upsert('usedVoucherCodes', 
  { code: 'XYE953ZVU767' }, 'code', 
  { updateColumns: db.doNothing }).run(pool);

// same code, already used: returns undefined
const b = await db.upsert('usedVoucherCodes', 
  { code: 'XYE953ZVU767' }, 'code', 
  { updateColumns: db.doNothing }).run(pool);

INSERT INTO "usedVoucherCodes" ("code")
  VALUES ($1)
ON CONFLICT ("code")
  DO NOTHING
RETURNING to_jsonb ("usedVoucherCodes".*) || jsonb_build_object('$action', CASE xmax
    WHEN 0 THEN
      'INSERT'
    ELSE
      'UPDATE'
    END) AS result

["XYE953ZVU767"]

{
  "code": "XYE953ZVU767",
  "$action": "INSERT",
  "redeemedAt": "2024-06-23T15:24:11.733317+01:00"
}

INSERT INTO "usedVoucherCodes" ("code")
  VALUES ($1)
ON CONFLICT ("code")
  DO NOTHING
RETURNING to_jsonb ("usedVoucherCodes".*) || jsonb_build_object('$action', CASE xmax
    WHEN 0 THEN
      'INSERT'
    ELSE
      'UPDATE'
    END) AS result

["XYE953ZVU767"]

undefined

updateValues

You can use the updateValues option to specify alternative column values to be used in the UPDATE branch of the query. Only one set of values can be provided: these will be used for any and all rows that get updated.

This may be useful, for example, when keeping a count, using a table such as this:

CREATE TABLE "nameCounts" 
( "name" text PRIMARY KEY
, "count" integer NOT NULL
);

In the following query, we insert a new value with a count of 1 if a name doesn’t already exist in the table. If a name does exist, we increment the existing count instead:

import * as db from 'zapatos/db';
import pool from './pgPool';for (let i = 0; i < 2; i++) {
  await db.upsert('nameCounts',
    { name: 'Alice', count: 1 }, 'name',
    { updateValues: { count: db.sql`${"nameCounts"}.${"count"} + 1` } }
  ).run(pool);
}

INSERT INTO "nameCounts" ("count", "name")
  VALUES ($1, $2)
ON CONFLICT ("name")
  DO UPDATE SET
    ("name", "count") = ROW (EXCLUDED. "name", "nameCounts"."count" + 1)
  RETURNING to_jsonb ("nameCounts".*) || jsonb_build_object('$action', CASE xmax
      WHEN 0 THEN
        'INSERT'
      ELSE
        'UPDATE'
      END) AS result

[1, "Alice"]

{
  "name": "Alice",
  "count": 1,
  "$action": "INSERT"
}

INSERT INTO "nameCounts" ("count", "name")
  VALUES ($1, $2)
ON CONFLICT ("name")
  DO UPDATE SET
    ("name", "count") = ROW (EXCLUDED. "name", "nameCounts"."count" + 1)
  RETURNING to_jsonb ("nameCounts".*) || jsonb_build_object('$action', CASE xmax
      WHEN 0 THEN
        'INSERT'
      ELSE
        'UPDATE'
      END) AS result

[1, "Alice"]

{
  "name": "Alice",
  "count": 2,
  "$action": "UPDATE"
}

deletes

The deletes shortcut, unsurprisingly, deletes rows from a table (delete, unfortunately, is a JavaScript reserved word). It takes the table name and an appropriate Whereable or SQLFragment, and by default returns the deleted rows as a JSONSelectable.

Again, you can narrow or broaden what’s returned with the returning and extras options, as documented above for insert.

For example:

import * as db from 'zapatos/db';
import pool from './pgPool';await db.deletes('books', { title: 'Holes' }, { returning: ['id'] }).run(pool);

DELETE FROM "books"
WHERE ("title" = $1)
RETURNING jsonb_build_object($2::text, "id") AS result

["Holes", "id"]

[
  {
    "id": 1004
  }
]

truncate

The truncate shortcut truncates one or more tables. It takes a Table name or a Table[] name array, and (optionally) the options 'CONTINUE IDENTITY'/'RESTART IDENTITY' and/or 'RESTRICT'/'CASCADE'.

For instance:

import * as db from 'zapatos/db';
import pool from './pgPool';await db.truncate('bankAccounts').run(pool);

TRUNCATE "bankAccounts"

One context in which this may be useful is in emptying a testing database at the start of each test run.

First, we list all our tables. Zapatos provides some utility types such as AllBaseTables, to help ensure that we don’t forget any:

import type * as s from 'zapatos/schema';const allTables: s.AllBaseTables = [
  'appleTransactions',
  'arrays',
  'authors',
  'bankAccounts',
  'bigints',
  'books',
  'doctors',
  'emailAuthentication',
  'employees',
  'nameCounts',
  'numerics',
  'photos',
  'shifts',
  'stores',
  'subjectPhotos',
  'subjects',
  'tags',
  'usedVoucherCodes',
  'users',
];

We can then empty the database like so:

// *** DON'T DO THIS IN PRODUCTION! ***
await db.truncate(allTables, 'CASCADE').run(pool);

select, selectOne, and selectExactlyOne

The select shortcut function, in its basic form, takes a Table name and some WHERE conditions, and returns a SQLFragment<JSONSelectable[]>. Those WHERE conditions can be the symbol all (meaning: no conditions), a SQLFragment from a sql template string, or the appropriate Whereable for the target table (recall that a Whereable can itself contain SQLFragment values).

selectOne does the same, except that it gives us a SQLFragment<JSONSelectable | undefined>, promising only a single object (or undefined) when run.

selectExactlyOne function does the same as selectOne, except that it eliminates the undefined case (to give: SQLFragment<JSONSelectable>). Instead, it will throw an error (with a helpful query property) if it doesn’t find a row.

In use, they look like this:

import * as db from 'zapatos/db';
import pool from './pgPool';// select, no WHERE clause
const allBooks = await db.select('books', db.all).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("books".*) AS result
  FROM "books") AS "sq_books"

[
  {
    "id": 1000,
    "title": "Northern Lights",
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603082+01:00"
  },
  {
    "id": 1001,
    "title": "The Subtle Knife",
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603558+01:00"
  },
  {
    "id": 1002,
    "title": "The Amber Spyglass",
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603732+01:00"
  },
  {
    "id": 1003,
    "title": "The Curious Incident of the Dog in the Night-Time",
    "authorId": 1001,
    "createdAt": "2024-06-23T15:22:52.605245+01:00"
  },
  {
    "id": 1,
    "title": "A Brief History of Time",
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.463945+01:00"
  },
  {
    "id": 2,
    "title": "My Brief History",
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.463945+01:00"
  },
  {
    "id": 3,
    "title": "The Universe in a Nutshell",
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.467708+01:00"
  }
]

import * as db from 'zapatos/db';
import pool from './pgPool';// select, Whereable
const authorBooks = await db.select('books', { authorId: 1000 }).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("books".*) AS result
  FROM "books"
  WHERE ("authorId" = $1)) AS "sq_books"

[1000]

[
  {
    "id": 1000,
    "title": "Northern Lights",
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603082+01:00"
  },
  {
    "id": 1001,
    "title": "The Subtle Knife",
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603558+01:00"
  },
  {
    "id": 1002,
    "title": "The Amber Spyglass",
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603732+01:00"
  }
]

import * as db from 'zapatos/db';
import pool from './pgPool';// selectOne (since authors.id is a primary key), Whereable
const oneAuthor = await db.selectOne('authors', { id: 1000 }).run(pool);

SELECT to_jsonb ("authors".*) AS result
FROM "authors"
WHERE ("id" = $1)
LIMIT $2

[1000, 1]

{
  "id": 1000,
  "name": "Philip Pullman",
  "isLiving": true
}

import * as db from 'zapatos/db';
import pool from './pgPool';// selectExactlyOne, Whereable
// (for a more useful example, see the section on `lateral`, below)
try {
  const exactlyOneAuthor = await db.selectExactlyOne('authors', { id: 999 }).run(pool);
  // ... do something with this author ...

} catch (err: any) {
  if (err instanceof db.NotExactlyOneError) console.log(`${err.name}: ${err.message}`);
  else throw err;
}

SELECT to_jsonb ("authors".*) AS result
FROM "authors"
WHERE ("id" = $1)
LIMIT $2

[999, 1]

NotExactlyOneError: One result expected but none returned (hint: check `.query.compile()` on this Error)

import * as db from 'zapatos/db';
import pool from './pgPool';// select, Whereable with embedded SQLFragment
const recentAuthorBooks = await db.select('books', { 
  authorId: 1001,
  createdAt: db.sql`${db.self} > now() - INTERVAL '7 days'`,
}).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("books".*) AS result
  FROM "books"
  WHERE ("authorId" = $1
    AND ("createdAt" > now() - INTERVAL '7 days'))) AS "sq_books"

[1001]

[
  {
    "id": 1003,
    "title": "The Curious Incident of the Dog in the Night-Time",
    "authorId": 1001,
    "createdAt": "2024-06-23T15:22:52.605245+01:00"
  }
]

import * as db from 'zapatos/db';
import { conditions as dc } from 'zapatos/db';
import pool from './pgPool';// select, Whereables with conditions helpers
const alsoRecentAuthorBooks = await db.select('books', {
  authorId: 1001,
  createdAt: dc.after(dc.fromNow(-7, 'days')),
}).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("books".*) AS result
  FROM "books"
  WHERE ("authorId" = $1
    AND ("createdAt" > now() + $2))) AS "sq_books"

[1001, "-7 days"]

[
  {
    "id": 1003,
    "title": "The Curious Incident of the Dog in the Night-Time",
    "authorId": 1001,
    "createdAt": "2024-06-23T15:22:52.605245+01:00"
  }
]

import * as db from 'zapatos/db';
import type * as s from 'zapatos/schema';
import pool from './pgPool';// select, SQLFragment with embedded Whereables
const anOddSelectionOfBooksToDemonstrateAnOrCondition = await db.select('books', 
  db.sql<s.books.SQL>`${{ id: 1 }} OR ${{ authorId: 2 }}`
).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("books".*) AS result
  FROM "books"
  WHERE ("id" = $1)
  OR ("authorId" = $2)) AS "sq_books"

[1, 2]

[
  {
    "id": 1,
    "title": "A Brief History of Time",
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.463945+01:00"
  }
]

Similar to our earlier shortcut examples, once I’ve typed in 'books' or 'authors' as the first argument to the function, TypeScript and VS Code know both how to type-check and auto-complete both the WHERE argument and the type that will returned by run.

The select and selectOne shortcuts can also take an options object as their third argument, which has a large set of potential keys: columns, order, limit, offset, lateral, alias, extras, groupBy, having, distinct and lock.

columns

The columns key specifies that we want to return only a subset of columns, perhaps for reasons of efficiency. It takes an array of Column names for the appropriate table, and works in just the same way as the returning option on the other query types. For example:

import * as db from 'zapatos/db';
import pool from './pgPool';const bookTitles = await db.select('books', db.all, 
  { columns: ['title'] }).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT jsonb_build_object($1::text, "title") AS result
  FROM "books") AS "sq_books"

["title"]

[
  {
    "title": "Northern Lights"
  },
  {
    "title": "The Subtle Knife"
  },
  {
    "title": "The Amber Spyglass"
  },
  {
    "title": "The Curious Incident of the Dog in the Night-Time"
  },
  {
    "title": "A Brief History of Time"
  },
  {
    "title": "My Brief History"
  },
  {
    "title": "The Universe in a Nutshell"
  }
]

The return type is of course appropriately narrowed to the requested columns only, so VS Code will complain if we now try to access bookTitles[0].authorId, for example. (Note: this works only when strictNullChecks are in operation).

The columns option does not enable column aliasing — i.e. you can’t use it to do SELECT "column" AS "aliasedColumn" or its equivalent — but column aliasing is easily achieved using the extras option instead.

order, limit and offset

The limit and offset options each take a number and pass it directly through to SQL LIMIT and OFFSET clauses. The order option takes a single OrderSpecForTable or an OrderSpecForTable[] array, which has this shape:

interface OrderSpecForTable<T extends Table> {
  by: SQLForTable<T>;
  direction: 'ASC' | 'DESC';
  nulls?: 'FIRST' | 'LAST';
}

Putting them together gives us queries like this:

import * as db from 'zapatos/db';
import pool from './pgPool';const [lastButOneBook] = await db.select('books', db.all, { 
  order: { by: 'createdAt', direction: 'DESC' }, 
  limit: 1, 
  offset: 1,
}).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("books".*) AS result
  FROM "books"
  ORDER BY "createdAt" DESC
  LIMIT $1 OFFSET $2) AS "sq_books"

[1, 1]

[
  {
    "id": 1,
    "title": "A Brief History of Time",
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.463945+01:00"
  }
]

I used destructuring assignment here (const [lastButOneBook] = /* ... */;) to account for the fact that I know this query is only going to return one response. Unfortunately, destructuring is just syntactic sugar for indexing, and indexing in TypeScript doesn’t reflect that the result may be undefined unless you have --noUncheckedIndexedAccess turned on. That means that lastButOneBook is now typed as a JSONSelectable, but it could actually be undefined, and that could lead to errors down the line.

To fix this, we can use the selectOne function instead, which turns the example above into the following:

import * as db from 'zapatos/db';
import pool from './pgPool';const lastButOneBook = await db.selectOne('books', db.all, {
  order: [{ by: 'createdAt', direction: 'DESC' }], 
  offset: 1 
}).run(pool);

SELECT to_jsonb ("books".*) AS result
FROM "books"
ORDER BY "createdAt" DESC
LIMIT $1 OFFSET $2

[1, 1]

{
  "id": 1,
  "title": "A Brief History of Time",
  "authorId": 5,
  "createdAt": "2024-06-23T15:24:10.463945+01:00"
}

The { limit: 1 } option is now applied automatically. And the return type following await needs no destructuring and is now, correctly, JSONSelectable | undefined.

lateral and alias

Earlier we put together some big LATERAL joins of authors and books. This was a powerful and satisfying application of Postgres’ JSON support … but also a bit of an eyesore, heavy on both punctuation and manually constructed and applied types.

We can improve on this. Since SQLFragments are already designed to contain other SQLFragments, it’s a pretty small leap to enable select calls to be nested inside other select calls in order to significantly simplify this kind of LATERAL join query.

We achieve this with an additional options key, lateral. This lateral key takes either a single nested query shortcut, or an object that maps one or more property names to query shortcuts.

lateral property maps

Let’s deal with the latter case — the map of property names to query shortcuts — first. It allows us to write an even bigger join (of books, each with their author and tags) like so:

import * as db from 'zapatos/db';
import pool from './pgPool';const booksAuthorTags = await db.select('books', db.all, {
  lateral: {
    author: db.selectExactlyOne('authors', { id: db.parent('authorId') }),
    tags: db.select('tags', { bookId: db.parent('id') }),
  }
}).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("books".*) || jsonb_build_object($1::text, "lateral_author".result, $2::text, "lateral_tags".result) AS result
  FROM "books"
  LEFT JOIN LATERAL (
    SELECT to_jsonb ("authors".*) AS result
    FROM "authors"
    WHERE ("id" = "books"."authorId")
  LIMIT $3) AS "lateral_author" ON true
  LEFT JOIN LATERAL (
    SELECT coalesce(jsonb_agg(result), '[]') AS result
    FROM (
      SELECT to_jsonb ("tags".*) AS result
      FROM "tags"
      WHERE ("bookId" = "books"."id")) AS "sq_tags") AS "lateral_tags" ON true) AS "sq_books"

["author", "tags", 1]

[
  {
    "id": 1000,
    "tags": [
      {
        "tag": "His Dark Materials",
        "bookId": 1000
      },
      {
        "tag": "1/3",
        "bookId": 1000
      }
    ],
    "title": "Northern Lights",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    },
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603082+01:00"
  },
  {
    "id": 1001,
    "tags": [
      {
        "tag": "His Dark Materials",
        "bookId": 1001
      },
      {
        "tag": "2/3",
        "bookId": 1001
      }
    ],
    "title": "The Subtle Knife",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    },
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603558+01:00"
  },
  {
    "id": 1002,
    "tags": [
      {
        "tag": "His Dark Materials",
        "bookId": 1002
      },
      {
        "tag": "3/3",
        "bookId": 1002
      }
    ],
    "title": "The Amber Spyglass",
    "author": {
      "id": 1000,
      "name": "Philip Pullman",
      "isLiving": true
    },
    "authorId": 1000,
    "createdAt": "2024-06-23T15:22:52.603732+01:00"
  },
  {
    "id": 1003,
    "tags": [
      {
        "tag": "mystery",
        "bookId": 1003
      }
    ],
    "title": "The Curious Incident of the Dog in the Night-Time",
    "author": {
      "id": 1001,
      "name": "Mark Haddon",
      "isLiving": true
    },
    "authorId": 1001,
    "createdAt": "2024-06-23T15:22:52.605245+01:00"
  },
  {
    "id": 1,
    "tags": [
      {
        "tag": "physics",
        "bookId": 1
      }
    ],
    "title": "A Brief History of Time",
    "author": {
      "id": 5,
      "name": "Stephen Hawking",
      "isLiving": false
    },
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.463945+01:00"
  },
  {
    "id": 2,
    "tags": [
      {
        "tag": "physicist",
        "bookId": 2
      },
      {
        "tag": "autobiography",
        "bookId": 2
      }
    ],
    "title": "My Brief History",
    "author": {
      "id": 5,
      "name": "Stephen Hawking",
      "isLiving": false
    },
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.463945+01:00"
  },
  {
    "id": 3,
    "tags": [],
    "title": "The Universe in a Nutshell",
    "author": {
      "id": 5,
      "name": "Stephen Hawking",
      "isLiving": false
    },
    "authorId": 5,
    "createdAt": "2024-06-23T15:24:10.467708+01:00"
  }
]

The result here is a books.JSONSelectable, augmented with both an author property (containing an authors.JSONSelectable) and a tags property (containing a tags.JSONSelectable[] array).

Note that we use selectExactlyOne in the nested author query because a book’s authorId is defined as NOT NULL REFERENCES "authors"("id"), and we can therefore be 100% certain that we’ll get back a row here.

We could of course turn this around, nesting more deeply to retrieve authors, each with their books, each with their tags:

import * as db from 'zapatos/db';
import pool from './pgPool';const authorsBooksTags = await db.select('authors', db.all, {
  lateral: {
    books: db.select('books', { authorId: db.parent('id') }, {
      lateral: {
        tags: db.select('tags', { bookId: db.parent('id') }, { columns: ['tag'] })
      }
    })
  }
}).run(pool);

SELECT coalesce(jsonb_agg(result), '[]') AS result
FROM (
  SELECT to_jsonb ("authors".*) || jsonb_build_object($1::text, "lateral_books".result) AS result
  FROM "authors"
  LEFT JOIN LATERAL (
    SELECT coalesce(jsonb_agg(result), '[]') AS result
    FROM (
      SELECT to_jsonb ("books".*) || jsonb_build_object($2::text, "lateral_tags".result) AS result
      FROM "books"
        LEFT JOIN LATERAL (
          SELECT coalesce(jsonb_agg(result), '[]') AS result
          FROM (
            SELECT jsonb_build_object($3::text, "tag") AS result
            FROM "tags"
            WHERE ("bookId" = "books"."id")) AS "sq_tags") AS "lateral_tags" ON true
      WHERE ("authorId" = "authors"."id")) AS "sq_books") AS "lateral_books" ON true) AS "sq_authors"