FlatBuffers and FFI

The short version

• FFI defines how TypeScript calls a C function and receives values such as integers, strings, pointers, and buffers.
• FlatBuffers defines how structured data is laid out inside a byte buffer.

ABI pointer convention

The FFI layer uses two categories of pointer arguments. Getting this wrong produces memory corruption.

Parameter kind	FFI type	JS type	Examples
JS-side buffer the C function reads/writes	FFIType.ptr	ptr(buffer) → Pointer	struct ndb_config *, struct ndb_txn *, struct ndb_filter *, struct ndb_builder *, struct ndb_keypair *
Opaque native pointer returned from a C call	FFIType.u64	bigint	struct ndb *, struct ndb_note *, struct ndb_tags *

Never swap these categories. A JS buffer passed as u64 or a native pointer passed as ptr() will produce silent data corruption or segfaults.

Why profiles use both

The native profile API returns a pointer and a length:

ndb_get_profile_by_pubkey(...) -> pointer + byte length

The bytes are a serialized NdbProfileRecord. Native nostrdb builds and stores that record with flatcc, and the profile database contains the resulting FlatBuffer. packages/bunostrdb/src/ndb.ts therefore:

1. Calls ndb_get_profile_by_pubkey or ndb_get_profile_by_key through FFI.
2. Copies the native bytes into a bounded Uint8Array.
3. Creates a flatbuffers.ByteBuffer.
4. Uses the generated NdbProfileRecord reader from packages/bunostrdb/src/ndb-profile-record.ts.
5. Converts the generated reader object into the public ProfileRecord shape.

The generated file is only the schema reader. The FFI declaration is still required to call the native function and obtain the pointer.

The public adapter type is NdbProfileFields in packages/bunostrdb/src/ndb.ts: one string | null field per schema string, reactions: boolean (schema default true when absent), and record-level received_at/note_key as bigint. BunNdb.profileRecordFromPtr constructs it directly from the generated accessors with no casts. Native behavior verified by spike: all 8 content fields round-trip (including reactions: false), unknown content fields are skipped by the flatcc JSON parser, and received_at is time(NULL) at ingestion — tests assert > 0n, never an exact value.

Why notes do not currently use a generated TypeScript binding

Notes cross the boundary differently. ndb_get_note_by_id returns a pointer to the native struct ndb_note. The TypeScript wrapper either reads that native layout through FFI helpers or asks native nostrdb to serialize the note as JSON and then calls JSON.parse.

That memory is not a serialized FlatBuffer, so generating TypeScript bindings from note.fbs would not decode the value returned by ndb_get_note_by_id. The schema file and the returned runtime representation are separate questions.

When to use FlatBuffers in this project

Use generated TypeScript FlatBuffers bindings when all of the following are true:

• A native API returns or accepts a serialized FlatBuffer.
• TypeScript needs to inspect or construct that buffer.
• The schema is the contract for the bytes crossing the boundary.

Use ordinary FFI declarations and buffer handling when the native API uses:

• Scalar values such as integers or booleans.
• C structs with a stable ABI layout.
• Native pointers with dedicated accessor functions.
• JSON or another text representation.

Do not generate TypeScript bindings for every C header or every .fbs file. Code generation should follow the format actually used by the API crossing into TypeScript, not merely the existence of a schema in the native checkout.

Current generation boundary

The source of truth for the profile record is:

deps/nostrdb/schemas/profile.fbs

The repository contains generated native bindings under deps/nostrdb/src/bindings/ and generated TypeScript readers at packages/bunostrdb/src/ndb-profile.ts + packages/bunostrdb/src/ndb-profile-record.ts (one file per table, flatc >= 25.x). The TypeScript files are committed so normal builds do not require flatc to be installed.

To regenerate the TypeScript bindings, use a pinned flatc version and run:

just gen_profile

(scripts/generate-profile-bindings.sh: runs flatc --ts into a temp dir, fails closed on flatc version drift, copies the per-table outputs to packages/bunostrdb/src/ndb-profile.ts + packages/bunostrdb/src/ndb-profile-record.ts, then idempotently inserts the explicit constructor() {} Bun-coverage patch.)

Do not edit packages/bunostrdb/src/ndb-profile*.ts by hand. Make schema or adapter changes in profile.fbs or packages/bunostrdb/src/ndb.ts, respectively, then regenerate when the schema changes. The constructor() {} lines in the committed file are the script's post-flatc patch, not hand edits — they keep Bun function coverage at 100% (behavior is identical; field initializers still run).

Testing guidance

Generated code should not be tested line by line just to increase coverage. The useful tests are contract tests around the boundary:

• Verify that a real profile returned by native nostrdb is decoded into the expected public ProfileRecord fields.
• Include optional and missing profile fields where the schema permits them.
• Exercise both profile lookup paths: by public key and by profile key.
• Test safety checks around the native pointer, byte length, and maximum copy size in packages/bunostrdb/src/ndb.ts.

A small fixture-based test may also decode a known FlatBuffer directly with NdbProfileRecord, but that is a schema compatibility test rather than a test of generated implementation details. If another native API later starts returning a FlatBuffer to TypeScript, add its schema binding and boundary tests at that point.

Deferred plan: runtime validation at trust boundaries

Status: deferred; revisit when the public JSON/API surface expands or a runtime data-integrity problem is observed.

TypeScript declarations and generated FlatBuffers accessors provide compile-time guidance, but they do not exist at runtime. A runtime schema can therefore add useful protection when data is still untrusted or structurally unknown. The important distinction is where that validation happens:

• Validate raw values before crossing into native code. Examples include parsed event JSON, filter JSON, hex identifiers, URLs, integer ranges, and NIP-05 values.
• Validate structured objects at the public API boundary if the library is promising callers a stable, checked shape.
• Do not automatically wrap every generated FlatBuffers getter. A getter that already returns string | null, boolean, number, or bigint gains little from a second generic type check. Domain checks such as URL or hex syntax may still be worthwhile for selected fields.

Runtime schemas are not a replacement for binary-buffer verification, bounded native reads, ABI checks, or cryptographic validation. In particular, validating an object after an accessor has already read a malformed native buffer is too late to serve as the primary memory-safety check.

Proposed follow-up

When this plan is revisited:

1. Inventory every boundary that currently accepts JSON.parse() output or returns a public structured object.
2. Add schemas only for those boundaries, keeping FlatBuffers readers and FFI primitives lightweight.
3. Decide whether the project needs a schema dependency. Prefer a modular library such as Valibot when dependency/bundle size is the priority, or Zod when API ergonomics and ecosystem familiarity are more important.
4. Keep the schema definitions close to the public types and test valid, missing, malformed, and semantically invalid values.
5. Measure the cost on ingestion and profile/query hot paths before validating every returned field in production.

Until then, continue using focused validators such as assertNoNul, numeric range checks, fixed-length hex checks, bounded native copies, and explicit object construction in packages/bunostrdb/src/ndb.ts.