Plugin reference

blob

Helpers for raw byte buffers — build, dump, view by pointer.

8 functionsnamespace blobsource plugins/blob/BlobPlugin.c
= vs :=

Two assignment operators. Plain = grows the target's capacity as needed; := is capacity-bounded — it never grows, fills up to the declared capacity, and sets Overflow to 1 on truncation. With = a shorter source replaces the full value (new length = source length); with := the existing tail beyond the source is preserved.

Use = for general assignment; use := for fixed-size fields (e.g. a string member of a packed struct) where the buffer must not reallocate. See Language basics → Assignment for the full table and a worked example.

Introduction

The blob plugin provides a few convenience helpers for working with binary data (blobs): it turns text into a blob, builds a human-readable hexdump of a blob for inspection, and forms a non-copying (zero-copy) blob reference from a raw memory address. The plugin itself holds no state: every call is standalone and stateless.

What is a blob?

A blob is the language's raw byte-sequence type: it stores arbitrary binary data, including 0 bytes (unlike a string, which ends at the first 0 byte). A local blob must be declared with a size — blob $B[64]; — its length is given by .Length (assignable to shrink it), $B.Fit() trims the capacity to that length, and you access individual bytes with $B[i] indexing. The blob plugin makes working with this type easier.

Loading the plugin

Mind the load path: blob.FromText and the other functions described here are in the blob PLUGIN (../plugins/blob/BlobPlugin), NOT in the builtin:blob module. The latter provides the language's built-in blob operations; the two are not the same.

plugin "builtin:print";
plugin "../plugins/blob/BlobPlugin";

A typical flow

blob $B[64];
$B = blob.FromText("Hello, blob!");
printf("%d bytes\n", $B.Length);
print.Print(blob.ToDump($B));

A blob is also a struct — assign

Everything above treats a blob as a flat run of bytes. The other half of the story is that the same buffer can carry a schema: bind one with the assign statement and the bytes become named, typed members, with no copy and no wrapper object. This is what makes blob.ToDump such a useful companion — you write through the field names and immediately look at the bytes they produced.

plugin "builtin:struct";
plugin "../plugins/blob/BlobPlugin";

struct Frame packet { u8 magic; u8 kind; u16 length; u32 id; }

blob $F[sizeof(Frame)];
$F assign Frame;              // schema binding is a STATEMENT — no '='

$F.magic  = 0xD0;
$F.kind   = 0x02;
$F.length = 260;
$F.id     = 0xDEADBEEF;

printf("size=%d\n", $F.StructSize);
print.Print(blob.ToDump($F));
output
size=8
00000000  D0 02 04 01 EF BE AD DE                           |........        |

The dump is worth reading closely, because it shows exactly what packet bought you. The eight bytes are the four members back to back with no padding — magic, kind, then length and id in little-endian order (04 01 is 260; EF BE AD DE is 0xDEADBEEF). Without the packet qualifier the compiler's natural alignment would have padded length to a 2-byte boundary and id to a 4-byte one, and the frame would no longer match the wire format.

A buffer longer than one element is an array: elements sit back to back with .StructSize as the stride, .Count reports how many fit, and $F[i].member addresses them. Members can also be added or removed at runtime, per schema or per individual blob. The full picture — alignment vs packet, struct arrays, schemas on strings, and the migrate flag — is in Language basics → Structs over blobs (the type itself — indexing, = vs :=, slices, .Fit() — is introduced right before it, in Language basics → Blobs), and the runtime helpers are listed under Built-ins → builtin:struct.

What to know about every function (the basics)

  • Every call is stateless and standalone: no open/close, no handle. The functions are thread-safe.

  • FromText copies the whole text into the blob (its length is strlen, i.e. up to the first 0 byte); a NULL text is treated as an empty text.

  • The output of ToDump / ToDumpWidth is for INSPECTION (a human-readable hexdump), not for data interchange. For round-trippable encoding, crypto.ToHexBlob / FromHexBlob are the right choice.

  • The hexdump layout follows the classic `hexdump -C` / `xxd` format: an 8-digit offset, uppercase hex bytes (with an extra gap at the half-width point), then the ASCII rendering between | marks.

  • The width of ToDumpWidth must be between 1 and 256; outside that it raises a runtime error.

  • RefFromPointer is an unsafe, low-level primitive (see the function's description); most scripts never need it.

How to read the signatures

The type after the -> arrow is the return type. For example, blob.ToDumpWidth(data, width) -> string takes a blob and an integer width and returns a string (the hexdump).

Creation

Producing a blob from text, or from a raw memory address.

blob.FromText

blob.FromText(text) -> blob

Creates a new blob whose contents are the bytes of the given text. This is the usual way to turn a string into a blob.

Parameters
Parameter Type Description
text string The text to turn into a blob. The whole text is copied; NULL behaves as an empty text.
Return value

A new blob with the text's bytes. Its length is the length of the text (up to the first 0 byte).

Example
blob $B[64];
$B = blob.FromText("Hello, blob!");
printf("%d\n", $B.Length);
Output after running
12
When to use

When you must pass a text to a function that expects a blob (such as hash or network operations), or you want to inspect a text's contents at the byte level with ToDump.

blob.RefFromPointer

blob.RefFromPointer(pointer, size) -> blob

Forms a non-copying (zero-copy) blob reference to an existing memory area. The returned blob does NOT own the data: when it is freed, only the wrapper is freed, the referenced bytes stay untouched.

Parameters
Parameter Type Description
pointer int The raw address of the memory area as an integer.
size int The size of the area in bytes (cannot be negative; if non-zero, the pointer cannot be 0 either).
Return value

A non-owning blob pointing at the given memory. A negative size, or a null pointer with a non-zero size, causes a runtime error.

Example
// A low-level, unsafe primitive.
// Typically used by plugin authors who export a raw
// pointer+size pair (e.g. a video frame buffer) so the
// script can access the bytes without a copy.
// Ordinary script code almost never needs it.
Output after running
(no runtime output — see the description)
When to use

Only when you received a raw pointer+size pair from a plugin and want to access the bytes without a copy. IMPORTANT: you are responsible for ensuring the referenced area outlives the blob pointing at it — otherwise you cause a use-after-free. If you are not sure you need this, use FromText or copy the data instead.

Inspection

Building a human-readable hexdump of a blob for debugging.

blob.ToDump

blob.ToDump(data) -> string

Builds a human-readable hexdump of the blob in the classic `hexdump -C` / `xxd` layout, with 16 bytes per row.

Parameters
Parameter Type Description
data blob The binary data to display.
Return value

The hexdump as a string: per row an 8-digit offset, the bytes in uppercase hex (with an extra gap at the half-width point), then the ASCII rendering between | marks.

Example
blob $B[64];
$B = blob.FromText("Hello, blob!");
print.Print(blob.ToDump($B));
Output after running
00000000 48 65 6C 6C 6F 2C 20 62 6C 6F 62 21 |Hello, blob! |
When to use

For a quick, eyeball inspection of binary data (file contents, a network packet, a hash) while debugging. For inspection only — for machine data interchange or round-trip encoding use crypto.ToHexBlob / FromHexBlob.

blob.ToDumpWidth

blob.ToDumpWidth(data, width) -> string

Like ToDump, but you choose the number of bytes per row.

Parameters
Parameter Type Description
data blob The binary data to display.
width int The number of bytes per row (between 1 and 256).
Return value

The hexdump as a string, with the given row width. A width outside the range causes a runtime error.

Example
blob $B[64];
$B = blob.FromText("Hello, blob!");
print.Print(blob.ToDumpWidth($B, 8));
Output after running
00000000 48 65 6C 6C 6F 2C 20 62 |Hello, b|
00000008 6C 6F 62 21 |lob! |
When to use

When 16 bytes per row is not ideal — for example for narrower output (8 bytes), or to align the rows to a particular record size for easier reading.

Practical notes

What the blob plugin is good for

  • Turning text into a blob so you can pass it to functions that expect a blob (hash, network, file).

  • Eyeball inspection of binary data: the hexdump immediately shows the bytes and the ASCII content.

  • Customizing the row width in the hexdump when 16 bytes per row does not match the data's structure.

  • A special, low-level case: wrapping a raw pointer+size pair into a blob, without a copy (RefFromPointer).

Inspection vs interchange

The output of ToDump / ToDumpWidth is meant for humans and is NOT suitable for data interchange or reading back: it contains the offsets, the gaps, and the ASCII column. If you want to store or transmit a blob in text form and then convert it back exactly, use the pair crypto.ToHexBlob (blob → hex string) and crypto.FromHexBlob (hex string → blob).

Safety — RefFromPointer

RefFromPointer wraps a raw memory address into a blob without a copy, and does not own the data. This is fast but dangerous: if the referenced area is freed or becomes invalid while the blob still points at it, a use-after-free occurs. Use it only when you know exactly how long the referenced memory lives; otherwise FromText gives a safe copy.

Error handling

A wrong argument count or type, an out-of-range hexdump width (1..256), an invalid pointer/size pair for RefFromPointer, and out of memory are reported by the runtime as a runtime error, and the script stops. FromText and the hexdump functions always give a deterministic result for a valid input; they have no “neutral-return” error state, so it is worth ensuring the validity of the input beforehand.

blob.Fill

blob.Fill(ref $B, start, count, value) -> int

Sets count bytes from start to value (0..255) — a bounds-checked memset on the script’s own blob, in place (the blob is a ref, nothing is copied). Returns the count. A range past the end is refused whole, with the numbers in the error; a zero count is fine. Since rev31zzug.

blob $Frame[64];
i64 $N = blob.Fill(ref $Frame, 16, 32, 0xFF);   // bytes 16..47 := 0xFF

blob.FillStep

blob.FillStep(ref $B, start, step, count, value) -> int

Like Fill, but writes every step-th byte: start, start+step, … (count of them). The strided fill a sieve or a periodic pattern needs — the C loop CPython gets from a slice assignment. The last index must be inside the blob (checked before any write); step is at least 1.

// mark the multiples of $I in a sieve of $N bytes
i64 $M = blob.FillStep(ref $Sieve, $I * $I, $I, ($N - 1 - $I * $I) / $I + 1, 1);

Note: a call may stand on its own as a statement (blob.Fill(ref $B, 0, 4, 0);) — a statement that starts with blob.Name( is a namespace call, not a blob declaration. The assignment form ($R = blob.Fill(...)) works too, and the verbs return a useful number anyway.

blob.CountByte

blob.CountByte(data, value) -> int

Counts how many times the byte value occurs in the blob: it walks every byte once and returns the number of bytes that are equal to value (blob.CountByte($B, 0) is the number of zero bytes in $B, blob.CountByte($B, 65) the number of 65s). It reads only — the blob is passed by value, nothing is written — and it looks at every byte of the blob’s declared size, not just the part a struct schema names. value must be a byte, 0..255; anything else is an error (blob.CountByte: the byte value must be 0..255). An empty blob gives 0.

It is the counting half of the bulk-byte quartet: Fill/FillStep write a pattern, CountByte tells how many bytes carry a value, FindByte tells where the first one is. The typical use is a flag array kept in a blob, one byte per item, where the answer is “how many are still 0” or “how many are marked”:

blob $Flags[1000];
blob.Fill(ref $Flags, 0, 1000, 0);          /* all clear */
blob.FillStep(ref $Flags, 0, 7, 143, 1);     /* every 7th byte := 1 (143 of them) */
i64 $Marked = blob.CountByte($Flags, 1);     /* 143 */
i64 $Clear  = blob.CountByte($Flags, 0);     /* 857 */
if (blob.CountByte($Buf, 0) == $Buf.Length) { /* the buffer is still all zero */ }

It replaces this script loop, which gives the same number but byte by byte in the interpreter:

i64 $N = 0; i64 $I = 0;
while ($I < 1000) { if ($Flags[$I] == 1) { $N++; } $I++; }

The difference is one C pass over the bytes versus one interpreted iteration per byte — on the benchmark sieve the prime count is blob.CountByte($Sieve, 0), and the loop it replaced cost about as much as the whole marking phase. Use it whenever the question is “how many bytes equal X”; when the question is “where is the first X”, use FindByte, and when it is “how many are not X”, subtract the count from $Flags.Length.

blob.FindByte

blob.FindByte(data, value, from) -> int

The index of the first byte equal to value at or after from, or -1 (a memchr). A from outside the blob returns -1, not an error.