I'm trying to understand a strange behavior I observed. I ran into this when I started seeing ghost data show up in my application.

It would seem to me that calling toBytes with an empty Uint8Array should always return the same result no matter what previously happened with fromBytes.

Here's a sample.

import * as sd from "sirdez"

const test = sd.use(
  sd.struct({
    foo: sd.string(sd.utf8js, sd.uint8),
  })
)

const objOne = test.fromBytes(new Uint8Array())
console.log(objOne) // -> { foo: '' }

// this taints future calls to fromBytes()
const arrOne = test.toBytes({ foo: "bar" })
console.log(arrOne)

// Why is this returning { foo: 'bar' } and not { foo: '' }
// It seems that the call to toBytes leaves some state hanging around?
const objTwo = test.fromBytes(new Uint8Array())
console.log(objTwo) // -> { foo: 'bar' } 

Using this serde declaration:

sd.use(sd.array(sd.bytes(sd.uint32), sd.uint32));

I get the following error:

RangeError: offset is out of bounds
    at Uint8Array.set (<anonymous>)
    at Object.ser (.../node_modules/sirdez/src/serdes/bytes.ts:9:17)
    at .../node_modules/sirdez/src/serdes/array.ts:10:12
    at contextSer (.../node_modules/sirdez/src/context.ts:27:7)
    at Object.toBytes (.../node_modules/sirdez/src/use.ts:14:34)
    at serialize (.../src/datasources/srs.ts:391:42)
    at CacheClient.set (.../src/lib/redis-cache/cache.js:203:48)
    at CacheClient.tryGet (.../src/lib/redis-cache/cache.js:131:24)

Looking at the source code I should be able to create a context with a larger size than the default value. For that I would expect to call use second parameter, size with a size.

However, the signature available for sd.use in my current project is this:

function use<T>(
  { ser, des }: Serdes<T>
): UsableSerdes<T>

I was expecting to have something like this:

function use<T>(
  { ser, des }: Serdes<T>,
  size?: number
): UsableSerdes<T>

• package version: 0.0.6

What am I doing wrong?

I've noticed that usize is not exported in noeval.ts, even though usize doesn't use eval. I guess it's a bug, because in noeval.ts you had to export every serdes other than struct and number manually, and it's easy to forget to export something.

What about using such folder structure so it's harder to miss something:

• shared_serdes contains serdeses which can be used both in eval and noeval
• serdes contains serdeses which use eval
• noeval_serdes contains versions of serdeses from serdes, but without eval usage

There are 4 index files for every combination Node.js/not Node.js and eval/noeval. shared.ts contains common exports

I ran into a problem with sirdez when trying to run it in the context of a Cloudflare Worker.

Workers (and probably most serverless environments) do not allow eval or new Function calls.

https://github.com/cloudflare/wrangler/issues/1268

Once I narrowed the problem down to sirdez (which I could only see in the worker dev environment, not from production logs) I was pleased to see there was a noeval option.

It took some digging around and experimentation though to figure out how to actually use it.

Having a section of the README dedicated to this would be helpful. Or bring it more up front in the docs.

Note also that Socket.dev calls out sirdez with a warning since it uses new Function.

https://socket.dev/npm/package/sirdez/overview/0.0.6

You might think about having the default be to use the noeval version and let people opt-in to the more unsafe behavior.

You mentioned in the docs that there is a performance hit when using noeval structs.

https://weisrc.github.io/sirdez/data_types/struct.html#variants

much slower because it is implemented using a for loop

I tried to follow the link on the README to Performance Results to better understand the impact but its a broken link.

https://github.com/weisrc/sirdez/blob/main/TODO

Updated code, tests, build configs and docs. Will need to change the major version, as it is a breaking change

Not sure why these files should be ignored by eslint. Do I need to change noeval.ts to eval.ts?

I've simplified code, moved some types to src/types, so typedoc will generate documentation for it, and added docs page for sd.oneOf with hopefully clear explanation

P.S. I think #7 checks failed because I had accidentally used npm i instead of yarn

My try on generic function for use case #6

For the following structure:

const OneOf = sd.oneOf({
  a: sd.uint8,
  b: sd.array(sd.uint8, sd.uint8)
});

It will create Serdes<T> where T is:

type OneOf = {
  type: 'a';
  value: number;
} | { 
  type: 'b';
  value: number[];
}   

P.S. Thank you for such an elegant and nice to use library

Is there a recommended approach to differentiating different structs?

Currently I define a type as the first prop in every struct. And then check the first value in the encoded bytes. So far it works fine, but I don't know if it's a good idea.

const Foo = sd.use(
    sd.struct({
        type: sd.uint8,
        foo: sd.uint8,
        // ...
    }),
);

const Bar = sd.use(
    sd.struct({
        type: sd.uint8,
        bar: sd.uint8,
        // ...
    }),
);

// ...

switch (bytes[0]) {
    case Type.Bar:
        const bar = Bar.fromBytes(bytes);
        break;

    case Type.Foo:
        const foo = Foo.fromBytes(bytes);
        break;

    default:
        break;
}

p.s: thank you for this library, took me forever to find it, but i'm so glad I did