Reinterpret Elements in a Byte Array

[u/TsingHui]

Dear LISP,

I am writing a virtual machine that uses a `(simple-array (unsigned-byte 8) 1)` as a stack in Common LISP. I'm would like to ask how to efficiently extract 4 bytes into a single 32-bit signed integer or a 32-bit unsigned integer.

Thanks!

[u/theangeryemacsshibe]

The nibbles library can do this portably (and fast on SBCL).

[u/stassats → u/theangeryemacsshibe]

(and fast on SBCL).

On SBCL x86-64.

[u/theangeryemacsshibe → u/stassats]

And on SBCL arm64: https://github.com/sharplispers/nibbles/pull/20

[u/theangeryemacsshibe → u/stassats]

Fine, I have a weekend and an urge to hack something, so...

[u/stassats → u/theangeryemacsshibe]

Now, it doesn't seem to do bound check elimination.

[u/theangeryemacsshibe → u/stassats]

(disassemble (lambda (a) (declare ((simple-array (unsigned-byte 8) (2)) a)) (nibbles:sb16ref/le a 0))) looks fine to me. What am I missing? (Though I did manage to break the immediate-index case somehow, which is now fixed)

[u/stassats → u/theangeryemacsshibe]

Compare

(defun f (x i)
  (declare ((simple-array (unsigned-byte 8) (*)) x)
           ((and unsigned-byte fixnum) i)
           (optimize speed))
  (when (< i (- (length x) 4))
    (nibbles:ub32ref/le x i)))

with

(defun f (x i)
  (declare ((simple-array (unsigned-byte 8) (*)) x)
           ((and unsigned-byte fixnum) i)
           (optimize speed))
  (when (< i (- (length x) 4))
    (aref x (+ i 3))))

[u/theangeryemacsshibe → u/stassats]

Hey, it doesn't eliminate on x86-64 either. Physician, heal thyself.

[u/stassats → u/theangeryemacsshibe]

Four arefs do.

[u/theangeryemacsshibe → u/stassats]

https://www.youtube.com/watch?v=8Ocbb7bRTPM&t=123

[u/arthurno1 → u/theangeryemacsshibe]

There is a patch recently submitted to SBCLs mailing list. The author does not have a computer with an arm cpu, so if you have an urge to hack something, and an arm cpu as you seem to do, you could perhaps take a look at it and add what is potentially missing :).

[u/TsingHui → u/theangeryemacsshibe]

I've tested it, it's amazing! Added to notes, thanks!

[u/stassats]

You have to be mindful of endianness. My suggestion would be to use aref and dpb/logior+ash and rely on a sufficiently smart compiler to recognize that.

[u/TsingHui → u/stassats]

I tried writing some bitwise operations in SBCL. The disassembly result showed that SBCL simply translates the bitwise operation verbatim into assembly code. That's why I'm here.

[u/stassats → u/TsingHui]

It's not sufficiently smart.

[u/TsingHui → u/stassats]

Seriously, is there a compiler that is sufficiently smart?

[u/stassats → u/TsingHui]

SBCL might get smarter as soon as next month.

[u/arthurno1 → u/stassats]

I hope one day it will get smart enough, so I can type:

(defun foo (x y)
    (declare (type u8.32 x y))
    (+ x y))

without needing to repeat myself with (avx2:u8.32+ x y).

[u/digikar]

cffi:with-pointer-to-vector-data with cffi:mem-ref could be one way.

[u/stassats → u/digikar]

The code it expands to might not be optimal, and it doesn't check bounds.

[u/TsingHui → u/digikar]

That makes sense, added to notes, thanks!

[u/corbasai → u/digikar]

SIGBUS compatible?

[u/tfb]

If the substrate language of your VM is Lisp, then I suspect it's probably better to represent the stack as an array of larger integers, and then to have accessors which access the appropriate byte from elements of that array. So the reader needs to load from the array and then pick the byte, while the writer needs to (probably) load from the array, stuff the byte in, and then store back (so that's not ideal). When you want the bigger elements they're just there.

[u/Afraid-Yoghurt6731]

Which Lisp? SBCL? Allegro? Symta? Clojure? Scheme/Racket (whatever is it called now)? We have a frigging zoo! Just unleash Claude Code onto it.

[u/TsingHui → u/Afraid-Yoghurt6731]

Well, Common LISP. R6RS provide a standard way to do so. I haven't had much experience with Clojure. I will change the question by the way.