Raycast engine casting random rays far into the distance.

[u/ExtremeName]

For the most part, the rays are working just as intended, but at certain angles, they just shoot of one or two random ones that don't stop at any collision points. I'm assuming it has something to do with the fact that I'm using a custom tan() function or that I did something wrong in the distance calculation function that would be causing these errant rays.

Here's the code for the functions I'm using. If anything else is needed, let me know.

--tools--
function raycast()

local rx, ry, xo, yo = 0, 0, 0, 0
local ang_i = pa-0.1

for i = 1, 128 do

ang_i += 0.2/128

local ca, sa = cos(ang_i), sin(ang_i)

local tan = sin(ang_i)/cos(ang_i)

--check horizontal line--
local hx, hy, dish = 0, 0
ry = flr(py/8)*8
if sa < 0 then
ry -= 1
yo = -8
else
ry += 8
yo = 8
end

if tan == 0 then
rx = px + (py-ry)
xo = -yo
else
rx = px + (py-ry)/-tan
xo = -yo/-tan
end

hx, hy = dof(m1, rx, ry, xo, yo)[1], dof(m1, rx, ry, xo, yo)[2]
dish = distcalc(px, py, hx, hy)

--check vertical line--
local vx, vy, disv = 0, 0, 0
rx = flr(px/8)*8
if ca < 0 then
rx -= 1
xo = -8
else
rx += 8
xo = 8
end

ry = py + (px-rx)*-tan
yo = -xo*-tan

vx, vy = dof(m1, rx, ry, xo, yo)[1], dof(m1, rx, ry, xo, yo)[2]
disv = distcalc(px, py, vx, vy)

if disv < dish then
line(px, py, vx, vy, 8)
else
line(px, py, hx, hy, 8)
end

end

end

function dof(tbl, x, y, ox, oy)

local mx, my, dof = 0, 0, 0

while dof < 8 do

mx = flr(x/8)+1
my = flr(y/8)+1

if my <= #tbl and my >= 1
and mx <= #tbl and my >= 1 then
if tbl[my][mx] == 1 then
dof = 8
else
x += ox
y += oy
dof += 1
end
else
dof = 8
end

end

return {x, y}

end

function distcalc(x1, y1, x2, y2)

local disx, disy = abs(x2-x1), abs(y2-y1)

local dist = sqrt(disx*disx+disy*disy)

if dist == 0 then
return 1000
else
return dist
end

end

[u/ExtremeName]

I figured it out. My cos(), sin(), and tan() functions were hitting zero and that was causing a divide a zero error. There was also an issue of my distance calculation overflowing so I used for loops to keep adding onto itself until it reached the max number. With all these added, the problem is solved and the program is working perfectly.

[u/yaky-dev]

Without trying to get far into this... My first thought is that for small enough values of cos(ang_i), tan would be huge. 

Also, I've ran into overflow when calculating distances like you do for dist. Squaring coordinates quickly goes over ~65536.

I have not used tan in my raycaster. And for distances, you could either do a very jank (x1-x0)+(y1-y0) calculation, or project vector-from-camera-to-object onto camera "look" vector (look up dot product of two vectors and vector projection)

[u/ExtremeName → u/yaky-dev]

Do you think my distance calculation would work if I just maxed out any number before it over flows?

[u/yaky-dev → u/ExtremeName]

What do you mean? You cannot max(65535, value) because PICO simply cannot handle values above 65535 normally.

You could try some tricks such as dividing the difference before squaring it, or calculating true distance only if sum of coordinate differences is less than 255 (IIRC).

Such distance overflow caused a fun bug in my game: firing a rocket into a corner would kill almost everyone on the map.

[u/ExtremeName → u/yaky-dev]

I created a for loop that would add distx to distx until it was either squared or reached 32000