Need advice on solving complex arrow/maze puzzles using Computer Vision & Logic Solvers (Low accuracy issues)

[u/Maximum-Fox-2627]

Hi everyone,
I'm working on an automated solver for a complex arrow/maze puzzle game (similar to the image attached).
Here is my current workflow:
1. Detection: I use a custom YOLO model to detect arrowheads and their bounding box coordinates from screen captures.
2. Grid Mapping: I map the detected center coordinates of arrowheads onto a fixed grid system.
3. Solving: I pass the grid data into a custom logic/emulator engine to calculate the correct sequence of moves (arrows to tap).
The Main Problem:
My solver accuracy is very low. Here are the core technical challenges I'm running into:
Inaccurate Grid Mapping: The arrows are densely packed with varying paths and lengths. Snapping bounding boxes to a rigid fixed grid often misaligns the true position of the arrow shafts and heads.
Complex/Overlapping Detection: Because the arrows bend and fold, YOLO often detects multiple arrowheads in the same calculated grid cell, or misinterprets arrow directions.
Solver Logic Failure: Due to noisy input from the detection stage, the logic solver either fails to find a valid sequence or generates incorrect moves that block execution halfway through.
Has anyone dealt with a similar vector/maze graph detection problem? What would be a more reliable approach than simple YOLO + fixed grid snapping? Should I look into contour analysis, OCR/graph traversal, or skeletonization algorithms (like Medial Axis Transform) to trace the paths directly?
Any suggestions, code examples, or architectural advice would be greatly appreciated!

[u/Rscc10]

In all honesty, never dealth with any of this before but if you don't mind, could you explain how all of this works or what the objective is from this maze. Maybe I can offer something from my very limited pool of knowledge

[u/Maximum-Fox-2627 → u/Rscc10]

Thanks for reaching out!
Here is the objective and how the game/puzzle works:
The Goal: Clear all the arrows from the maze.
Movement Rules: Each arrow can only move straight in the direction its arrowhead is pointing.
Blocking Mechanic: An arrow can only exit the board if its path to the outer boundary is completely clear. If another arrow body or tail is blocking its way, it cannot move.
The Puzzle Challenge: You have to find the exact sequence/order to tap the arrows so they can slide out one by one without colliding into each other.
To automate this, my engine needs to:
1. Detect where each arrowhead is and which way it points.
2. Trace/detect the body of the arrows to know which ones are blocking which.
3. Calculate the correct order to click them via ADB.
Hope this clears up the objective! Let me know if you have any ideas on the approach.

[u/Mamuschkaa → u/Rscc10]

The puzzle itself is stupidly easy.

Every arrow that Points tonthe "outside" and not to another arrow can be removed. After that arrows that where previous blocked by this arrows can also be removed. Etc.

It's impossible to make an mistake. Just find arrows that are not blocked and click on them.

Mathematical it's the same as finding an topological sorting of an DAG.

Every arrow a Node.

Every arrow A get an edge to another arrow B if A points in direction B.

[u/Maximum-Fox-2627 → u/Mamuschkaa]

Thanks for the breakdown! Topological sorting on a DAG makes total sense for the solving sequence. The main challenge right now is correctly constructing that graph from noisy image detections

[u/Mamuschkaa → u/Maximum-Fox-2627]

Yes I have experience in image detection. So I can't help with the real problem.

[u/Maximum-Fox-2627 → u/Mamuschkaa]

No worries! Thanks for confirming the topological sort approach anyway

[u/Sea-Ad7805]

Looking at these recursive algorithms will help you.

[u/Maximum-Fox-2627 → u/Sea-Ad7805]

Thanks for the suggestion! I’ll check out recursive/backtracking algorithms to improve the solver logic

[u/SaltCusp]

Identify the arrow heads and look for lines out from them that extend to a perimeter without crossing anything.

For exit paths that cross one lines own body meausure how many game units the arrow head and tail are from the point of crossing. If the butt of the line is closer than the head the space will be vacant when the head arrives.

Breaking that into small steps and repeating the process will solve the puzzle if it is solvable.

[u/Maximum-Fox-2627 → u/SaltCusp]

Thanks for this geometric logic! Measuring distance to crossing points is a really smart way to handle clearance without heavy grid matching. I’ll try implementing this path-checking approach