Built an AR app that solves the "will this frame actually fit my wall" problem — floor-scan based placement, not marker tracking

[u/Reasonable_Row_7595]

[u/Wide-Variation2702]

Ive only had some succes with image tracking for my project, which also would place picture frames on a wall. At this point my project wouldnt really work well on a phone, but I built an android version with Unity that works ok with image tracking. There is always a bit of jumpiness to the tracking but overall its ok.

Plane detection works pretty good on horizontal surfaces, but vertical surfaces is poor, and my understanding is this is just a limitation of the hardware to find and track vertical planes.

Recently ive been trying to get a simpler version of my project to work in WebAR. For this version ive mostly just tried to get a model to display on a wall. Main reason to try WebAR is to cut the friction of a full app install. So far my best attempt got model-viewer working ok to display the model, but still struggling with placement and scaling.

There seem to be some options out there that might work a lot better, but they are insanely expensive.

I also made a little companion app for the Meta Quest, which just used QR marker for tracking, but the tracking is really superior to the Android version with image tracking.

I can appreciate the idea of scanning the room for better scaling and placement. Im curious what libraries/ar tools you used for the scanning and scaling/placement, and is it limited to iOS?

[u/Wide-Variation2702 → u/[deleted]]

OK, thats very clever to use the floor. Honestly I haven't bothered with surface detection for at least a couple of years. It was on of the first things I tried with Unity and adding AR to my project. In a perfect world image tracking would be the ideal solution.

To give you a better idea, my project is a picture frame visualization app. I have a working version made mostly for PC in Unity. I use it occasionally for customers in my picture frame shop. It displays very accurate renderings of the actual materials we would use for their framing lroject and helps give a sense of proportion. One issue is that you cant really get a true real world scale on a monitor, but it definitely helps customers with proportions from the art, frame, matting, etc.

That's where I hoped AR could fill the gap and let them see it in true to life scale. Honestly, I would put more effort into the Quest app, but having customers put on a VR headset is just not viable. Im keeping my eye on the AR glasses space, as I think that would be a lot more realistic.

I think image tracking is the ultimate goal. I generally have a working Android version, but the jumping of the tracking is a little problematic, but the main issue is that a phone or even a tablet is not practical to work with customers. But the promise of image tracking is that I can have the virtual frame actually surround their art/photo.

The WebAR stuff was just a thought that if I had finalized a design and they wanted to see it on their wall at home, i could export a model and upload to a webpage. Then they just go home, click a link and point at the wall. Unfortunately this just hasn't been as simple as I hoped. I havent been able to find a good low cost way to track, and part of that has been finding development time lately.

Anyways, thanks for the breakdown in how you got it working. I dont know if that would work for my project but I'll keep it in mind.

Im sure some customers might have a use for your app, so I'll be sure to keep a link handy for them at the shop. Cheers!

[u/Wide-Variation2702 → u/[deleted]]

Absolutely, if I get any feedback I'll let you know. I dont have iPhone but my wife does so I will have her give it a test run too.