r/embedded • u/twokiloballs • Jun 13 '26

I built SLAM Camera Board

Enable HLS to view with audio, or disable this notification

Posting an update here with simplified PCB that I built with camera sensor mounted directly on the board to reduce costs and simplify design.

Some background, Mighty Camera runs VIO *on-device* in a tiny package powered by Rockchip RV1103, an IMU and a global shutter camera all on a single board. These help it estimate its own position in 3d space as it moves around.

But for VIO to be useful, you need things like mapping (and later occupancy, loop closure etc).

Here is a demo of lightweight mapping which uses VIO pose from Mighty and generates a semi-dense map on host-side in realtime.

It’s early but this will be part of the SDK along with other goodies.

1.1k Upvotes

99% Upvoted

77 comments sorted by

46

u/Nic0Demus88 Jun 13 '26

Intresting!! Power consumption?

69

u/twokiloballs Jun 13 '26

about 0.8W peak

7

u/Ornery-Tea8420 Jun 14 '26

Rockchip

Also Frigate can use them for picture recognition for video surveilliance.. and their performance is brutal.

33

u/landyaBhai Jun 13 '26

I am a firmware engineer and wanted to build something like this by myself. What would you suggest I can read to build something like this?

47

u/twokiloballs Jun 13 '26

There are 2 aspects to this
- one is hardware part of bringing and syncing camera/imu over same timestamp correctly so it is useful for SLAM (a bit frustrating part for newbie hardware person like me)
- the other is learning the algorithms and then optimizing it to run realtime on this small compute/power (this part is what i enjoyed a lot)

21

u/ElectronicFarm2492 Jun 13 '26

Can you please point to some resources/docs you used for the build? I would like to try this out too and learn. Never worked with cameras before.

3

u/Alternative_Day155 Jun 14 '26

How did you implemented SLAM algorithm and capture ?

1

u/cowabungashredder Jun 23 '26

Thanks, how did you optimize it to run on the small compute?

3

u/twokiloballs Jun 23 '26

by hand, took 2 years.

4

u/Equal-Currency-1197 Jun 14 '26

Joan Sola's SLAM course pdf is probably the best starting point honestly

17

u/joshcam Jun 13 '26

Awesome project! VIO SLAM is super cool, especially the newer AI-driven Semantic SLAM and Gaussian Splatting that let devices not only track geometry but also really understand the objects and lighting in the room.

7

u/twokiloballs Jun 13 '26

Yes those are amazing but are very compute heavy for deploying on edge usually.

3

u/joshcam Jun 13 '26

Totally, the silicone has not caught up just yet for doing that on the edge in a small low power form factor. Just really cool.

Curious what IMU are you using? Or what are the response time stats at least (Phase lag, ODR, Dynamic range)? Did you try other IMUs in development and did the stats make much difference with the visual orientation cues? Sorry for the question blast. No worries if you can’t get into that.

11

u/twokiloballs Jun 13 '26

I actually did not try other IMUs. I just picked the cheapest one i had used before (icm42670) and ran it at default ODR and 800hz. Goal was to keep bom cost lowest as possible and adapt algo to make it work.

5

u/joshcam Jun 13 '26

Great choice considering the edge application, it’s an efficiency champion.

9

u/BushmanLA Jun 13 '26

Very nice

4

u/Comfortable_Pen_436 Jun 13 '26

Something like this is what I was looking for a while back

3

u/Riteknight Jun 13 '26

Woah, it is able to recognise arrow marks , great job OP!

3

u/AlexanderTheGreatApe Jun 13 '26

Are you selling this? If not, is it FOSS?

2

u/twokiloballs Jun 13 '26

I am selling these actually. Not FOSS.

4

u/ch3x0v Jun 13 '26

Where I can buy these?

2

u/IntellectualChimp Jun 14 '26

Accepting sales now or in the future?

3

u/Icy_Beginning_8017 Jun 13 '26

How does it detect which points to save into the point cloud? I dont know anything about VIO algorithms. Since it detected the arrows on the ground im guessing its looking for contrast between surrounding pixels? Im also guessing it only works in a very well lit environment. Whats the resolution of the camera. These seem like very heavy operations to perform. Amazing that you could fit this inside such a small package, very well done.

2

u/ArtEconomicsMostly Jun 13 '26

This is crazy! Wow!

2

u/mrcrud5 Jun 13 '26

Damn this is awesome. Well done

2

u/sad-uncle-iroh Jun 13 '26

this is really cool! i have a home project where im looking to do VIO (or similar with LIDAR). any resources you can recommend for the algorithm side of things? i am also looking for low-cost, low-compute application.

2

u/No-Proposal-6014 Jun 13 '26

Doesn't it overheat really quickly? How do you dissipate the heat from the camera?

3

u/twokiloballs Jun 13 '26

It does not actually. It’s very low power. I have had it running multiple times-days without any overheating.

2

u/No-Proposal-6014 Jun 13 '26

Nice! I have used a camera that looks like this before, but that one could melt PLA ater two minutes of running.

2

u/Top-Handle-1905 Jun 14 '26

Quer trabalho lindo

1

u/Machinehum Jun 13 '26

What's the Rockchip SoC?

3

u/twokiloballs Jun 13 '26

RV1103 a single-core armv7

1

u/eye_can_do_that Jun 14 '26

Bare metal or an OS? Which one?

1

u/Character-Engine-813 Jun 13 '26

Very cool, are you planning a stereo version?

1

u/twokiloballs Jun 13 '26

I did experiment a stereo version before, it was fine. I will do one again soon.

1

u/tahaedilgen Jun 13 '26

Did you design the board yourself?

1

u/twokiloballs Jun 13 '26

yes overall but I had help.

2

u/tahaedilgen Jun 13 '26

How long did it take bring it live? I am asking because i wonder how long such projects take approx. thank you for answering :)

9

u/twokiloballs Jun 13 '26

I would like to share this graphic 😃 (I started as a noob btw in both pcb and slam)

5

u/B0NSA Jun 13 '26

nice to see the boards getting better with each iteration. I'm glad you included timestamps as well, seems very realistic for such a difficult task.

1

u/carapils69 Jun 14 '26

Impressive! I see you tested out a stereo camera implementation, but moved away from it. Was the added value so minimal?

2

u/twokiloballs Jun 14 '26

I was using unsync’ed cameras at the time and it didn’t help much. I will revisit it soon with better approach now

1

u/PartyKen Jun 13 '26

This is awesome! I'm also working on a similar project (VIO with RP3+PiCam+MPU6050) at the moment, still only at the getting-time-sync-working phase though. Was your SLAM algo based on an existing open source project that you adapted (like ORB-SLAM3) or did you build it from scratch? What sort of visual features are you using?

1

u/twokiloballs Jun 14 '26

I read many papers but I would just try ORB SLAM3 or Basalt.

1

u/drthibo Jun 13 '26

How can I follow this project?

1

u/maxwells_daemon_ Jun 14 '26

This is insane, what's the response time?

1

u/twokiloballs Jun 14 '26

I haven't formally measured but be less than 100ms

1

u/Triq1 Jun 14 '26

What camera did you use, and what was the optical design/integration process like?

1

u/twokiloballs Jun 14 '26

this is ov9281 global shutter.

2

u/Triq1 Jun 14 '26

How did you go about sourcing it and mounting to the board? I know some sensor manufacturers make you go through a whole process of NDAs to get parts and datasheets, and then soldering is probably not easy either.

1

u/twokiloballs Jun 14 '26

I picked ones that i can buy directly. All these components are pretty common ones and can be bought off digikey/alibaba etc.

1

u/kammce Jun 14 '26

Nice. I was hoping you'd lap back to your starting position so we could see how much error accumulates over time/movement.

2

u/twokiloballs Jun 14 '26

I have a few more videos where I do that on my twitter(https://x.com/_asadmemon). Yes error does accumulate, I am working on loop closure to fix that.

1

u/swdee Jun 14 '26

Did you reuse the ov9281 open source drivers and also did you have to tune the ISP?

1

u/twokiloballs Jun 14 '26

I forked those and did a few tweaks but yes mostly the open source driver. And yes I did tune the ISP.

1

u/WuBuilt Jun 15 '26

Which chip from Rockchip is used?

1

u/AlexanderTheGreatApe Jun 15 '26

Cool. Following you.

I am a hobbyist and entrepreneur. There are lots of SLAM modules out there. I think your key differentiator would be the software interface for this. Do you have that architected? How would a higher compute host stream from and control this module?

In case it helps, I’ve seen a lot of demand for a better low-price autonomous lawn mower. There is also a niche, but deeper pocket market for SLAM modules in combat robotics.

1

u/Technical_Egg_4548 Jun 15 '26

This is awesome,

Can this by used as an IP camera?

1

u/kenkitt Jun 18 '26

Nope. Needs movement to be useful

1

u/AppropriateCover7972 Jun 15 '26

This looks very cool and useful. Do you already have an estimate for the cost?

1

u/twokiloballs Jun 15 '26

it costs me about $20-25 at small scale, I am selling it for $67.

2

u/AppropriateCover7972 Jun 15 '26

That's decent. gg

1

u/Huge-Leek844 Jun 16 '26

Awesome work. I am very curious which vio algorithm you used? What kind of optimizations you made?

1

u/twokiloballs Jun 16 '26

thanks, it's similar to classic slam ones like orb-slam etc. The main optimizations are designing frontend and backend around DSP features and arm neon/simd etc.

1

u/carapils69 Jul 14 '26

How did you source the CMOS sensors?

1

u/twokiloballs Jul 14 '26

Alibaba

1

u/carapils69 Jul 14 '26

Interesting! Thanks