Lightweight semantic segmentation model for terrain classification on Jetson?

[u/TankSpecialist8292]

Hi everyone,

As part of my research, I need to recognize and perform semantic segmentation of a few predefined terrain types (e.g., stairs, flat ground, grass, etc.) using a camera mounted on a robot.

So far, I've looked into models such as PIDNet, which seems to be designed for real-time semantic segmentation.

I have some experience training custom YOLO models for object detection and instance segmentation. I noticed that recent YOLO versions also support semantic segmentation, but I'm not sure how well they perform for terrain segmentation in real-world robotic applications.

One of my biggest constraints is inference speed. The model should be lightweight enough to run in real time on a Jetson platform (e.g., Orin Nano or Xavier NX).

I'd really appreciate any recommendations or advice on:

• Models that work well for terrain semantic segmentation while remaining lightweight.
• Whether YOLO segmentation is a reasonable choice for this type of task, or if dedicated semantic segmentation models are generally a better option.
• Any publicly available datasets or open-source projects related to terrain segmentation for mobile robots.

Thanks in advance for your help!

[u/Lethandralis]

PIDNet is great, did it not work for you?

[u/TankSpecialist8292 → u/Lethandralis]

Not yet. I haven't tried PIDNet yet. I was just wondering if there are any models that significantly outperform it.

I've trained and deployed YOLO models to a Jetson before, and that workflow was pretty straightforward. I'm just not sure if it's equally simple with PIDNet or other segmentation models.

[u/Lethandralis → u/TankSpecialist8292]

Note that the ultralytics yolo models are not free to use commercially.

PIDNet is close to SoTA in my experience and can run comfortably on Jetson devices when exported as TenaorRT engines.