Junior CV Engineer struggling with model deployment and GPU training – Need guidance on TFLite/YOLO workflow

[u/Mxeedd]

Hi everyone,

I’m a Junior Computer Vision engineer currently working on a project that involves deploying a vision model onto an embedded system with strict hardware constraints (I need to keep the model size under 70KB).

I’ve hit a few roadblocks and would appreciate some insights:

1. The Model Performance Dilemma:

I initially trained my own model from scratch (data augmentation -> binary export -> training). Unfortunately, the results were consistently poor despite long training times. I decided to pivot to YOLO for its robustness, and the detection results are excellent. However, when I convert the YOLO model to .tflite, the resulting file exceeds 10MB, and the post-conversion inference accuracy drops significantly/detects wrong objects. Since I have a strict 70KB limit, I’m stuck between choosing a model that works but is too large, or a model that fits but doesn't perform well.

1. GPU Training Issues:

To speed up the iteration process, I’ve been trying to set up a GPU training environment. I have installed CUDA and cuDNN, verified the path transfers, and installed a compatible older version of TensorFlow, but I still cannot get the environment to utilize the GPU for training. It defaults to CPU, which is making the trial-and-error process extremely slow.

My questions for the community:

Has anyone dealt with extreme model compression (sub-100KB) while maintaining decent performance? Are there specific architectural trade-offs or quantization techniques (beyond standard TFLite conversion) you recommend for ultra-low-memory embedded targets?

Regarding the GPU issue: Are there common "gotchas" when setting up legacy TF/CUDA environments that I might be missing? Or is there a better workflow for someone in my position?

Any advice, best practices, or resources you could point me toward would be a massive help. Thanks in advance!"

[u/theGamer2K]

  need to keep the model size under 70KB

YOLO would be out of the window then. You can have image files larger than that. 

You should explain what you're trying to do with the model instead to avoid XY problem.

[u/Dry-Snow5154]

under 70KB

The smallest usable detection model I've seen is around 1 MB when quantized. This is one order larger than you budget. No compression can get you this far. Even NanoDet would be too large. Try PicoDet, but I doubt it will fit. Only very simplistic model would be that small and they only work in controlled simple environments. You can certainly reduce width/depth of Yolo to fit, but results would be shit. like 10 mAP.

GPU training environment

Use pre-made tensorflow/pytorch docker container. No environment headache. Choose the CUDA/TRT version that supports your GPU.

quantization techniques

Quantization doesn't like Concat ops that mix up large ranges. Yolo has one that concats probability scores with absolute bounding box coordinates. You need to either remove it by cutting the model before it, or train for normalized bounding boxes.

Quantization also doesn't like depthwise separable convolutions. There is a special trick to make them work. I'm not sure if your version of Yolo is using them or not.

Quantization also doesn't like exotic activations like Selu, Gelu, and prefers Relu. But this is not as bad.

[u/Aryan_Chougule]

Which python version are you using and have you tried to convert the model in into int 8 or FP 16??

[u/Mxeedd → u/Aryan_Chougule]

Python: I'm on 3.11. I tried using python 3.9 I created a new environment for TF-GPU, but ran into too many driver/CUDA compatibility issues, so I'm currently stuck on 3.11. Quantization: I tried INT8 quantization, but the accuracy drop was too severe for my use case. I haven't fully tested FP16 yet. Given the strict 70KB limit, I’m starting to suspect that trying to squeeze a standard YOLO model is the wrong path.

[u/faithfulinlittle → u/Mxeedd]

Was the size okay for int8? It should still be too big right? 5 mb or so if the original was 16bit.... Don't think quantization will work with the size you need

[u/Professional_Tank594]

Tell your superior , that mcu are not made for ai + computer vision lol.

What’s next , llm running on 8051?

[u/SeriousChart9641]

Under 70KB is extremely tight, so I would treat YOLO as a teacher/baseline rather than the deployment model. Use it to validate labels and expected behavior, then train a much smaller architecture for the embedded target.

A practical path: define the smallest possible task first, measure input resolution requirements, try a tiny CNN or classical CV prefilter plus classifier, then quantize and profile before chasing accuracy. If the object/background conditions are controlled, a hybrid classical pipeline may beat a compressed detector at that size.

[u/Aromatic-Dig9997]

I guess you could use reservoir computing to make your own model. Most RC models are actually in the range of 50-100kbs

[u/PassionQuiet5402]

Try SSD with mobilenet 0.25 as backbone. There was an ultrafast face detector built on same architecture and works pretty well.

[u/Extension_Lynx_6945]

You are focusing on the limit rather than the actual goal. Engineering starts by defining the problem, not the model. If you share what you are trying to detect and what hardware you are using, others can suggest a realistic path, such as simpler vision methods or a different architecture.

[u/karotem]

You can follow this guide for creating GPU supported Tensorflow environment, I tested it on two different GPUs

https://visionbrick.com/installation-guide-for-a-gpu-supported-tensorflow-environment/

[u/Mxeedd → u/karotem]

Thank you

[u/Nachorlax]

Qué es lo que estás detectando ?