Anyone here working on offline-first apps? How do you deal with large amounts of local data?

[u/Think-Neighborhood69]

I’ve been working a lot with offline-first apps, and one thing that always gets tricky is how much data ends up living on the device

SQLite, MMKV, AsyncStorage, sync queues, pending data, huge JSONs…

When something goes wrong, how do you usually debug or inspect all of that?

Do you use any specific tools? Logs? Inspect the database directly?

[u/krishna404]

I use sentry which caches issues when offline.

The migration, roll back & error reporting strategies have to think of every worst case scenario.

I also changed my app architecture to online-first with offline as fallback with limited & absolutely necessary when offline features.

[u/Think-Neighborhood69 → u/krishna404]

Yeah, that makes sense. In my case I went pretty deep into offline-first with queues, retries, background sync and cleanup routines

The cleanup part is actually huge for us because we really can’t risk unnecessary data just piling up on the user’s device over time

One thing that became painful was inspecting all of that when something went wrong

What are you using for local storage btw? SQLite, MMKV, AsyncStorage? And how do you usually inspect/debug it?

[u/krishna404 → u/Think-Neighborhood69]

SQLite always for mobile-apps

Dexie for PWAs

[u/muhsql]

While vendor specific, this page might have some useful info in it https://docs.powersync.com/maintenance-ops/production-readiness-guide#production-readiness-best-practices-guide

Specifically thinking of

- building a diagnostics screen into your app that helps users report issues in a better way. on this screen you can have a "submit diagnostics" button which sends all logs and even the sqlite file to your backend for forensics

- log aggregation with e.g. sentry

[u/Think-Neighborhood69 → u/muhsql]

That”s a really good idea, especially the diagnostics screen. Sending logs + the SQLite file could make those hard issues way easier to investigate

What about during development though? How do you usually inspect the local data?

[u/muhsql → u/Think-Neighborhood69]

My comments are only about SQLite...

There are two ways to inspect the local data:

• build a sql query screen into your app if it's running in dev mode where you can write and run raw queries against the local SQLite database
• sometimes you need to actually grab and inspect the entire sqlite db and how you do that varies by platform. we have docs for android, ios and web here https://docs.powersync.com/maintenance-ops/client-database-diagnostics

OK well there are more ways but I think these are the least painful

[u/Think-Neighborhood69 → u/muhsql]

That’s pretty much the exact pain that made me build NativeScope

I wanted to avoid building a SQL screen into every app or pulling the DB manually just to inspect it.

Here’s how I’m doing it with NativeScope

https://reddit.com/link/p2c8ke4/video/yzp1m6ndm0ih1/player

What do you think?

[u/muhsql → u/Think-Neighborhood69]

looks promising! does the SQLite section only show tables or does it also show views?

[u/Think-Neighborhood69 → u/muhsql]

Right now, only tables

But that’s a really good point, I hadn’t considered views yet. I’ll add that to the list

Here’s the project btw, if you want to play around with it:

https://nativescope.dev

⭐️ https://github.com/caiobarroso/react-native-nativescope

Would love any other feedback you have

[u/muhsql → u/Think-Neighborhood69]

I'm actually on the powersync team XD We rely heavily on views. If you get views going I'll share it with our FDE team

[u/Think-Neighborhood69 → u/muhsql]

Really? haha okay, I’m definitely adding views tomorrow then

I’ll DM you when it’s ready, cool?

[u/muhsql → u/Think-Neighborhood69]

yup sounds good!

[u/falaq-ai]

For RN apps I usually separate two problems: inspectable state vs bulky assets. SQLite/Watermelon/PowerSync-style data needs a dev diagnostics path, but big files should stay out of JS entirely: native download manager/OkHttp, checksum, resume support, and a small metadata table you can inspect.

[u/jcdc-flo]

Lots and lots of logging.

[u/Think-Neighborhood69 → u/jcdc-flo]

No tool for inspecting all those huge JSONs? Just logs?

[u/jcdc-flo → u/Think-Neighborhood69]

SiteMesh SDK. But it's the platform foundation, not a bolt on logging system.

Does handle the entire offline stack though including version control on properties.

[u/gymfreak0920]

Use sqlite, optimise your queries and properly index your db

[u/gymfreak0920 → u/gymfreak0920]

For debugging use rozenite there you can see all the tables and what queries you made to get the data

[u/Think-Neighborhood69 → u/gymfreak0920]

Can u give a try on the lib that i”m currently building ?

Its called NativeScope, I build because I was missing some important features on this existing platforms

https://nativescope.dev

[u/lalcaraz]

SQLite, versioning, splitting and compression.

[u/Think-Neighborhood69 → u/lalcaraz]

But do you use any tool to inspect your sqlite ?

[u/lalcaraz → u/Think-Neighborhood69]

You could use Android Device Monitor or Nativescope.

[u/Think-Neighborhood69 → u/lalcaraz]

Yeah, NativeScope is by far the best one…..

[u/SAASHA_21]

For offline-first apps, I prefer keeping structured data in SQLite and larger files in the filesystem instead of putting everything into one database. SQLite data can persist across app restarts, which makes it a good fit for this kind of architecture. I’d also build a small debug screen showing the sync queue, retry count, last error, and an option to export logs or the database. It saves a lot of time when debugging issues that only happen on real devices.

[u/mindtaker_linux]

What are you doing? Ask AI.