All data effectively lost

OS: OSX 10.15.2 (19C57)
Browser: Firefox 72.0.2 (64-bit)
Extension: 1.5.3

I had commented on this thread but it appears to be dead at this point: [SOLVED] "requested database object could not be found"

After hitting the 2GB limit in Firefox, I can’t view any of my categories, tags, or bookmarked pages and I constantly get the warning that data may be lost soon.

At this point I just want to get a list of urls and the categories/tags that there were attached to. Backing up to GDrive is useless because Google compartmentalizes app-stored data so it is not accessible to me. Backing up locally requires the external helper app but it does not work.

How can I just get a simple JSON or XML export of urls and tags?

Sorry I missed your response in the other thread.
So there are 2 things you can do now:

  1. Make a copy of the Sqlite and .files file just in case, because we may soon be able to recover those with other means. We are working on an api that would allow you to query the data more effectively from outside of the extension
  2. You can try and purge the images indeed, I’ve tried it with my extension and it worked, but that was chrome and not with a potentially locked database. What you need to do is to go to “about:addons” > “settings” > “debug addons” > “Inspect” on the Memex item > “console” and then type and confirm:
    await storageMan.backend.dexieInstance.table('pages').toCollection().modify({ screenshot: null })

This process can take a while, or will immediately fail because of the storage blockers. In that case we would need to wait a bit until we have the API out for another try. When it is successful the console will after a while spit out a number, which is the amount of pages it processed.

Sorry about all those troubles. You won’t believe how hard it is to make Memex work offline first in the browser.
It’s been very challenging, because browsers sometimes only obey to the standards partially (like you’ve experienced now) or not at all. Some low level database implementations even differ from operating system to operating system. So in essence the browser/operating system combination creates a whole spectrum of different setups we need to take into account. On top of that, unlike with any other development flow, we cannot roll out updates selectively to some users to test, but need to always roll them out to everyone, and we have to wait 3-5 days on every update we push, because the chrome store’s review policy. Anyhow. I feel with you this is a super shitty situation for you and I hope we can resolve it soon.

For anyone else that is inclined, the shortcut for inspecting/debugging this extension in FF is:

about:devtools-toolbox?type=extension&id=info%40worldbrain.io

I ran the update and it appears to have cleared thumbnails. Also ran a count of pages and got 12,009 back. Still getting the missing object error and storage limit warnings but will try a browser restart later today.

Thanks @tautologistics for the updates and the shortcut to get to the console!

Hope it is solved with the restart. Keep us posted :pray: