Mass file retrieval failures (parts.mismatch, MESSAGE_IDS_EMPTY) with large dataset (~1.1M files)

[paul19920801]

I’m experiencing widespread failures retrieving files from Teldrive. Files are listed correctly, but downloads/streams fail.

Errors:

parts.mismatch expected=1 actual=0
stream.parts_fetch_failed
rpc error code 400: MESSAGE_IDS_EMPTY
Downloads return ~20 bytes

Example:

ERROR [FILE] parts.mismatch expected=1 actual=0
ERROR [FILE] stream.parts_fetch_failed error=file parts mismatch

Environment:

Self-hosted Teldrive
PostgreSQL backend
Telegram channel storage
~1,100,000 files in dataset
Bots configured but not actively used (to my knowledge)
Logged in via Telegram user session (QR login)

Database observations:

Active channel (channel_id = 2506079362)
Example valid entry:
[{"id": 7679}]
Invalid entries also exist:
id = 0 (~1,300 rows)
NULL (~8,900 rows)

Behaviour:

Files visible in Telegram manually (same account)
File listing works in Teldrive
Retrieval fails for many files (both old and new)

Important context:

Previously ran multiple Teldrive instances sharing a DB
Later split instances into separate setups
Channel remained the same

Problem:

Even when:

message IDs exist
files exist in Telegram
session is authenticated

👉 Teldrive returns MESSAGE_IDS_EMPTY and cannot fetch data

Questions:

Can message IDs become invalid after splitting instances / reusing DB?
Does Teldrive depend on original uploader context (bot vs user session)?
Are there known limits/issues when working with very large channels (~1M+ files)?
Is there a supported way to re-index or repair parts mappings?
How should invalid entries (id = 0 / NULL) be handled?

Impact:

Large portion of library unusable
Downloads unreliable
Concern about data integrity

Request:

Looking for:

Root cause of MESSAGE_IDS_EMPTY in this scenario
Recommended recovery or re-index approach
Confirmation if this is expected behaviour or a bug

Happy to provide additional logs or DB dumps if needed.

[aZuiZui]

Sometime messages and db are out of sync.

teldrive check --user yourusername -c yourconfig

https://deepwiki.com/search/teldrive-check_85c4a14c-da04-4f3b-afa5-22bc56446b66?mode=fast

This will run a check with the messages and db to make sure they are consistent. You can use --dry-run if you want to see whats wrong. If you run it without, it will just fix the issue. I don't see the point of the --dry-run because the files is either not there and db entry is wrong.

[paul19920801]

Figured out what happened. I had a rogue script which somehow wrote a million small files in a few days, and it pushed my actual files so far back, I am attempting to delete files - it's "cleared" the entire channel (the entire channel was just these small files) - but despite the GUI saying no files exists - it's still possible the delete script to delete files despite them not showing in the GUI

[akhil-rana]

I am facing similar issue, some random files giving this error. Although this is not parts mismatch failed in my case

i also have millions of files, some of them are very small indeed.

i have already tried running check command and everything.
The particular files which throw this error, cannot be downloaded even from UI..
Whenever i try to download any of such file i get:

INFO [HTTP] status=200 method=GET path=/api/files query=page=1&order=asc&sort=name&operation=find&query=8%2BThe%2BGood%2BLife%2B(Live%2Bat%2BX96%2B-%2B1997).flac ip=172.18.0.1:48802 latency=140.830476ms user-agent=Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20... response_size=333
2026-05-09 13:46:45 ✗ ERROR [FILE] stream.parts_fetch_failed file_id=019d3336-4d90-7493-8def-18bd167950ac user_id=0 error=retry middleware skip: rpcDoRequest: rpc error code 400: MESSAGE_IDS_EMPTY
2026-05-09 13:46:45 ✓ INFO [HTTP] status=200 method=GET path=/api/files/019d3336-4d90-7493-8def-18bd167950ac/8 The Good Life (Live at X96 - 1997).flac query=hash=c590e3c78e68facbbfccbd9718b12a7b&download=1 ip=172.18.0.1:48802 latency=1.884195977s user-agent=Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20... response_size=75
2026-05-09 13:47:03 ✓ INFO [HTTP] status=200 method=GET path=/api/files/019d3336-4d90-7493-8def-18bd167950ac query= ip=172.18.0.1:44330 latency=295.695803ms user-agent=Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20... response_size=447
2026-05-09 13:47:04 ✗ ERROR [FILE] stream.parts_fetch_failed file_id=019d3336-4d90-7493-8def-18bd167950ac user_id=0 error=retry middleware skip: rpcDoRequest: rpc error code 400: MESSAGE_IDS_EMPTY
2026-05-09 13:47:04 ✓ INFO [HTTP] status=206 method=GET path=/api/files/019d3336-4d90-7493-8def-18bd167950ac/8 The Good Life (Live at X96 - 1997).flac query=hash=c590e3c78e68facbbfccbd9718b12a7b ip=172.18.0.1:44326 latency=1.764208718s user-agent=Mozilla/5.0 (X11; Linux x86_64; rv:150.0) Gecko/20... response_size=75

[aZuiZui]

This happens because the db is reporting the file is there but it is not etc. You will need to run a check. The command would be teldrive check --user yourusername -c config.xoml this will run a check and repair anything that doesn't match.

I think there is a patch that stop the panic attack but won't address the missing file unless you run the command. Probably a way to fix this would be a check before commit such similar TCP. At the moment it's probably working similarly to udp where all writes are expected to be valid which is impossible.

[akhil-rana]

This happens because the db is reporting the file is there but it is not etc. You will need to run a check. The command would be teldrive check --user yourusername -c config.xoml this will run a check and repair anything that doesn't match.

I think there is a patch that stop the panic attack but won't address the missing file unless you run the command.

I have already run the check command though

[aZuiZui]

This happens because the db is reporting the file is there but it is not etc. You will need to run a check. The command would be teldrive check --user yourusername -c config.xoml this will run a check and repair anything that doesn't match.

I think there is a patch that stop the panic attack but won't address the missing file unless you run the command.

I have already run the check command though

I would stop all new writes to the db and teldrive. Run it again.

Enable debugging and get the necessary logs. Someone here would be able to help identify the issue.

[paul19920801]

From my understanding unfortunately you are out of luck. I made peace with the fact that I lose 5tb of data.

What happened - based on my research was all the small millions of files pushed the telegram library so far back, that the usable data was out of API reach, and when that happens it's impossible to recover the data.

How this happened I don't know. I have two Teldrive accounts - one had 5tb, and another holds 81tb.

The 81tb account stopped working a few months back and gave me a channel full notification, once I made a new channel all was fine.

The 5tb was the one that didn't give me any channel full. It just continued to write data, to the point where my old data was unreachable.

If this happens to my large drive - I am pretty much screwed.

but it was on the large account where protection of some kind kicked in. On the small account nothing stopped files from being cleared off.

[aZuiZui]

How did you go above the limit, it's hard no...

There is a capped of 500k messages. I can't imagine the tool pushing over it. It's supposed to split to a new channel unless it's a bug.

[paul19920801]

How did you go above the limit, it's hard no...

There is a capped of 500k messages. I can't imagine the tool pushing over it. It's supposed to split to a new channel unless it's a bug.

No clue.. suspect it was a script that went rogue - I'm usually hyper aware. but as I type this I see on my working channel is also writing small files currently on 207804 files but some are being deleted and new ones added so it seems to be doing what is expacted. TO have over a million files on my telegram drive most of them small is just off

[aZuiZui]

The only time where I think it would go over is if you only had one channel and didn't enable the auto create channel option.