Ant Download Manager (AntDM) Forum
Would you like to react to this message? Create an account in a few clicks or log in to continue.

Possible Memory Leak

3 posters

Page 1 of 2 1, 2  Next

Go down

Possible Memory Leak Empty Possible Memory Leak

Post by jvidal Wed 27 May - 20:47

Hi!

I was downloading the ISOs for W10 v2004 and I noticed that AntDM's memory usage went through the roof while downloading, well over 1GB, up to about 1.2GB.
Then I downloaded another ISO with FDM and it did it while using only about 115-120MB...

Is this normal?

thanks!

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Thu 28 May - 6:11

Please give me link on this ISO file I will test it.

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Thu 28 May - 6:33

OK. Here it goes:
https://software-download.microsoft.com/pr/Win10_2004_Spanish(Mexico)_x64.iso?t=97aa96c2-d64a-4bd9-996a-7797d823f304&e=1590726637&h=da0279c5b564e440f9885265420e660b

in less than 30 seconds, the memory usage went over 700MB (just tested it again).after about a minute 1.3GB and the download stalls. And while AntDM is not downloading (download stalled), the memory usage stays at 1.3GB...then, after about aminute or so, the download continues, with ANTDM still using over 1GB of RAM.
for me it's not such a big deal, I have 16GB RAM, but for someone with maybe 4 GB this can be a real problem...
after a few (2 or 3) minutes, the download completes and memory usage returns to about 10~11 MB.

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Thu 28 May - 8:41

I could not reproduce your problem.
In my test, the memory fluctuation was ~45-80MB. And 16MB after downloading.
Please do the software export settings (from main menu) and send me on support@antdownloadmanager.com. I will try to repeat test with your settings.

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Thu 28 May - 9:19

I was able to approach your mem usage ~460MB when I set the buffer to 10MB and 64 threads (64*10MB to 640MB.).
These are extreme (not recommended settings). There may be even more memory allocation (even if there are fewer threads) if a slow SSD disk is used.
The reason is that due to the slow disk the buffers do not have time to be written to the file and they quickly accumulate in memory.

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Thu 28 May - 19:06

I do not have an SSD, I have a regular disk (which is , of course, a lot slolwer than an SSD).
My settings are 8 threads and 5120 buffer size, not even near 64 and 10...

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Fri 29 May - 6:41

I tested your settings on a slow virtual machine.
No memory leak. After downloading the file, the memory is fully returned to OS.
The reason is that I wrote above.
Files buffers do not have time to be written to disk when download starts. They accumulate in RAM so as not to slow down the download speed.
There is no limit of accumulation at the top, so as not to slow down the download speed and not to let the demanding user to write - "slowly" or "IDM faster". Such users often like to write it.

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Fri 29 May - 7:07

So, in short, this behavior is normal?

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Fri 29 May - 7:16

Acceptably.
The main thing is to discard the accumulated memory after downloading.

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Fri 29 May - 7:52

which, AntDM does, after downloaing, memoryusage returns to normal. The only thing that bothers me is the unsually high memory usage while downloading big files. never seen it before.

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Fri 29 May - 8:05

I do not like it too, but I go on about users who measure the quality of the downloader by speed.

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Fri 29 May - 19:30

What also puzzles me is why other DMs (like FDM) download the same file while using a LOT less memory...different approach, maybe?

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Sat 30 May - 5:38

Sure!

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Sat 17 Oct - 19:14

I just wanted to report that as of v1.19.6 b75078 this problem still exists.
I downloaded a 4GB file and the memory usage spiked to ~1.2GB while downloading, with HUGE disk activity and the download constantly stalling.
I don't think this is normal.
Then I downloaded another 4GB file with FDM and it went smooth as silk. no high memory usage and no unusual disk activity.
I would REALLY appreciate it if you could look into this issue...
thanks!

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Mon 19 Oct - 9:58

What is your Download buffer size in Config\Downloads\Extra?
Try to set it ~100KB
What is the test result now?

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Mon 19 Oct - 18:09

currently it's 5120Kb (that's about 5MB), I don't see how that could cause AntDM to use up to 1.3GB of RAM...
I'll set it to 100kb and see what happens...

update: I did it, the result was awful. the download starts, then it stalls for several minutes after only 34MB, with 100% disk activity. Then the download continued, but still with 100% disk activity. terrible. worse than before (except the memory usage, wich didn't go over 170mb, but at the expense of extremely high disk usage, which impacts system performance)

update2: then I tried setting it to about 2MB and it was even worse...the memory usage went up to about 800MB and it wouldn't go down even after cancelling the download. it took several minutes to return to normal (with 100% disk activity in the meantime). this shouldn't happen, I have 16GB of RAM!

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by khanyash Mon 19 Oct - 19:37

jvidal,

Can the issue be reproduced with 1 GB file download?

I'm on a limited plan with 1.5 GB daily quota. If the issue can be reproduced with 1 GB file download, I can test here.

khanyash

Messages : 69
Date d'inscription : 2019-03-11

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Tue 20 Oct - 5:12

khanyash wrote:jvidal,

Can the issue be reproduced with 1 GB file download?

I'm on a limited plan with 1.5 GB daily quota. If the issue can be reproduced with 1 GB file download, I can test here.

nope, I tried the 1GB file from https://www.thinkbroadband.com/download and it went fine. memory usage maxed at about 170MB

Then I tried the Ubuntu 20.04.1 iso image (about 2,5GB) and got the problem again. 1GB Memory usage and constant (non-stop) disk activity.

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Tue 20 Oct - 7:21

It is obvious the speed of your HDD can not cope with the flow of data received from the Internet.
If you set a large buffer, data will accumulate in RAM filling it up to failure and will be gradually written to the disk.
If you seta small buffer, the data will be write to the disk all the time taking up its operating time.

The buffer size has a dual role. It is the size of the accumulation of data and it is the accumulation factor when data cannot be written to disk.

You should reduce the speed at which data arrives. For example, to reduce the number of streams or to reduce the speed of the network (snail, machine).

What is your internet speed? What is your disk SSD or HDD?

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Tue 20 Oct - 7:41

My internet speed is 400Mbps (about 50MB/s) and I have a regular HDD (but a pretty fast one, it can transfer at about 170Mb/s), so it shouldn't be an issue. And, like I said, with another Download manager (FDM) I don't have this problem AT ALL.

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Tue 20 Oct - 7:50

In this specific case 400Mbps+HDD is problem. For example, I won't see this problem, because I have 60Mbps+SSD.

jvidal wrote:And, like I said, with another Download manager (FDM) I don't have this problem AT ALL.
3 or 5+ version?

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Tue 20 Oct - 17:45

FDM 6

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Tue 20 Oct - 23:58

for example, I just downloaded the Win10 20H2 ISOs, no problem at all. at full speed (50MB/s). FDM's memory usage=180MB. and disk activity completely normal.

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by antdm Wed 21 Oct - 12:55

jvidal wrote:for example, I just downloaded the Win10 20H2 ISOs, no problem at all. at full speed (50MB/s). FDM's memory usage=180MB. and disk activity completely normal.
Try the today beta.

antdm

Messages : 669
Date d'inscription : 2017-08-14

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by jvidal Wed 21 Oct - 21:07

I'll do. thanks!!

jvidal

Messages : 442
Date d'inscription : 2018-12-17

Back to top Go down

Possible Memory Leak Empty Re: Possible Memory Leak

Post by Sponsored content


Sponsored content


Back to top Go down

Page 1 of 2 1, 2  Next

Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum