what controls virtualmin upload speed?

14 posts / 0 new
Last post
#1 Sun, 11/03/2019 - 14:23
adamjedgar

what controls virtualmin upload speed?

I have an upload file size of 850MB that I am performing.

My mobile phone is hotspotted to my laptop and the mobile network speed on laptop browser (Ookla speed test) is around 40mbps up and down.

Its been over half an hour and Virtualmin has not even uploaded 400MB yet. This is very slow considering the available speed on my phone

How can i improve Virtualmin upload speed? (I am uploading as root through webmin file manager)

Mon, 11/04/2019 - 12:46
miklovestech

I noticed it goes much faster if you increase your resource limits of PHP.

Mon, 11/04/2019 - 13:55
Dibs

Is it possible that the speed test is showing (Mega) bits\second and the file size is (Mega) bytes? So difference of a factor of 8?

Mon, 11/04/2019 - 14:08
adamjedgar

I am uploading from Webmin file manager as root. Also, I am running php-fpm. In Webmin>Server>Apache Webserver>Apache Modules> php7.0 and 7.3 are both disabled. Which php resource limits affect upload speed in php-fpm? what limits did you set on yours to improve it?

also, i changed over to sftp and its still only going at 166kiB/s...that means ETA is 1 hour 23 mins. This cant be right for a 40mbps upload speed can it?

http://www.speedtest.net/result/8733186368

Isnt 40mbps like 5MB per sec? If so thats 300MB every minute!

I have had a much larger files download in less than a dozen minutes at these speeds. What could be choking it?

AJECreative is the home of $5 webhosting, $15/month VPS servers (1cpu,1gb RAM, 25GB storage)
Centos7, Debian9, or Ubuntu18LTS
Available Control Panels = Centos-Webpanel, Cyberpanel, or Virtualmin

https://ajecreative.com.au

Mon, 11/04/2019 - 14:37 (Reply to #4)
Dibs

I believe the network speed is bits per second and the file size is bytes per second - as such you are comparing apples and oranges.

http://www.calctool.org/CALC/prof/computing/transfer_time - will give you some idea.

EDIT: most internet connectivity (retail) isn't symmetric - i.e. up is usually a fraction of down. which will explain why you've downloaded files in a fraction of the time this one is taking to upload.

Thu, 11/14/2019 - 13:23
adamjedgar

Internet upload vs download speeds are not necessarily faster download vs upload....my mobile phone for example is very often 50:50 and even better.

This has nothing to do with bits vs bytes either...A 300mb file on a 50mbps phone connection should transfer in less than 10 minutes no problem.

The trouble is, virtualmin is uploading at like 100kbps...this is only a fraction of the available speed on the phone.

The choke is the Virtualmin system.

For example, in Ispconfig (which I have on another server), I can control the ftp data transfer rate...it is just a normal LAMP configuration...the difference being only the control panel itself and pureftpd. Btw, ispconfig just sites on top of the lamp install and doesn't customise the underlying webserver during install.

There must be a setting in the Virtualmin system somewhere where this is controlled?

EDIT... I wonder....https://unix.stackexchange.com/questions/48502/how-to-limit-ftp-speed/48... Will take a look at this on desktop PC when I get up.

Now this article has some very interesting information...(I'm reading chapter 9 but there is more chapters in the book..for those like me who are new to this, definitely read chapter 2 -introduction first. Overall, this is quite enlightening)

https://www.lartc.org/howto/lartc.qdisc.classless.html

AJECreative is the home of $5 webhosting, $15/month VPS servers (1cpu,1gb RAM, 25GB storage)
Centos7, Debian9, or Ubuntu18LTS
Available Control Panels = Centos-Webpanel, Cyberpanel, or Virtualmin

https://ajecreative.com.au

Thu, 11/14/2019 - 22:37
Dibs

@adamjedgar

Internet upload vs download speeds are not necessarily faster download vs upload....my mobile phone for example is very often 50:50 and even better.

I can't comment on your phone - most internet connections are asymmetric and not symmetric.

This has nothing to do with bits vs bytes either...A 300mb file on a 50mbps phone connection should transfer in less than 10 minutes no problem.

A factor of 8 is significant in my book.

That aside - if you know your connection to be symmetric'ish and high and have tested "direct" upload speeds to the server (say via FTP) and they are way greater - then deduction would say the issues lies with the mechanism you are using, which as you say is Virtualmin.

Fri, 11/15/2019 - 10:20
adamjedgar

Well one can never say never in terms of local internet speeds...they are fluid I know. However even at 20 Mbps, we are talking upload speed of about 1.5MB per second...that's almost 100MB per minute is it not? So a 1GB file should upload in 10 minutes...I am seeing like almost 1 hour in Virtualmin for a 600-800 MB zip file.

I wondered if virtualmij has built in restrictions on ftp speeds as part of quota or something like that?

Would restricting noproc and nofile or max spare servers etc...would tinkering around with these settings also affect ftp upload speeds?

AJECreative is the home of $5 webhosting, $15/month VPS servers (1cpu,1gb RAM, 25GB storage)
Centos7, Debian9, or Ubuntu18LTS
Available Control Panels = Centos-Webpanel, Cyberpanel, or Virtualmin

https://ajecreative.com.au

Fri, 11/15/2019 - 10:25
Dibs

Have you tried a "raw" ftp upload to see if that also behaves similarly?

If it doesn't - i.e. goes as expected - then the issue is bound to lie with whatever Virtualmin is actually using (under the hood) to do the file transfer.

Sat, 11/16/2019 - 05:26
Jfro

There is so much in between.

I guess you don't have a bare metal box?

I guess you don't have a minimum throughput at your hosters network.? Conditions in contract, depending how they split up all low trafic high trafic loads and so more.

I guess maybe your hoster have some contracts / hardware software with other party's where depending on.. such total speed is split up and divided depending which contracts conditions.

So for real knowing more what is or should be possible you have to ask your hoster, and your isp first, at which conditions you are running the connections (isp) / server on.

All speedtests on their own are only giving some extra info , but are not reliable , doing more different tests , with different loads and time frames, also different networks and locations, all together then you get a better overall insight.

Further the total speed to your box nic from the web should be split up in speed per connection, also here all in between is responsible for how max / min.

Oyea throughput bringing other services down - making them to slow should normally be throttled , so also a lot of depending of your hardware to. ( virtuel hardware i guess) (also for example error correction and packet routing handlers , yes no "raw data" and so on in total route on example single small files , and depending on lot of settings do also) ( when on disk types of filesystem, block size and so much more)

You do speedtests on your box with SSH CLI and curl /wget

I don't use ftp as is not secure, always use sftp for you information.

Sat, 11/16/2019 - 05:45
Jfro

About CLI testing the speedtester out of this topic have one to https://www.speedtest.net/apps/cli

Again do some reading before using it and ask upfront how your hoster is handling some!

Then you can try this at different time frames and so on...

Speedtest® CLI
Internet connection measurement for developers

Speedtest CLI brings the trusted technology and global server network behind Speedtest to the command line. Built for software developers, system administrators and computer enthusiasts alike, Speedtest CLI is the first official Linux-native Speedtest application backed by Ookla®.

With Speedtest CLI, you can easily:

    Measure internet connection performance metrics like download, upload, latency and packet loss natively without relying on a web browser
    Test the internet connection of your Linux desktop, a remote server or even lower-powered devices such as the Raspberry Pi® with the Speedtest Server Network™
    Set up automated scripts to collect connection performance data, including trends over time
    Use Speedtest in your programs by wrapping it in the programming language of your choice
    View test results via CSV, JSONL or JSON

ONLY After having more reliable results you can test apllications as ftp and search for bottle necks, tuning and so on!

Important  You should when doing test also read /save the logs and how server / loads are doing on your box.

For a result as close to reliable, you should block all other traffic to your hosters / ip/nic if you have such option and only allowing the tests ip's / networks. I think you don't have that but for other readers. You can block all other network trying with firewall on your box, and only allow test traffic maybe this helps, or you get a overview out of logs wich part is the other trafic. See my important to deduct.... ;)

We have at one of our hosters vlans / routers /gateways with own firewall rules before the instances / vps we can configure ourselves.

Wed, 11/20/2019 - 03:24
Wed, 11/20/2019 - 14:24
adamjedgar

i was uploading using my phone as hotspot to one of my other servers with same service provider , on a centos web panel system, via ssh yesterday and fast. It uploaded at 100MB (yes megabytes) per minute. The same 1GB joomla website file uploaded in under 11minutes.

AJECreative is the home of $5 webhosting, $15/month VPS servers (1cpu,1gb RAM, 25GB storage)
Centos7, Debian9, or Ubuntu18LTS
Available Control Panels = Centos-Webpanel, Cyberpanel, or Virtualmin

https://ajecreative.com.au

Wed, 11/20/2019 - 16:10
Jfro

Are you sure same network part / segments at your hoster and same server ( i mean if not bare metal but shared as vps systems are) ? Time differences as mayby backups or whatever running somewhere and other traffic, a lot is / (coud be) still different i guess.

Even you "home" connection could have differencies in network traffic local or isp. ( i wasn't there but streams automatic updates, cloud backups and so more, or other users on same "segments" ;) )

Then you say filemin on virtualmin box and ssh on the centos webpanel , that is really very different! IN topic start. ( filemin probably https on port 443 is not the same as ssh) If i need fast large / many files  i login on the server with ssh and do curl or wget ,, else i use sftp .

(centos webpanel which control panel is insecure sofar i know and "also" buggy) ( with also i hope the next update round virtualmin fix some of thier bugs you can find on this forum.)

DID YOU READ THIS BEFORE USING THAT PANEL?  https://www.centos.org/forums/viewtopic.php?f=12&p=278926&sid=3dfe443c82089f071f2541e17b16bfb6#p278926 and some more on the web.... ;) EVen if they are good i dislike persons / organisation abusing others name/brand... then not using such at all kind of ethical thing. ;)

Again do some more reading and research i did give enough links and tetxts to test more real above for example the speedtest cli option. ( in combi with log files and some monitoring for used resources at same time)

Topic locked