NextCloud web interface painfully slow

scottalanmiller

@bnrstnr said in NextCloud web interface painfully slow:

EDIT: This instance is actually Fedora 27, not 28.

Not your issue, but troubleshooting two versions behind is never ideal. Get up to Fedora 29 before investigating further.

But NC requires a lot of resources and is always decently slow on the web page. But not ten seconds slow.

Our NC is Fedora 29, heavily tuned, has like six cores, 12GB of RAM, doesn't have any noisy neighbours (not cloud) and you still feel the lag if doing the web interface.

black3dynamite

@bnrstnr said in NextCloud web interface painfully slow:

@black3dynamite said in NextCloud web interface painfully slow:
For the 504 Gateway Time-Out, and if you are using nginx has proxy, do you have this inside the location block in your nextcloud.conf file?
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
I have nothing relating to timeouts in my nginx conf.

Should I add it? I'm assuming it's using whatever the nginx default is.

For Nextcloud, yes. It also helps with when updating Nextcloud via the web interface too.

scottalanmiller

@bnrstnr said in NextCloud web interface painfully slow:

We have WOW cable internet, like 240/20ish

I've poked around and found some other users NC installs and their login pages load almost instantly for me

Vultr storage instances are very slow, they are meant for capacity, not speed. This will definitely contribute to the issue.

Make sure you have swap available. 2GB isn't terrible, but not quite enough for even a tiny instance to run unimpeded.

bnrstnr

@scottalanmiller said in NextCloud web interface painfully slow:

Make sure you have swap available. 2GB isn't terrible, but not quite enough for even a tiny instance to run unimpeded.

I could definitely jack this up to whatever I want.

I will also work on getting this up to Fedora 29, I didn't even realize it was still on 27 :face_with_open_mouth_cold_sweat:

Also, I just submitted a ticket, we'll see what they say.

scottalanmiller

@bnrstnr said in NextCloud web interface painfully slow:

@dafyre said in NextCloud web interface painfully slow:

This. After you reboot, pay attention to the STEAL time. I've never noticed that in the display before... In your glances pic above, steal % is at 48%.
I had to go google it to see what it was, lol... The short version is that steal % is how long your VM has to wait until the Hypervisor gives it more CPU time.
This article, though dated, seems to give good info.
(Understanding CPU Steal Time)

This almost sounds like I have a noisy neighbor on my VPS?

always possible, but rarely seen as a CPU spike.

scottalanmiller

@bnrstnr said in NextCloud web interface painfully slow:

@scottalanmiller said in NextCloud web interface painfully slow:

Make sure you have swap available. 2GB isn't terrible, but not quite enough for even a tiny instance to run unimpeded.

I could definitely jack this up to whatever I want.

I will also work on getting this up to Fedora 29, I didn't even realize it was still on 27 :face_with_open_mouth_cold_sweat:

Also, I just submitted a ticket, we'll see what they say.

I'd add 2GB of swap.

JaredBusch

@scottalanmiller said in NextCloud web interface painfully slow:

@bnrstnr said in NextCloud web interface painfully slow:

@dafyre said in NextCloud web interface painfully slow:

This. After you reboot, pay attention to the STEAL time. I've never noticed that in the display before... In your glances pic above, steal % is at 48%.
I had to go google it to see what it was, lol... The short version is that steal % is how long your VM has to wait until the Hypervisor gives it more CPU time.
This article, though dated, seems to give good info.
(Understanding CPU Steal Time)

This almost sounds like I have a noisy neighbor on my VPS?

always possible, but rarely seen as a CPU spike.

It is not a spike. it is his system constantly waiting on CPU resources from the host.

bnrstnr

There were 260 failed login attempts since the last successful login.

I may be getting slammed by script kiddies here too... This is with fail2ban on. My last login was almost exactly 2 hours ago. I will change my SSH port and see if that helps anything.

DustinB3403

@bnrstnr said in NextCloud web interface painfully slow:

There were 260 failed login attempts since the last successful login.

I may be getting slammed by script kiddies here too... This is with fail2ban on. My last login was almost exactly 2 hours ago. I will change my SSH port and see if that helps anything.

Change your login method to use keys instead of U&P. . .

scottalanmiller

@bnrstnr said in NextCloud web interface painfully slow:

There were 260 failed login attempts since the last successful login.

I may be getting slammed by script kiddies here too... This is with fail2ban on. My last login was almost exactly 2 hours ago. I will change my SSH port and see if that helps anything.

That's totally normal traffic and not being slammed at all. That's light.

bnrstnr

@scottalanmiller said in NextCloud web interface painfully slow:

That's totally normal traffic and not being slammed at all. That's light.

I usually see that many in a couple hours with fail2ban off, with it on I might see 260 in a week.

jmoore

@bnrstnr i agree with Scott here, that is nothing in terms of attempts. My stuff will get thousands in a few minutes when it happens.

bnrstnr

@jmoore said in NextCloud web interface painfully slow:

@bnrstnr i agree with Scott here, that is nothing in terms of attempts. My stuff will get thousands in a few minutes when it happens.

Alright, that's good, one less thing to check off my list. Like I said, it's more than I normally see, but I definitely believe you guys. Thanks

Dashrender

@travisdh1 said in NextCloud web interface painfully slow:

@bnrstnr How's the client ISP connection? I've seen some major issues at handoff points to Level3 since their merger with Comcast.

Comcast? I thought Level 3 was bought by Century Link?

http://news.centurylink.com/2017-11-01-CenturyLink-completes-acquisition-of-Level-3

scottalanmiller

@Dashrender said in NextCloud web interface painfully slow:

@travisdh1 said in NextCloud web interface painfully slow:

@bnrstnr How's the client ISP connection? I've seen some major issues at handoff points to Level3 since their merger with Comcast.

Comcast? I thought Level 3 was bought by Century Link?

http://news.centurylink.com/2017-11-01-CenturyLink-completes-acquisition-of-Level-3

It was.

travisdh1

@Dashrender said in NextCloud web interface painfully slow:

@travisdh1 said in NextCloud web interface painfully slow:

@bnrstnr How's the client ISP connection? I've seen some major issues at handoff points to Level3 since their merger with Comcast.

Comcast? I thought Level 3 was bought by Century Link?

http://news.centurylink.com/2017-11-01-CenturyLink-completes-acquisition-of-Level-3

I'm more than slightly mixed up apparently.

bnrstnr

Vultr's reply...

Hello,

It does not seem that any other guests on the host are using an excessive amount of CPU power.

First, I would check your configurations on your running services to see if anything is out of the ordinary. If all seems to be in check, we can migrate your instance to a new host to see if that resolves the issue.

Let us know your findings and how you'd like to proceed.

DustinB3403

@bnrstnr said in NextCloud web interface painfully slow:

Vultr's reply...

Hello,

It does not seem that any other guests on the host are using an excessive amount of CPU power.

First, I would check your configurations on your running services to see if anything is out of the ordinary. If all seems to be in check, we can migrate your instance to a new host to see if that resolves the issue.

Let us know your findings and how you'd like to proceed.

Have em move it and see if that fixes the issue. Nothing in what you've shown seems excessive so make them do some work as a means of troubleshooting.

travisdh1

@DustinB3403 said in NextCloud web interface painfully slow:

@bnrstnr said in NextCloud web interface painfully slow:
Vultr's reply...
Hello,

It does not seem that any other guests on the host are using an excessive amount of CPU power.

First, I would check your configurations on your running services to see if anything is out of the ordinary. If all seems to be in check, we can migrate your instance to a new host to see if that resolves the issue.

Let us know your findings and how you'd like to proceed.
Have em move it and see if that fixes the issue. Nothing in what you've shown seems excessive so make them do some work as a means of troubleshooting.

Yeah, it's not like that's going to be more than clicking a button for them.

1337

Can't remember where I saw it but some company with a lot of cloud instances were using scripts that would automatically kill an instance and start it on another server if it started being slow. They found that once it start getting slow it would never recover it's original speed and get back to it's former self so this is how they solved it. You never know what else is running on the same host and how many VMs.

Can you yourself someway force the instance to move to another host on vultr?