Problem with virtualmin apache

10 posts / 0 new
Last post
#1 Mon, 06/29/2009 - 11:39
marky

Problem with virtualmin apache

Hello, I have problem with webserver Apache... I am owner of highly visited website (~ 100 000 UIP/day), but apache does not want to accept connections, so 95% of users gets error message Connection Timeout...

there are no apache errors in error log, neither there are no kernel errors.

There is my apache configuration file: http://pastebin.com/m58178501

Anybody knows what to do, to solve this?

Mon, 06/29/2009 - 11:54
andreychek

Are you sure that you aren't running into DNS issues?

Do you see any errors if you go to intodns.com, and enter one of the domain names you're having trouble with there?

-Eric

Mon, 06/29/2009 - 12:52 (Reply to #2)
marky

No, it is not caused by DNS, because when I try to access direct server IP, it does not work too...

Also when I disable Apache server load is about 0.9, but when Apache is enabled - it creates 500 processes, and server load raises up to 416.33

Mon, 06/29/2009 - 14:04 (Reply to #3)
andreychek

Aha, ok.

It sounds like you may have a scaling problem then :-)

Apache -- or one of the dependencies, such as PHP or MySQL, may be having trouble keeping up with the load.

Have things worked for you in the past with this server? If so, has anything changed recently?

While any number of things could be at fault, a common reason for this problem is a slow MySQL query.

You might consider enabling the "log_slow_queries" in the my.cnf file.

Also, the "mytop" program is something you could install and run to get an idea of what processes are running within MySQL -- you could use that to track down what's taking up so much time.

-Eric

Mon, 06/29/2009 - 14:46
Joe
Joe's picture

So, we have no data to go on here, so anything we say about how to scale your application would be wild-ass guessing. And guessing just wastes everybody's time, so let's figure out what is causing slowness before trying to fix it.

Does top give you any clues about what process is most demanding?

No clues in the error_log? Timeouts or anything like that?

Eric's suggestions about MySQL are also worth doing.

--

Check out the forum guidelines!

Mon, 06/29/2009 - 15:23
marky

Nothing - no log error messages. Server worked well within last week. There were done no changes in HW or configuration.

But there is too much connections in state SYN_RECV - about 2000. When I turn on firewall, this number goes down to 300, but no change in speed of connection.

There is Apache status: http://www.world-banking.info/server-status

This is really wierd problem, I have already read and tried all tutorials & help suggestions, that I have found at google, but nothing helped...

Mon, 06/29/2009 - 15:27 (Reply to #6)
Joe
Joe's picture

You're still not showing us the information we asked for. Is there anything in top eating CPU? Do the MySQL tools Eric mentioned give you any information about what queries are slow and where the work is going?

I seem to recall that you had some performance problems a couple of weeks ago, and they all boiled down to problems in the way the application was using MySQL and the lack of indexes on very heavily used large tables. I'm guessing this is more of the same.

--

Check out the forum guidelines!

Mon, 06/29/2009 - 15:30 (Reply to #7)
Joe
Joe's picture

Oh, and I'll mention that your server-status shows that Apache is hitting its configured server limit...and I'm guessing this is happening because the application is slow. Apache is doing exactly what it's supposed to do, but something is making requests take longer to complete than they should...probably database issues.

--

Check out the forum guidelines!

Mon, 06/29/2009 - 15:37
marky

Apache takes 10%; MySQL takes about 8% of CPU; slow query log is turned on and slow query time is set to 1 sec., the slow query log is clear - no slow queries recorded...

Mon, 06/29/2009 - 17:59
ronald
ronald's picture

I remember those 2 sites on that server and after troubleshooting a lot of stuff, those sites were screaming fast.

I would now have a tendency to look into the network and per haps even the network of your datacenter. Per haps a bad switch / router, per haps iface or you are on 100mb line and need a 1gb, you know stuff like that.

a traceroute:

  1. 1.69ms 1.53ms 1.54ms nameintelligence.com Reverse IP | Ping | DNS Lookup |
  2. 1.68ms 1.36ms 1.66ms ip-64-246-162-161.ipd.ccom.net Reverse IP | Ping | DNS Lookup |
  3. 1.65ms 1.43ms 1.68ms 216.145.30.190 Reverse IP | Ping | DNS Lookup |
  4. 1.67ms 1.27ms 1.61ms 19b1-19pe1-vlan259.sea.fibercloud.net Reverse IP | Ping | DNS Lookup |
  5. 1.67ms 1.37ms 1.68ms ge-6-17.car4.Seattle1.Level3.net Reverse IP | Ping | DNS Lookup |
  6. 14.65ms 2.42ms 7.68ms ae-32-52.ebr2.Seattle1.Level3.net Reverse IP | Ping | DNS Lookup |
  7. 12.67ms 2.34ms 1.62ms ae-1-100.ebr1.Seattle1.Level3.net Reverse IP | Ping | DNS Lookup |
  8. 18.70ms 18.33ms 27.68ms ae-7.ebr2.SanJose1.Level3.net Reverse IP | Ping | DNS Lookup |
  9. 18.66ms 19.36ms 20.66ms ae-62-62.csw1.SanJose1.Level3.net Reverse IP | Ping | DNS Lookup |
  10. 19.59ms 48.55ms 19.68ms ae-4-99.edge2.SanJose3.Level3.net Reverse IP | Ping | DNS Lookup |
  11. 20.62ms 20.25ms 21.66ms xe-4-2-0.sjc10.ip.tiscali.net Reverse IP | Ping | DNS Lookup |
  12. 186.60ms 187.26ms 187.64ms so-0-0-0.prg11.ip.tiscali.net Reverse IP | Ping | DNS Lookup |
  13. 188.63ms 189.03ms 189.57ms master-internet-gw.ip.tiscali.net Reverse IP | Ping | DNS Lookup |
  14. 189.63ms 190.20ms 190.61ms praha-4d-c1-vl272.masterinter.net Reverse IP | Ping | DNS Lookup |
        • Request Timed Out
        • Request Timed Out

Do you have your ISP IP's in your network settings? Cause the trace can reach your ISP but not your box.

Topic locked