Typo3 very slow - Can someone confirm or has tips or information to add?

  • Wutar
  • 09/27/10
  • Offline
Posted: Thu, 2013-06-20 17:21

Hi there,

I just installed a Typo3 6.1.1 with the Introduction Package (and nothing more) on an up to date Virtualmin Vhost. Virtualmin is running on an virtual (Windows Server 2012 Hyper-V guest) Ubuntu 12.04 (all packages are up to date) with 12 GB memory and access to the whole CPU (i7-2600) and nearly no load at all.

And I have page load times (and parstimes) of 3 to 5 seconds for pages which are already cached by the Typo3 caching mechanism and 8 seconds and (far) more for pages which aren't.

I am trying to figure out the problem for the last two days and am now giving up. The project in question will either be not implemented with Typo3 or get completely different hard- and software. Anyhow as I remember that I had similar problems (also not this severe) with a Typo3 4.x on a similar system (dedicated Ubuntu Server with Virtualmin) about 2 years ago and Google had a lot of Typo3 performance tips but none relevant to my problem (as none did help), I guess the problem comes from something very special about my setup and is maybe also Virtualmin related. So I thought maybe I'm not the only one with this kind of problem among the Virtualmin users and it could be a good idea to start this thread to gather some information (and maybe get a solution someday which could help someone).

So here some things I tried and some other data regarding the Virtualmin installation and the server:

  • First of all the system in general runs great (as Virutalmin is great ;-)). There are some Joomla, Drupal, OXID and SMF-systems installed as well as a custom system based on the Zend Framework. All these have load and parse times far below 1 second (some below 0.1 second).
  • Switching to the CGI-Wrapper or even mod_php did not change anything.
  • All tries to tweak the FCGID performance didn't change anything (among other things I tried what was suggested here: https://www.virtualmin.com/node/15019#comment-65851).
  • All tries to tweak the MySQL performance didn't change anything (I tired most of the suggestions from the PMA-Advisor).
  • I did a lot of benchmarks I found all over the web which had results of other systems posted as well so I could compare. I did IO-checks, CPU-, MySQL- and PHP-benchmarks. The results were always great and topped most of the posted reference systems.
  • There is a failover VM on another dedicated server (which has however the same hardware configuration) which shows the exact same behavior.
  • During the request and parsing time (so the 3 to 5 seconds in case of a cached page) htop shows nothing unusual; well php5-cgi has CPU utilization of at most 10%, maybe this is a little odd as it could use much more.
  • I set MySQL slow_queries to 0.8 seconds and still there are none.
  • It doesn't look like there is one function or loop or whatsoever in the Typo3 code which is causing this and where the process would 'hang' but as if the process over all would be very slow. Here is the TypoScript tree with parsing times (in ms) of a cached page:
    Script Start         2 
    Back End user initialized        796
    Process ID      1699
    Process ID/beUserLogin         1736
    Process ID/fetch_the_id initialize/        1929         
    Process ID/fetch_the_id domain/       2430         
    Process ID/fetch_the_id rootLine/         2430         
    Start Template        2436     
    Get Page from cache       2436         
    Parse template        2437 
    Setting the config-array      2647  
    Setting language and locale      2647 
    Page generation       2648 
    php, initialize       2847
    php, render        2850 
    page  PAGE    2850 
    Include libraries         2851 
    substituteMarkerArrayCached       4568     
    Include libraries         4571
    substituteMarkerArrayCached        4572
    Page generation/Local anchor fix, all      4574     
    Page generation/Tidy, cached      4575     
    Page generation/Index page        4575  
    Print Content        4958  
             4970 
  • As you see in the TypoScript tree where the time is consumed and as I did a lot of testing any network related problems can be excluded.

That's all I know so far and maybe this is useful for someone some time. If anybody can add something I could also test and post results as I will not delete the Typo3 Installation in the near future.

Best regards


Resolved

  • Wutar
  • 09/27/10
  • Offline
  • Sat, 2013-06-22 14:18

I spent some more of my time to troubleshoot this some further as I couldn't just leave it be...

And I was able to find the culprit: The MySQL DB respectively the InnoDB engine. Setting all Typo3 tables to the MyIASM (or MEMORY but beware of data loss) engine 'fixes' the problem instantly but of course isn't satisfying.

So I digged a little further and it looks like the InnoDB settings aren't touched by Virtualmin and so the default values are used. This doesn't seem to be a problem on 'little' systems but you can see from my first post that on the system in question a lot of applications with a real lot of databases and tables were installed (also only barely used). Still I only noticed performance problems in Typo3 as it is really massive in terms of DB usage compared to all the others systems (a single page load can trigger nearly 100 INSERTs and DELETEs on its cache_* tables which are InnoDB per default). On the problematic system one of these queries took about 0.05 to 0.5 seconds, which summed up. Optimizing the InnoDB settings changed that to 0.0001 to 0.0005 seconds, just like it should. Typo3 now has parstimes (far) below 400 ms.

I basically followed the MySQL InnoDB setting suggestions posted here: http://www.mysqlperformanceblog.com/2007/11/01/innodb-performance-optimi... Further information can be found here: http://dev.mysql.com/doc/refman/5.5/en/optimizing-innodb.html

In my specific case I added the following configuration:

innodb_buffer_pool_size = 8G
innodb_log_file_size = 256M
innodb_log_buffer_size = 4M
innodb_flush_log_at_trx_commit = 2
innodb_thread_concurrency = 8
innodb_flush_method=O_DIRECT
innodb_file_per_table
 
table_open_cache = 5000

(Obviously the first two values are the ones which normally have the most impact. And taking into account that the default of innodb_buffer_pool_size is 128M the changes were quite massive. But I guess the other options are also quite useful for my case.)

Best regards


I'm glad you figured out the

  • andreychek
  • 01/05/09
  • Offline
  • Sun, 2013-06-23 22:45

I'm glad you figured out the issue -- thanks for letting us know how you fixed it!