Nov 30, 2011

USER_INT is not evil, your code is!

Quite often I hear a wrong idea that USER_INT plugins is a source of all evil when it comes to TYPO3 performance. I have to bust this myth into dust.




When TYPO3 renders a page, it has to do a lot of stuff. What happens when you request a page under /profile/details/ URL? Firsts, TYPO3 has to resolve the URL to a page id. It calls RealURL to do that. RealURL makes a lot of stuff internally. In the happiest case it makes at least one database query. In the worst case number of database queries will be more than a number of tree levels to the page. TYPO3 page tree implementation is not optimised for performance at all, so you start loosing speed already here. Did anybody thought “nested set”? Nope, no such thing in TYPO3.


Next, after RealURL returns the id, TYPO3 starts making all kind of checks. Firsts, it checks if the user is logged in. This check involves a query on user and session tables (join). Session id is an integer. Session tables are rarely cleaned up, so in most cases it is a complete table scan for the string (no indexes used). You loose tons of time here. Now, due to some strange logic, TYPO3 does this two times. So you loose that twice. Even with MySQL query cache, you still loose.

Next, TYPO3 fetches the thing called “rootline”. Rootline is a way from the current page towards the root of the web site. It takes at least the same number of database queries as the deepness of the page in the tree. So you loose speed here too. Notice that TYPO3 generates a huge query each time and that query cannot use indexes properly. Pages can be limited by two flags (deleted and hidden), start time, end time and user groups. User group clause is a LIKE clause, which kills indexes. Query cache is no use here because start time and ent time use exact second value: next second the query cache won't give you the result quickly.

Next, TYPO3 starts collecting TypoScript templates. For that, it evaluates all conditions. This happens on every request. 200 conditions (yes, I saw that much), 200 evaluations. Do you have userFunc conditions? They are called. Do you have database queries in them? You got the idea... Now multiply that by two because conditions are evaluated twice.

Next, TypoScript is parsed or fetched from cache. If it is already cached, you are lucky. If not, all your TypoScript is parsed. The more TypoScript you have, the more doomed your web site.

You see? You already loose. You already lost at least half a second.

(All of the above is simplified, of course, but it is correct in general).

Next TYPO3 either fetches or generates the content. Let's assume fetching, it is faster. The content is fetched from cache and ready to be sent.

Only now USER_INT objects are called. Suppose you have a registration form USER_INT object. The corresponding plugin simply makes the form. No database queries. No loops. Just read the template, replace a couple of markers. Is that slow? Not at all, compared to the all stuff described above! Even a database query to check user's existence followed by the INSERT query is faster in the USER_INT than user authentication process!

So USER_INT by itself is not evil. Absolutely not. The evil is the programmer, who does not know how to code with performance in mind. Database structure, database indexes, loops optimisation, callUserFunc over-usage, reflection – it all slows down the code. Don't blame the technology, blame people, who can't use it properly.

Developers should always code with three things in mind:
  • performance
  • security
  • simplicity
These three things are connected. When coding with these principles in mind, the code becomes small, fast and efficient.

And, of course, developers need to learn. Learn every time, all life. New things are uncovered daily, so “staying in the loop” of current events is important for every developer.

Stop kicking USER_INT. Start kicking programmers!

5 comments:

  1. Good posting. Another hint on the name USER_INT: INT stands for "Internal Caching", which means TYPO3 does not cache this plugin, but it does that itself. Do that! Always think about if you can build clever caches to your plugin and test if they really improve performance.

    ReplyDelete
  2. lol :) I didn't know conditions are executed twice :D

    Now take an extbase extension and add all the time used for dispatching and constructing the MVC-environment... heavy impact, if we think of it in detail. The more it is fascinating how fast most TYPO3 instances work, despite all of these calculations.

    Even if you're running on USER_INT, you can still benefit from caching inside your extension. In my recent project I used extbase's caching pretty successful, so that by caching only certain data, I was able to speed up the whole extension by factor 10 (yes, there's quite a lot of data ;)

    ReplyDelete
  3. Unfortunately they are. I was surprised too when I found out.

    Conditions are evaluated first before trying to fetch compiled TS from the cache. If they are not in cache, they are evaluated for the second time when compiling. We had a case when the condition was a userFunc, which returned two different results in those two evaluations. As a result, TYPO3 never could fetch conditions from the cache. userFunc returned 1 – TS is not in cache. Compile and evaluate the condition, it returned 2 now. TS goes to cache with the value 2. Next page call, the function returns 1 again – not in cache, compile, get 2 again, write to cache. And so on. Took about 3 hours to figure out what was wrong there (live site, no staging, no debugger).

    ReplyDelete
  4. The problem was that condition function called another condition, which was dependent on the available TS :) It was giving back different results when TS was and was not there.

    ReplyDelete
  5. Needless to say that typoscript extensiontemplates are a big performance problem too if you try to avoid conditions.

    ReplyDelete