Nov 30, 2011

PhpStorm 3.0 is out

Just got a message from JetBrains:

PhpStorm 3.0 is now available and introduces the Smart Duplicated Code Detector to help you quickly find similar blocks through your entire code base and securely get rid of them without losing the intended functionality.
Also a new integrated UML tool allows you to quickly get a bird’s-eye-view of your project structure, or even a semantic view of the changes in the recent VCS commits made by your colleagues.
Furthermore, to ensure that your code works fast and doesn’t cause any performance bottlenecks, the IDE now includes a profiler, a new PHPUnit test runner and JSTestDriver.
The new key functionality in PhpStorm 3.0 includes:
- Smart Duplicated Code Detector
- PHP UML roundtrip diagrams
- Profiling results browser for Xdebug and Zend Debugger engines
- PhpUnit 3.6 integration
- CoffeeScript support
- Significant improvements to FTP/SFTP Sync
- TFS support and revision graph for GIT
- Streamlined UI across all operating systems

Submitting FLUID forms

If you use f:form view helper and submit data using GET requests, make sure you add noCacheHash="true" to the f:form. For example:


<f:form action="register" method="post" id="tx_myext" noCacheHash="true">

If you don't, you will put a copy of the page to the cache for each submitted request. That's not necessary.

It is good to put this option also when you submit using POST because there is no need for cHash in such case.

USER_INT is not evil, your code is!

Quite often I hear a wrong idea that USER_INT plugins is a source of all evil when it comes to TYPO3 performance. I have to bust this myth into dust.




When TYPO3 renders a page, it has to do a lot of stuff. What happens when you request a page under /profile/details/ URL? Firsts, TYPO3 has to resolve the URL to a page id. It calls RealURL to do that. RealURL makes a lot of stuff internally. In the happiest case it makes at least one database query. In the worst case number of database queries will be more than a number of tree levels to the page. TYPO3 page tree implementation is not optimised for performance at all, so you start loosing speed already here. Did anybody thought “nested set”? Nope, no such thing in TYPO3.

Nov 29, 2011

Two types of developers

I divide all developers into two types: team developers and solo developers. Any developer can be good or bad but these two types are fundamentally different in their thinking and approaches to work. It is important to know with whom you work and adjust accordingly.

Team developers

Team developers work well in a team. They need other players. They can work on their assigned tasks but only if those tasks are a part of a whole. Team developers do not like to assume a complete responsibility on the project.

Typically team developers usually communicate well with others, they like to communicate and socialize. They can do the assigned work of any kind but they generally do not have or strive to leran new stuff. They are not entrepreneurs at heart. However they work great in the team. So they are especially useful in big projects when management needs "resources" to fulfil the task.

Team developers can become good team leaders because they understand the environment they work in.

Blog migration, RSS feed woes

As you may have noticed, I migrated to an external blog service. Domain name is the same but I am using Blogger now. There were several reasons to migrate:
  • I do not want to maintain the infrastructure. The blog ran on my own server, so I had to watch logs, do updates, etc. I rather not to do that because this blog is a hobby and not a real project. 
  • Better blogging capabilities. As a wrote already, TYPO3 is not really good for blogging at the moment. Blogger gives far more power tools
  • RTE is much better. When I prepared posts in the past, I always had to fight the TYPO3 RTE. It inserted &nsbp; all over the place, it changed markup all the time, added new paragraphs. So I sent at least 20∞ of time reformatting the article after it was published. Now I simply type and it works. 
  • Better spam handling. There is one guy from India, who regularly tries to abuse my blog by posting offensive words in his language together with phrases like "TYPO3 is great". Other Indians (for whom I have a great respect as a nation), would come here and see all that junk and turn away. This abuser is filtered away automatically now. Never could understand why would people do such destructive things...

About RSS

The side effect of changing the blog is that everybody, who subscribed to my RSS feed, got a lot of old articles marked as unread. I apologize for that. It happens when the blog platform changes. Just mark them as read.

Also I ask subscribers to check the feed they use. The right feed is http://feeds.feedburner.com/dmitry-dulepov. If you use anything else, please, subscribe to http://feeds.feedburner.com/dmitry-dulepov. All other feed links will cease to exist soon.

After using Blogger for a day I feel excited. It is very convenient. I think I'll blog more now. I have ideas for the whole set of articles on TYPO3, better coding, doing stuff in a right way. So stay tuned :)

Nov 28, 2011

Internal knowledge

There is one way to introduce problems in the code, which I'd like to discuss today.

Suppose you have a module, which creates database records. The module has an integer field with a set of flags (each flag is a single bit in the integer). Flags 2 and 4 mean thing X. Flags 2 and 6 mean thing Y. Flag 1 must be used only together with flag 3 or flag 5. The module knows how to manage those flags and what to make of them. Let's name it “module 1”.

Now imagine another module (“module two”). It needs to get information from the first module, that matches certain criteria. Module one would be able to provide that information by matching flags.

Here is the tricky part. The most obvious solution would be simply to query the database directly from module two. Proper solution for module two would be to ask module one for the information. Querying module one's tables using a combination of flags is wrong. Such query uses the internal knowledge of module one outside of the module one. Thus, the implementation of module two becomes dependent on the implementation of module one.

Do you see potential problems here? If module one changes, module two breaks. This "solution spread" introduces unnecessary dependencies between logically separate modules, which is neither necessary, nor good.

A proper way would be to create an API in the module one that returns records, required by module two. If the internal implementation of module one changes, module two will work because all internal proceedings happen inside a single place.

Next time, when you are about to introduce cross-module dependencies, think of minimising them in favor of API usage.

This article was inspired by a http://forge.typo3.org/issues/32136 in TYPO3.

Nov 27, 2011

Productivity

Instead of giving you a full post about productivity, I'll give you a link to much better information than I was going to write. The Ultimate Productivity Blog gives you a 100% productivity recipe.
Read it. You will love it.

Frustrated by the tt_news upgrade

Today I wanted to write a blog post about productivity. Coming to the BE I noticed that I still had TYPO3 4.4.9 running. So I went to upgrade. It usually takes 10-20 minutes to upgrade a TYPO3 version for me. Well, it took nearly 2.5 hours and I had to revert from tt_news 3.1.0 to old good tt_news 2.5.0 because 3.1.0 simply does not work properly. Here is what does not work:
  • When the plugin is inserted as a content element, it does not appear on the page. The reason is that it does not add itself to TypoScript anymore (to tt_content.list.20). I had to investigate and add it manually.
  • New page browser in tt_news does not replace certain markers out of the box. Why couldn't the developer simply use a pagebrowse extension instead of doing a complex thing himself?
  • The formatting is changed too much.
You can tell me that TYPO3 4.4.9 and tt_news 2.5.0 is too old. Yes, that's true. You are absolutely right. But there is one important thing to know. I had no problems updating TYPO3 core or any other extension. However I had huge problems with tt_news. As usual.
People, when you write your code, remember a gold rule: the less problems people have with your creation, the more they will like it. Upgrade should be smooth. Behavior should be compatible!
I hate tt_news. (Georg, yes, I know about news2 but I am not ready to migrate to it. I am out of time.)
No productivity blog post for today because my productivity for today is out of order, thanks to tt_news.

Nov 18, 2011

Future from Microsoft

I came across a video, which is a Microsoft's version of the future. Here it is:

This is beautifully rendered video but... it is fully Microsoft video :) Microsoft is different from many other companies in the way that they never invented much. They worked on inventions of others. For example, almost everything in this vide is either multitouch, or iPad-like or a 3D monitor from a science fiction. Nothing of this is new, everything was seen in movies already (take "V", for example for interfaces).
The other thing that bothered me, is that Microsoft believes it is the future, while most of it is present. HUDs (head-up displays) are quite common. BWM has them in cars, there are even eyeglasses with HUDs now (you can read e-mail with them or see geographical pointers). Multitouch exists for years in all Apple products. Gestures without touching was recently patented, which means implementation is is the labs already. 3D holograms exist for years. You can read news with phones, which are getting very slim, donate, set appointments and send instructions. Nothing of that is new! So where is the future in this video?
The kitchen has the same microwave (which makes your products less healthy for you). It is all present, nothing is from the future.
Pumps? In future??? Nuh.
If Microsoft wanted to show more of the future, they would not look to pumps or luxury cars. They would show holographic meetings (like in "Resident Evil: Extinction"), all green energy, voice controlled equipment ("Hi, Siri!"), less traveling and more time with family. That would the future.
But that's Microsoft: shiny outside, no new ideas inside.

TYPO3 developers, say "no" to memory leaks!

There is one small mistake I often see in TYPO3 extensions that I review. It is related to freeing resources after you have done with them. Many people forget to do that. PHP is a language where you do not have to free memory. That's true. But resources (such as file handles, recordset, etc) are different. You have to close them explicitly! PHP may close file for you sometimes but even that is not guaranteed.
Let's see the problematic code:
function memoryLeakingGetDatabaseRow() {
    $resource = $GLOBALS['TYPO3_DB']->exec_SELECTquery(...);
    return $GLOBALS['TYPO3_DB']->sql_fetch_assoc($resource);
}
What is wrong here? This code fetches a recordset from the database but does not free it. This means that all structures MySQL had to prepare live in memory. This may include portions of the data retrieved from the database. That data lives for the whole request. If you make multiple calls you accumulate more and more such data in memory. That leaves less memory for other processes on the server. This data is tied to the database connection. If you have persistent database connections, this data is not released when your script terminates because the connection is not released. Thus you have data accumulating all the time. The result is slow down of the server after a certain period of time (low memory means more swapping and that is slow).
How this can be prevented? Code:
function memoryLeakingGetDatabaseRow() {
    $resource = $GLOBALS['TYPO3_DB']->exec_SELECTquery(...);
    $row = $GLOBALS['TYPO3_DB']->sql_fetch_assoc($resource);
    $GLOBALS['TYPO3_DB']->sql_free_result($resource);
    return $row;
}
This releases the recordset properly and frees all allocated memory.
[Note: of course, for a single result you can use exec_SELECTgetSingle but the point of this article was to show the problem and the solution]

Nov 17, 2011

How I came to TYPO3

In the comment to my previous article, I was asked how did I come to TYPO3. Here is the answer.
Me and my wife have a project in our country. The project has started in January 2001 as a set of static pages and grew to a large family portal with elements of a social network.
In 2001 it was just a diary, which my wife made online. Blogging was not a hot area at that time but she was blogging. People came and read, they were interested. So she asked me to make a forum on the site. So there was a forum. People asked for more: they wanted more articles, more information. The amount of information grew.
In 2003 we had our own rudimentary CMS. It allowed to manage pages and connect them to menu items. So menu structure was separate and pages were separate. Menu could be two levels deep maximum. The CMS was simple and fast, it could run on inexpensive shared hosting with a very limited CPU power, memory and disk space.
Number of articles grew and navigation became harder because we needed more nesting levels and greater editing capabilities. I worked full time on my daily job, so I could not create a “full force” CMS. I started to look around. I went to the opensourcecms.com and tried several CMSes there. At that time there was only one CMS there, which I more or less liked: Mambo (now it is called Joomla). TYPO3 also was there but the text said something like “the system is too heavy and complex to be installed as demo”.

Nov 15, 2011

Looking back at my professional life

I am 38, which is an age when people start looking back and see what they went through and evaluate their path. So do I. I am a software developer and I was like that for all past years. Have a look back with me.

Nov 7, 2011

Does your blog/forum help spammers?

Do you know that your blog or forum may help spammers to gather e-mail addresses of commenters even if you hide their e-mail addresses?

Recently I discovered one interesting blog and made a comment on one post. There was a checkbox to subscribe to updates of comments. I ticked that because I was interested in the topic. When comments started to come to my e-mail, I noticed that they were sent from the e-mail address of people, who commented. E-mails were hidden in the blog but as soon as you subscribe to comments, you get e-mail addresses of all commenters. That would help spammers a lot if the blog uses Gravatar service for user pictures. Gravatar binds e-mail addresses to pictures. Blogs can show pictures of users if the user provided a valid e-mail address. Thus, if the blog is known to use Gravatar, many users will supply a real valid e-mail. Here you get spammers subscribing and catching a lot of valid e-mails. Update/clarification: this is not a problem of Gravatar! This is your blogging software that can send such e-mails. Gravatar only shows pictures and it is not vulnerable at all.


There is another view on this problem: the blog says that e-mail address will not be revealed but, in fact, it reveals the address. So it may have legal privacy implications. I notified blog's owner about the issue.
Conclusion: never use user's e-mail to send anything from the web site.