OPT Benchmark

From Invenzzia wiki

When choosing a new library for their new project, many programmers pay attention to the performance. Template engines are not an exception. Almost everywhere, we can see assurances that the very library is very fast and we will never have any problems with it. In case of Open Power Template 2, we decided that the best way is to give the user a benchmark and the results, so that the interested programmer could check it on his own. So, let's go!

Template engines are often very complex scripts and it is actually impossible to produce a perfect benchmark. Unsually, we measure a small set of (basic) functionality and test the libraries in the most common cases. This is why you should not treat this and any other benchmark as an oracle when it comes to the performance.

Contents 1 What we are testing? 2 How we are testing? 3 The results 4 The benchmark sources

What we are testing?

Four template engines have been chosen for the test:

1. Open Power Template 2.0.0
2. PHPTAL 1.2.0 - similarly, to OPT, it uses an XML-like template language called TAL.
3. Smarty 2.6.26 - the most popular template engine must be included in the test.
4. Zend_View from Zend Framework 1.8.4 - many programmers use PHP as a template language, so it is good to have a template engine that uses PHP for writing templates.

How we are testing?

We prepared seven tests:

1. Loading libraries - in this test, we measure, how long takes PHP to load and parse the minimal source code of the template engine required to parse a template. The libraries are neither initialized nor configured.
2. Basic template - the libraries must display a simple template with "Hi universe" text and one variable.
3. Big text - the libraries must display a template with 20 KB Lorem ipsum text.
4. Simple lists - the libraries must generate a list of 100 items. Each item contains two variables.
5. Nested lists - again, we have 100 elements, but in nested lists: 10 items in the top list, and each item contains 10 extra subitems.
6. Including templates - the libraries must include an external template 10 times. The template names are provided with a list.
7. List with an external layout - this is a functional test. The libraries provide the best features recommended by their documentations to solve the specified problem. And the problem is very similar to the Simple lists test. We have a list of 100 elements, but the item layout must be somehow defined externally (i.e. in other file or in other part of the same template, if possible). The template engine must use the specified item layout to display all the 100 items.

Each library took the same test ten times. The worst result was removed, and the average of the rest was the final result. The tests from 2 to 7 attempt to show the execution time of particular features. This is why we would also like to know the results without measuring loading the library by PHP. In this case, we simply substracted the result of the first test (loading time) from the test result.

The time was measured with the following code:

<?php
$time = microtime(true);
 
// the test code
 
$total = (microtime(true) - $time);
file_put_contents('testx.bench.txt', $total."\n", FILE_APPEND);

The results

The second group of rows shows the results with the loading time substracted.

This article is a stub. You can help us by expanding it.

The benchmark sources

You can test the template engines on your own. The test sources and the collected results are available as a TAR.GZ archive to download. The pack contains the source codes of all the tested libraries.