Thymeleaf benchmark ?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Thymeleaf benchmark ?

Yapadesouci
Hi !

Thanks for this great template engine.
I want to know if there is somewhere a benchmark in order to see if it can be put in production.

Thanks in advance for your answer.
Continue !
Reply | Threaded
Open this post in threaded view
|

Re: Thymeleaf benchmark ?

danielfernandez
Administrator
Hi.

We haven't prepared any "official" benchmarks yet, although of course an important amount of testing on performance has been done during development. Anyway, preparing "scientific" benchmarks certainly is in the TO-DO list.

Remember also that Thymeleaf includes a built-in slf4j logger called "org.thymeleaf.TemplateEngine.TIMER" that outputs the milliseconds taken for processing each template when in DEBUG mode. This will probably help you measure Thymeleaf's performance in your specific scenarios.

Regards,
Daniel.
Reply | Threaded
Open this post in threaded view
|

Re: Thymeleaf benchmark ?

Yapadesouci
I have done a little benchmark. First of all I'm not an expert so it's possible I made a mistake. Don't hesitate to correct me.

I have implemented the same page in jsp and with thymeleaf. All is based on spring framework 3.0.6 so I used thymeleaf-spring. The test run on maven-jetty-plugin and there is no database access at all. The page has no include just few relative url implemented with c:url in jsp and th:src, th:href with thymeleaf.

The template cache is active on thymeleaf. Logger is put on WARN.

I used ab tools to bench with -n 5000 and -c 50 and run the test several times on my local machine.

The result is jsp can handle 3000 requests by second where thymeleaf handle 1200 requests by second.

I want to know if it's normal for you or if I'm doing something wrong.

Thanks in advance.
Reply | Threaded
Open this post in threaded view
|

Re: Thymeleaf benchmark ?

danielfernandez
Administrator
Hi, thanks a lot for sharing your numbers.

These results look quite feasible to me. Bear in mind that JSP pages are actually *compiled* into servlets before their first execution, and so when they execute they are no more than a normal Java class with a lot of "out.println(...)" calls in them.

This is something that non-JSP template engines don't do, and so it is quite difficult for such template engines to achieve levels of performance similar to JSP in many scenarios. Also, think that JSPs have on their side a lot optimizations provided by the application servers themselves (think that tomcat and jetty are basically JSP-and-servlet specialized servers), which non-JSP libraries don't.

So yes, as I said, that figures look quite feasible to me. There's always a performance price for using each technology: with JSPs you pay a price you wouldn't pay if you were using static precomputed HTML pages, with RoR you'd be paying the price of having a powerful and extremely productive programming language, etc.

So unfortunately, if you need to serve 3,000 pages by second from a single server, seems that Thymeleaf is not for you ;-)

Regards,
Daniel.
Reply | Threaded
Open this post in threaded view
|

Re: Thymeleaf benchmark ?

emilime
Hi Daniel,

I think the topic is interesting and it would be useful to have a comparison between Thymeleaf, Velocity & FreeMarker.
Reply | Threaded
Open this post in threaded view
|

Re: Thymeleaf benchmark ?

Zemi
Administrator
I would not expect any surprise on a Thymeleaf / Velocity / Freemarker / etc. benchmark.

There are 3 types of template engines:
1.- Based on XML processing: Thymeleaf, Wicket.
2.- Based on text-processing: Velocity, Freemarker
3.- Based on conversion to Java code: JSP, Play

(1) is slower than (2), which is slower than (3). You could see some benchmarks like:
    http://www.jtict.com/blog/rails-wicket-grails-play-lift-jsp/
or
    http://www.javaworld.com/javaworld/jw-11-2007/jw-11-java-template-engines.html

By testing Thymeleaf performance you would be really testing XML APIs and ONGL library.

But the point is that presentation tier technology performance is not very important in almost every web application. Unless you have very specific requirements, most of the response time will be spent on the business and persistence tiers.

Thymeleaf performance is OK. It would be difficult to improve it a lot (because the bottlenecks are on third-party code) without using a cache mechanism to store the HMTL output.

Reply | Threaded
Open this post in threaded view
|

Re: Thymeleaf benchmark ?

Jamie
I think you're right about rendering performance - if the pages are the result of doing some real work behind the scenes (fetching/manipulating/storing data) then having the view render at half the speed of a JSP isn't going to be noticed.

Making Thymeleaf "lighter" in terms of cpu and memory would be a lot of work, e.g.:
1) rather than parsing a template into a DOM, build a tree/array of nodes representing template operations.  e.g. if your template was:

<html><head><title th:text="${title}">foo</title></head><body><div id="bar"><div th:text="${title}">foo</div></div></body></html> 

then that could be represented as the following 5 nodes:
[text: <html><head><title>]
[output: ${title}]
[text: </title></head><body><div id="bar"><div>]
[output: ${title}]
[text: </div></div></body></html>]
   
This would make the template more memory efficient than if it was a DOM document - those 'text' nodes are just strings that happen to be xml elements.  For simplicity I've excluded things like loops etc - hopefully you get the idea.

2) rendering the template would involve traversing these nodes and executing each one.  The output could be written straight to a java.io.Writer, rather than building another DOM out of the template+data, then serializing that to a string, then writing out the string to wherever you want it.

This approach would consume a LOT less memory - currently with Thymeleaf if you're generating an html page that is 50k in size you'll need a 50k buffer to hold the resulting content, and other large cunks of memory for the template DOM and the result DOM.  Skipping the DOM building and serialization would be an improvement.

Having said all that, I really like Thymeleaf - I think it is very good.  A couple of years ago I started prototyping a 'natural template' engine very similar to Thymeleaf using the approach I described above, and although the idea works it would have taken more time than I had available to complete it (it would also not have been as flexible as Thymeleaf, or as well written) .  So I'm glad that Daniel has shared his excellent work.

Jamie