[MTOS-dev] Circular reference issue

[Hirotaka Ogawa]

Hi,

I'm trying to remove circular references in MT/MTOS 4.1.  The attached
patch is a tentative result of my work.

== Problems ==

1) MT::Template::context()

MT::Template::context() method may create a circular reference between
MT::Template::Context and MT::Template.  $tmpl has a reference to
$ctx, and $ctx has a reference to $tmpl.

2) MT::Builder::compile() and MT::Builder::_text_block()
   MT::Template::createElement(), MT::Template::createTextNode(),
MT::Template::Node::parentNodes()

All these methods may create a circular reference between MT::Builder,
MT::Template, and MT::Template::Node objects.  It is crucial that 6th
and 7th parameters of MT::Template::Node sometimes have references to
other large objects.

== Modifications ==

Basically, the patch includes 2 kinds of modifications to MT::Builder
and MT::Template classes.

1) MT::Template::context()

The patch stops storing a MT::Template object to
$ctx->stash('template') at context() method.  Even after applying this
modification, MT::Template::build() method stores
$ctx->stash('template') temporarily. Therefore, any methods which use
$ctx->stash('template') may properly work as far as they are called
from MT::Template::build().  This modification may have an impact on
compatibility.

2) MT::Builder::compile() and MT::Builder::_text_block()
   MT::Template::createElement(), MT::Template::createTextNode(),
MT::Template::Node::parentNodes()

The patch makes their references weak by employing Scalar::Util module
[http://search.cpan.org/~gbarr/Scalar-List-Utils-1.19/lib/Scalar/Util.pm].

== Test code ==

I have attached two test codes and logs to this message.

t1.pl creates MT::Template object from a short text and builds it
repeatedly.  And, t2.pl loads Main Index template from the database
and builds it repeatedly.  After repeated builds, both scripts output
the amount of remaining objects by using Devel::Leak::Object.

In both tests, the amount of remaining objects are enlarged by the
repeat count in the original code.  On the other hand, in the modified
code, the amount are kept constant.

But, as the log shows, t2.pl still has a large amount of
MT::Template::Node objects remaining in memory.

== Real applications ==

I have tested on a more realistic situation.

Using mod_fcgid and MT::App::Viewer as described at the former post
[http://www.sixapart.com/pipermail/mtos-dev/2007-December/000211.html],
I have issued sequential requests (1-1000) by using Apache benchmark,
and have measured RSS (resident set size) and VSZ (virtual memory
size) of the FastCGI process.

The results are available from
[http://spreadsheets.google.com/pub?key=p05llkILPlbIlEVNUgkL0dw].

As results shown, the patch drastically reduces memory usage,
especially in 1000 times requests, it can cut down almost 80% of
memory usage.

Unfortunately, I cannot say this patch is an oracle.  Memory usage is
still enlarged by each requests.

== Caution ==

The patch is not well-tested.

-- 
Hirotaka Ogawa makes no sense.
http://as-is.net/blog/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: t1.pl
Type: application/x-perl
Size: 356 bytes
Desc: not available
Url : http://www.sixapart.com/pipermail/mtos-dev/attachments/20080115/90e529ea/attachment-0007.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: t2.pl
Type: application/x-perl
Size: 401 bytes
Desc: not available
Url : http://www.sixapart.com/pipermail/mtos-dev/attachments/20080115/90e529ea/attachment-0008.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: t1-orig.log
Type: text/x-log
Size: 3880 bytes
Desc: not available
Url : http://www.sixapart.com/pipermail/mtos-dev/attachments/20080115/90e529ea/attachment-0009.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: t1-fix.log
Type: text/x-log
Size: 3220 bytes
Desc: not available
Url : http://www.sixapart.com/pipermail/mtos-dev/attachments/20080115/90e529ea/attachment-0010.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: t2-orig.log
Type: text/x-log
Size: 4907 bytes
Desc: not available
Url : http://www.sixapart.com/pipermail/mtos-dev/attachments/20080115/90e529ea/attachment-0011.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: t2-fix.log
Type: text/x-log
Size: 4261 bytes
Desc: not available
Url : http://www.sixapart.com/pipermail/mtos-dev/attachments/20080115/90e529ea/attachment-0012.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mtos-r1235-fix-circular-references.patch
Type: text/x-patch
Size: 2689 bytes
Desc: not available
Url : http://www.sixapart.com/pipermail/mtos-dev/attachments/20080115/90e529ea/attachment-0013.bin