[MTOS-dev] MT-Search throttling

[Byrne Reese]

Perhaps we should fork the discussion:

A) short term search throttling fix
B) longer term pre-request-init callback thingamajig


On 3/31/08 11:05 PM, "Fumiaki Yoshimatsu" <fyoshimatsu at sixapart.com> wrote:

> Hmm, now it seems the problem got too big and I don't think I am the
> right person to implement such a low level change.  I'll stop working on
> mt-search throttling for now to see how the idea goes into the actual
> code.
> 
> Fumiaki
> 
>> -----Original Message-----
>> From: Hirotaka Ogawa [mailto:hirotaka.ogawa at gmail.com]
>> Sent: Tuesday, April 01, 2008 2:50 PM
>> To: Byrne Reese
>> Cc: Fumiaki Yoshimatsu; MTOS dev
>> Subject: Re: [MTOS-dev] MT-Search throttling
>> 
>> On Tue, Apr 1, 2008 at 12:17 PM, Byrne Reese <byrne at sixapart.com>
> wrote:
>>> +1
>>> 
>>>  Really like this idea. Would be cool to do this for mt-tb.cgi too!
>>> 
>>>  Could this be done somehow prior to the whole app being loaded into
>> memory?
>> 
>> Probably yes.
>> 
>> As for just throttling by load average, things will be made much
>> easier if we have such a throttling mechanism inside of mod_cgi
>> module.  But, for the time being, we should try to do something
>> without such a gun.
>> 
>>>  The sequence diagram is hard to do in ASCII, but it would look like
> this:
>>> 
>>>  1. request received
>>>  2. check load average
>>>  3. bootstrap app
>>>  4. process comment/search/trackback
>>>  5. return result
>> 
>> Right.  It may be time for us to redesign and elaborate the
>> bootstrapper, in order to make it easier to add pre-post handlers as
>> you mentioned above and support Reed-san's dispatcher ;-)
>> 
>>> 
>>>  With the necessary signal handlers as well. :)
>>> 
>>> 
>>> 
>>> 
>>>  On 3/31/08 2:42 PM, "Hirotaka Ogawa" <hirotaka.ogawa at gmail.com>
> wrote:
>>> 
>>>> On Tue, Apr 1, 2008 at 5:19 AM, Byrne Reese <byrne at sixapart.com>
> wrote:
>>>>> Signals?
>>>>> 
>>>>>  http://www.ccsf.edu/Pub/Perl/perlipc/Signals.html
>>>> 
>>>> Yep.
>>>> 
>>>>>>> 1) Throttle by load average
>>>>>>> 2) Throttle by mt-search.cgi process times; only when
> mt-search.cgi
>>>>>>> cannot generate the result page within 5 seconds, it is forced
> to
>> stop
>>>>>>> processing.
>>>> 
>>>> (2) can be implemented by using SIGALRM.
>>>> (1) can be done just by checking load average at entry point of
>>>> mt-search.cgi, or by using a SIGALRM handler who checks load
> average
>>>> every N seconds.
>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>  On 3/31/08 2:39 AM, "Fumiaki Yoshimatsu"
>> <fyoshimatsu at sixapart.com> wrote:
>>>>> 
>>>>>>> Do you have any plan to implement more dynamic throttling
>> mechanisms?
>>>>>> 
>>>>>> Currently I have no plan - I even think that throttling may not
> be
>>>>>> necessary if it won't be implemented in the "right" way for 80%
> of
>> MT
>>>>>> users.  I am also hesitating to implement simple throttling
> because
>> of
>>>>>> the reason you mentioned, and I don't want to slow the
> processing
>> down
>>>>>> because of throttling that is useless for most of the users.
>>>>>> 
>>>>>>> 1) Throttle by load average
>>>>>>> 2) Throttle by mt-search.cgi process times; only when
> mt-search.cgi
>>>>>>> cannot generate the result page within 5 seconds, it is forced
> to
>> stop
>>>>>>> processing.
>>>>>> 
>>>>>> Sounds very interesting, though I don't have a good idea of how
> to
>>>>>> implement (1).  Can you shed some light on how to do it?
>>>>>> 
>>>>>> I am also thinking that it might be enough for the Search app
> to
>> run a
>>>>>> callback or two in init_request, but does not provide default
>> behavior
>>>>>> at all.
>>>>>> 
>>>>>> Fumiaki
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Hirotaka Ogawa [mailto:hirotaka.ogawa at gmail.com]
>>>>>>> Sent: Monday, March 31, 2008 5:23 PM
>>>>>>> To: Fumiaki Yoshimatsu
>>>>>>> Cc: MTOS dev
>>>>>>> Subject: Re: [MTOS-dev] MT-Search throttling
>>>>>> 
>>>>> 
>>>>> 
>>>>>> _______________________________________________
>>>>>> MTOS-dev mailing list
>>>>>> MTOS-dev at sixapart.com
>>>>>> http://www.sixapart.com/mailman/listinfo/mtos-dev
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> 
>> --
>> Hirotaka Ogawa makes no sense.
>> http://as-is.net/blog/