Date:  06/30/2011 03:39:31 PM Msg ID:  004303
From:  Cory Brasel Thread:  004299
Subject:  Re: buffer memory for channels and RAM
OK, thanks for the info again.   We are going to adjust the memory once every few days or so and see what the user base notices for performance.  Based on what you mention, I think our channels are correct then, as the 6th channel only gets used every few/several minutes during the day.  Will report back on what we see good/bad for future reference.  Thanks again. 
Sent by FoxWeb Support on 06/29/2011 05:44:33 PM:
1: You are right in that FoxWeb simply divides the buffer memory you specify in its configuration by the number of channels and then sets both foreground and background buffer memory to this value. A process can either be in the foreground or background (FoxWeb channels are always in the background, unless you are running them interactively and click on one of them), so only the background value takes effect.
2: The rule of thumb is just that. You should only increase the number of channels if you see that the higher channel numbers are used frequently. If higher numbers are only used once in a great while, then you may want to decrease the number of channels (which would free up RAM and other system resources for the rest of the channels). If your scripts execute quickly (up to a couple of seconds) and you don't have too much traffic, then 6 channels may be more than adequate. In many cases, request queuing will give you better performance than parallel processing.
3. As I mentioned in my previous message, it may be better to stop dividing the specified amount between your channels, but rather think of each channel separately. I would modify fw_start.prg, so that it manually sets the buffer memory. If you want, you can even make it use the configured value (without dividing):

IF ! StartupDone

    =SYS(3050, 1, fwConfig.BufferMemorySize * 1048576)

    =SYS(3050, 2, fwConfig.BufferMemorySize * 1048576)

    PUBLIC StartupDone

    StartupDone = .T.

ENDIF


The only difference between the above and the default FoxWeb behavior is that the value does not get divided by the number of channels, so if you specify 400 in the FoxWeb Control Center, each channel will get 400 * 1048576 bytes (equal to 400 MB) of buffer memory.
FoxWeb Support Team
support@foxweb.com email
Sent by Cory Brasel on 06/29/2011 04:26:12 PM:
Thanks for the info, I read through that link a few times....

Couple other questions from below-
1.  You mention it divides the buffer memory by # of channels and uses that amount to set the Sys(3050) values.  Does it use the same amount for both the foreground and background values since it uses *up* to that amount and not claim it immediately, or does it further divide that number in half or whatever and then set the fore/back froun values of SYS(3050)? I am assuming it uses the same value for both settings due to the use up to approach.
2.  After reading the link, it appears anything over 512MB in the SYS(3050) is not needed really and hurts the performance as you mentioned.  We have 2500 in the buffer memory with 6 channels so that means it is setting SYS(3050) to 416MB for each of our channels.  So, that should not be hurting us.  However, I suspect that we don't need that much memory and was planning on tweaking the channels up (which would mean less memory per channel) to see what impact that had.  My question is - in the Foxweb doc, you mention a good rule of thumb to at least start with is 3-6 channels per CPU.  With 4 CPUs, we probably don't have enough channels.  I was going to increase to 10-12, but was trying to figure out the memory impact.  Do you have a good rule of thumb to start with for buffer memory that is "normal" or a good base for each channel?

3.  The value that is put in the buffer memory field on the foxweb setup- should this be the same amount as your server RAM is, or a fraction of the total server RAM or something?  We have 4 GB RAM on the server and have 2500 in the buffer memory size in Foxweb.  Is 2500 too much to use for Foxweb, too less, etc?  Foxweb is pretty much the only application housed on the server other than a couple of very minor things and then of course the OS itself.  I would say we don't have very large tables or that complex of VFP programs...

I realize this is going to be trail and error for tweaking between the number of channels and memory.  In my original post below, I meant to say that I "suspect we are NOT configured optimaly...."  After reading your info and links, we may not be as off as I thought.

Again, any other info or comments, or recomendations from previous experience would be appreciated.  Thanks,
Sent by FoxWeb Support on 06/28/2011 08:08:22 PM:
It's not easy to produce answers to your questions. Part of the problem is that VFP does not have a centralized controller, that can allocate buffer RAM to the various VFP processes, based on how much RAM is available at any given moment.
FoxWeb takes the amount of buffer memory you entered in the FoxWeb Control Center, divides it by the number of channels and uses the result to set SYS(3050, 1) and SYS(3050, 2) for each channel. A couple of years back we thought about getting rid of this algorithm and simply let administrators determine the amount of buffer memory that each channel would be allocated, without regard for the total number of channels. In the end we scrapped this idea, because it would change the meaning of the current field and could have adverse effects to existing installations, if the configured value was not changed to match the new algorithm.
Regardless, One problem with determining the optimum amount of buffer memory for FoxWeb channels is that if you are running a large number of channels (and have configured your buffer RAM accordingly), but only a couple are being used at a time, you are missing the opportunity to utilize a higher amount of RAM per channel. VFP instances do not claim the buffer memory that was configured via sys 3050, but rather use *up* to that amount, if and when they need it.
A second, more important, problem is that increasing the amount of buffer memory may actually hurt performance. Please take a look at http://fox.wikis.com/wc.dll?Wiki~sys3050 for an explanation.
Lately, I have been recommending that customers override FoxWeb's configuration via a direct call to SYS(3050) in fw_enter.prg. This would essentially remove the total number of channels from the determination of the amount of buffer memory per channel and allow administrators to experiment with different values.
  
FoxWeb Support Team
support@foxweb.com email
Sent by Cory Brasel on 06/28/2011 04:34:35 PM:
I have been looking through prior postings and etc regarding optimal setups for number of channels /buffer memory and how it relates to actual RAM on the machine and etc...

I read that generally speaking, 3-6 channels per processor is a good target.

I am trying to find what the optimal buffer memory is with respect to how it gets divided among the total channels and how it relates to the actual RAM amount on the server.  We have a 4 CPU 4GB RAM server, we have 2500 in the buffer memory setting.  We migrated an older server to a new box with better CPU and memory, so I am looking to optimaize what we have.  I suspect the we are configured optimaly or have too much than what we really need or can use..

I get 1,466,695,680 for SYS(3050, 1) and 366,739,456 for SYS(3050, 2) when I check the values from VFP outside of Foxweb.

When Foxweb runs, is it changing the SYS(3050) values to the 2500 we have stored in the Foxweb setup?  For "basic" type of processes , what is optimal buffer size for each channel?  How does it realate to the actual amount of RAM on the machine?  I want to change our channels to 12 or so and trying to find good info on the memory and etc.

Any info or previous knowledge would be a great help.  Trying to optimize what we have now and use that knowledge for 11 more servers we are getting ready to replace.  Thanks.