cfloop using to much memory

We are trying to create a CSV file with about 250,000 rows with about 40 dynamic columns which we need to loop over so we can get the correct order. We see an issue coming from the nested loops on the large dataset and the server is running out of memory.   We took a step back on even doing a simple loop over over a large set of rows seems to have the same issues. Anyone have any ideas how to deal with this? If you run this code on watch the memory on the CF Server it will just keep climbing until garbage collection runs. <cfset outsideloop = 20000000> <cfloop from="1" to="#outsideloop#" index="x"> </cfloop> thanks, -- Sam

Are you using CF8 because they made many improvements to this very problem in 8. Dan ----- Excess quoted text cut - see Original Post for more -----

If you aren't, use a Java StringBuffer or StringBuilder to build up the CSV. If you do the concatenation using CF you'll use large amounts of memory because every concatenation creates a new String object. ----- Excess quoted text cut - see Original Post for more -----

I'm using a java stringbuffer which works great for the portion of the code, but the main issue is with the cfloop tag. ----- Excess quoted text cut - see Original Post for more -----

Can you post the code? Because a loop tag itself woudn't be causing memory to spike. It has to be something inside the loop tag. On Thu, Jun 26, 2008 at 2:35 AM, Sam Roach <roach@improvementdirect.com> wrote: ----- Excess quoted text cut - see Original Post for more -----

This empty loop as posted here will cause the same memory spikes.  You can create just a simple .cfm page and run this and watch the memory go out of control. Maybe it is a Coldfusion bug? <cfset outsideloop = 20000000> <cfloop from="1" to="#outsideloop#" index="x"> </cfloop> thanks, -- Sam On Thu, Jun 26, 2008 at 5:50 AM, Brian Kotek <brian428@gmail.com> wrote: ----- Excess quoted text cut - see Original Post for more -----

Well I'm not sure it is a bug, you're talking about 20 MILLION loop iterations. If nothing else you're going to get 20 million carriage returns in the output buffer. You could try wrapping it in <cfsetting enablecfoutputonly="true"> and, assuming you aren't in a cfoutput block, see if that makes any difference. Essentially, unless you're positive that no content is actually being generated in these loop calls, that alone will probably cause a lot of memory usage. What, by the way, is "too much memory"? And what exactly is the issue with the memory climbing until the GC runs...that's exactly what its supposed to do. On Thu, Jun 26, 2008 at 12:48 PM, Sam Roach <roach@improvementdirect.com> wrote: ----- Excess quoted text cut - see Original Post for more -----

By the way, this runs in about 5 seconds and uses no memory on my machine. <cfsilent> <cfset outsideloop = 20000000> <cfloop from="1" to="#outsideloop#" index="x"> </cfloop> </cfsilent> On Thu, Jun 26, 2008 at 1:02 PM, Brian Kotek <brian428@gmail.com> wrote: ----- Excess quoted text cut - see Original Post for more -----

Thanks, the cfsilent did the trick.. -- Sam On Thu, Jun 26, 2008 at 10:04 AM, Brian Kotek <brian428@gmail.com> wrote: ----- Excess quoted text cut - see Original Post for more -----

Yes I am using CF8. ----- Excess quoted text cut - see Original Post for more -----

Sam Roach wrote: ----- Excess quoted text cut - see Original Post for more -----