Offloading long running processes

Hi all, I'm uploading a file to a CFM page from Flex. The uploaded file needs   to be converted and this conversion can take a while. I want to free   the Flex app from locking the user into pending state while the   conversion takes place. Right now I have: 1) User uploads file via Flex app 2) CFM receives the file and copies it to a specified directory 3) using cfscript and com object I start the conversion 4) once conversion is done (can be 2 minutes or more) the page returns   the conversion info from the COM object (failure, success etc) to Flex   in form of XML 5) converted file is now ready to be loaded into Flex I'd like to make a change and inform the user as soon as the file is   uploaded, but then kick off my conversion process somehow so it can   run without the user being in a pending state. What are my options? I don't mind logging conversion status to a database and poll that   every 10 seconds or so, but how does the CF side work in terms of   kickstarting the conversion process after file upload and returning a   status to the right away? Should I simply run a scheduled script which checks for a conversion   queue in the database and handles them? I guess that's an option but   ideally I'd like the conversion process to be kicked off by the CFM   file that handles the upload so that conversion can start immediately. Any tips appreciated. Many thanks, Stefan

Are you on CF8?  Try a <cfthread>.  Just be careful, cfthread can cause trouble if used incorrectly.

Yes, I'm on CF8. I will take a look at cfthread. Thanks, Stefan email:   stefan@flashcomguru.com web:   www.flashcomguru.com twitter:   @stefanrichter On 19 Feb 2009, at 21:37, David Henry wrote: ----- Excess quoted text cut - see Original Post for more -----

Apparently CF Standard only supports two concurrent threads while the   rest is queued, not sure how useful that will be then. How does it work in detail, say I have 10 people uploading a document   at the same time wouldn't the page thread be essentially multithreaded   anyway (one for each user)? If so maybe I could just use cfflush and   then add the conversion code afterwards. Would that essentially be the   same as starting a cfthread from that page? Regards, Stefan On 20 Feb 2009, at 08:16, Stefan Richter wrote: > > Yes, I'm on CF8. I will take a look at cfthread. > > Thanks, > > Stefan >

Correct, each request is a separate thread.  The problem is that the response for the request remains open after a cfflush.  You probably want to spawn a thread that runs without a request/response tied to it.   cfthread allows the request from Flash to complete, cfflush would leave the response partially filled but still open.  If cfthread isn't an option then you could write a Java class to handle the threading.  Keep a reference to the object in the application scope so you can ask it when processing is done. <cfscript> //your implementation may vary if (StructKeyExists(application,"myFileProcObject"){ application.myFileProcObject=CreateObject("java","yourProcessingClass").init(); } threadId = application.myFileProcObject.processYerFile(file); </cfscript> <cfoutput>#threadId#</cfoutput> When you want to check if the file is done yet: <cfif application.myFileProcObject.isMyFileReadyYet(threadId)> File is ready <cfelse> File still processing </cfif> When ColdFusion doesn't do what I want, I break out the Java books.   Hope that helps. Stefan Richter wrote: ----- Excess quoted text cut - see Original Post for more -----

Thank you, I may have to revisit cfthread after all since cfflush is not working   properly for me. Here's why: I am uploading a file AND returning data   to the Flex client when the upload succeeds. If I cfflush that page I   get the response in Flex (as expected in   DataEvent.UPLOAD_COMPLETE_DATA and Event.COMPLETE) but the later end   of page processing is not captured by Flex (it 'disconnects' down   after the first flush) and any further information I'd like to send to   the Flex app never makes it. So the response that you say stays open   does not seem to be - maybe this is related to the upload operation   and a normal POST would act differently, I don't know. I am now thinking to either 1) see what cfthread can do to help 2) use cfflush as I have one and have the client poll the conversion   status via the database My challenge is to inform the client that the upload is done, but then   also inform him when the conversion is done. I guess I expect too much   trying to do this in one operation. Any thoughts? Polling would obviously be one possible workaround. Cheers Stefan On 20 Feb 2009, at 14:47, David Henry wrote: ----- Excess quoted text cut - see Original Post for more -----

Upload the file from flex to the CFM page.  Use cfthread to fire or the conversion. Conversion happens in the thread and the post ( of the file ) returns back to flex as completed.  At this point start a timer in Flex. The thread in CF converts the file and once the conversion is complete write the file name aswell as complete/failed ( I'll assume you put this conversion in try/catch ) to the database. As you mentioned earlier you don't mind polling so in flex use the timer which checks via a remote call or webservice every 10 seconds if the entry is in the database.  Once the entry appears the conversion is complete and the Flex client can be informed. Paul. Thank you, I may have to revisit cfthread after all since cfflush is not working   properly for me. Here's why: I am uploading a file AND returning data   to the Flex client when the upload succeeds. If I cfflush that page I   get the response in Flex (as expected in   DataEvent.UPLOAD_COMPLETE_DATA and Event.COMPLETE) but the later end   of page processing is not captured by Flex (it 'disconnects' down   after the first flush) and any further information I'd like to send to   the Flex app never makes it. So the response that you say stays open   does not seem to be - maybe this is related to the upload operation   and a normal POST would act differently, I don't know. I am now thinking to either 1) see what cfthread can do to help 2) use cfflush as I have one and have the client poll the conversion   status via the database My challenge is to inform the client that the upload is done, but then   also inform him when the conversion is done. I guess I expect too much   trying to do this in one operation. Any thoughts? Polling would obviously be one possible workaround. Cheers Stefan On 20 Feb 2009, at 14:47, David Henry wrote: ----- Excess quoted text cut - see Original Post for more -----

Sounds good to me, thanks. Stefan On 20 Feb 2009, at 15:33, Paul Kukiel wrote: ----- Excess quoted text cut - see Original Post for more -----

----- Excess quoted text cut - see Original Post for more ----- No, CFFLUSH will simply keep the request open for additional information after sending the contents of the buffer. And CFTHREAD won't be helpful to you on CF Standard - the concurrency limitation is intended to coax you into buying Enterprise if you want those features. Instead, I suggest you use the "poor man's" asynchronous processing - CFSCHEDULE. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ Fig Leaf Software provides the highest caliber vendor-authorized instruction at our training centers in Washington DC, Atlanta, Chicago, Baltimore, Northern Virginia, or on-site at your location. Visit http://training.figleaf.com/ for more information!

I have been thinking about a few of the new features in Flash 10.  One of those is the ability to load files into memory on the client.  So you could load the image as a byte array on the client and use a remote object call to send the data to CF do the resize then all your waiting on is the result event and you can do whatever else you need to in the mean time. At the moment my idea is theory but I think this will work. Here is some reading on how to read files with Flash 10. http://www.mikechambers.com/blog/2008/08/20/reading-and-writing-local-files- in-flash-player-10/ Paul Kukiel Hi all, I'm uploading a file to a CFM page from Flex. The uploaded file needs   to be converted and this conversion can take a while. I want to free   the Flex app from locking the user into pending state while the   conversion takes place. Right now I have: 1) User uploads file via Flex app 2) CFM receives the file and copies it to a specified directory 3) using cfscript and com object I start the conversion 4) once conversion is done (can be 2 minutes or more) the page returns   the conversion info from the COM object (failure, success etc) to Flex   in form of XML 5) converted file is now ready to be loaded into Flex I'd like to make a change and inform the user as soon as the file is   uploaded, but then kick off my conversion process somehow so it can   run without the user being in a pending state. What are my options? I don't mind logging conversion status to a database and poll that   every 10 seconds or so, but how does the CF side work in terms of   kickstarting the conversion process after file upload and returning a   status to the right away? Should I simply run a scheduled script which checks for a conversion   queue in the database and handles them? I guess that's an option but   ideally I'd like the conversion process to be kicked off by the CFM   file that handles the upload so that conversion can start immediately. Any tips appreciated. Many thanks, Stefan

Thanks Paul, however I'm not necessarily using images but all sort of documents   that need a server side conversion (not just a resize). I'm quite   clear on what I need to do on the Flash side, it's the CF end that I   need the most help with I think. Cheers Stefan email:   stefan@flashcomguru.com web:   www.flashcomguru.com twitter:   @stefanrichter On 19 Feb 2009, at 21:59, Paul Kukiel wrote: ----- Excess quoted text cut - see Original Post for more -----

OK.  But from what I have read you can read "any" file into a byte array and send it to CF not just images my emaple happened to mention images however. I had a similar uploaded where I was uploading upto 100 images from a Flex client to cf backend but was resizing them and cfthread because of the Thread restriction on CF Standard in my instance almost did nothing at all to help. Paul. Thanks Paul, however I'm not necessarily using images but all sort of documents   that need a server side conversion (not just a resize). I'm quite   clear on what I need to do on the Flash side, it's the CF end that I   need the most help with I think. Cheers Stefan email:   stefan@flashcomguru.com web:   www.flashcomguru.com twitter:   @stefanrichter On 19 Feb 2009, at 21:59, Paul Kukiel wrote: ----- Excess quoted text cut - see Original Post for more ----- http://www.mikechambers.com/blog/2008/08/20/reading-and-writing-local-files- ----- Excess quoted text cut - see Original Post for more -----

Not familiar with CFM, but is the CF code? If it is, how about <CFTHREAD>? Dan On Thu, Feb 19, 2009 at 4:21 PM, Stefan Richter <stefan@flashcomguru.com>wrote: ----- Excess quoted text cut - see Original Post for more -----