Verity options

So we've got a site with not a lot of content, I suppose somewhere between a full blown content site and one with 2 pages :) We'd like to have it spidered since our pages are all modular and included so there's not really a page to search. I'm playing with Vspider in Verity K2, but can't seem to modify the collection. I've managed to <CFEXECUTE> some net start and net stops but can't get it right. Anyone been down this road? What I'm trying to do is create a collection of the entire site, then go back and modify the collection remove particular files and directories. J John Wilker Web Consultant/Writer www.red-omega.com <http://www.red-omega.com/>;

I know very little about the vspider, but... Doesn't it respect robots.txt files? If you properly build your robots.txt files, you should be able to automatically exclude the files and folders you don't want indexed. Just a thought, Jerry Johnson

mmm I'm not sure. I would suppose it would respect robots.txt  I hadn't even thought of that. Thanks! I'll try that.   _____ Sent: Thursday, May 06, 2004 1:52 PM To: CF-Talk Subject: Re: Verity options I know very little about the vspider, but... Doesn't it respect robots.txt files? If you properly build your robots.txt files, you should be able to automatically exclude the files and folders you don't want indexed. Just a thought, Jerry Johnson   _____  

Another question I'm wrestling with. The robots.txt should solve one of my issues, I'll test that today. What about modifying the collection? Vspider has to spider localhost so I need to modify the keys to reflect our production server address. I can't seem to get it to do it. Do I need to stop the K2 to modify as well? J.   _____ Sent: Thursday, May 06, 2004 1:54 PM To: CF-Talk Subject: RE: Verity options mmm I'm not sure. I would suppose it would respect robots.txt  I hadn't even thought of that. Thanks! I'll try that.   _____ Sent: Thursday, May 06, 2004 1:52 PM To: CF-Talk Subject: Re: Verity options I know very little about the vspider, but... Doesn't it respect robots.txt files? If you properly build your robots.txt files, you should be able to automatically exclude the files and folders you don't want indexed. Just a thought, Jerry Johnson   _____   _____  

I am not sure if it is better to modify the keys in the collection, or just adjust them while outputting to the screen. I have been handling final tweaks to the URL on the display page after getting the results but before displaying them. I also handle sorting and grouping in this step. Jerry Johnson

that's not a bad idea. Thanks Jerry, much obliged!   _____ Sent: Friday, May 07, 2004 9:04 AM To: CF-Talk Subject: RE: Verity options I am not sure if it is better to modify the keys in the collection, or just adjust them while outputting to the screen. I have been handling final tweaks to the URL on the display page after getting the results but before displaying them. I also handle sorting and grouping in this step. Jerry Johnson   _____