We’ve been having a long running problem in the office when running jobs on our list processing software. The list processing engine performs de-duplication and pattern matching on a very large list of names and addresses (usually hundreds of Gigs of data is processed).
Our normal setup for running these jobs is :
Client (running the processing engine) – Off the shelf Win XP desktop on AMD (The engine is optimised for AMD processors and doesn’t run at all well on Intel processors as it isn’t capable of multithreading).
Server (data store) – Win Server 2003 on Intel with 2 teamed 1Gb network adapters.
These processing jobs frequently take over 48 hours to run. Sometimes the jobs will cause Windows to throw a “Delayed Write Failed” error and write these events in the event log :