I went down and tried to figure out the problem. I had little luck determining what was wrong but I did get the problem to manifest again. I then succeeded in cleaning the filesystem on the main volume on the server which had some corruption on it. Since then I haven't seen the problem so hopefully that fixed it. I'm going to go back to the factory tomorrow morning and attempt to clean the remaining filesystems and replace some disks which are really old.