How to: Read the Data and Find Out Where the Failure Is
What you are looking for is to see when the job was sent from the Job Scheduler and whether Source DD got the job or not.
If you find the Scheduler job log line for the job and there is no matching DD Job data to be found in the DD Keep folder, the problem is that Scheduler did not send the data to the Source DD. In that case you need to look at the DD Client Control debug log that is located at the host where Scheduler is running (Client control logs), and look for the time frame when the data is missing and see if there are any errors while Scheduler tried to send the job to DD. You can also look at the Router log to see if there was any incoming job connection during that time.
If the job file is not missing, then look at the Counter range from the Scheduler Log, and in this example you want to see 2858 to 2861:
Now the tricky part is to find the Job Data file: you can use the time stamp of the file to figure out which one it could be, but when you find file, open the xxxxx-HEAD.XML file for that job, make sure it has the correct JobId (job ID from the HO Scheduler), and status Done. Look at what the ErrMessage says, how many records were affected. This tells you how many records total DD picked up.
Open the xxxxx-IDAT.XML file, look at the Replication counter range, and see if it matches the one sent from the Scheduler. First, you will see some Table data setup, just look up the section after that, ParamFieldDef, and there you can see the counter range:
To see if all the Data was pulled for that range, open the xxxxx-RDAT.XML file. The tricky part here is to know where the Replication counter field is, but you can just look up the counter number. Each record is separated with <VLD Cnt=”xx”> where Cnt is the sequence counter for that record, so you can check if you have 4 records here and the Replication counter is correct in the range of 2858 to 2861 (both records included):
In the DD logs, you can look up if there are any errors that came up, and if the Records affected are not the same ones from both log files. Look for the Guid Job ID, and then check the processing info.