Pre-Processing of Data and Coding

Collected the data. Used a ftp client software to download data from pjm. All data was in the form of zip files. So, I used a batch zip extraction tool (Zipghost). Then, I downloaded JhLib library and used it for parsing csv files.

The following code was used to get the filenames of all the files from a folder:

using System.IO;

string directory = @"D:\Data\Real Time Hourly Market Price";
DirectoryInfo directoryInfo = new DirectoryInfo(directory);
FileInfo[] fileInfo = directoryInfo.GetFiles("*.*");
foreach(FileInfo fi in fileInfo)
{
Console.Write(fi.Name+", ");
}