Wednesday, March 5, 2008

Complete change

Changed the program completely. Now I upload all the data into one table as rows instead of columns. Earlier I was using columns because the database connectivity was very slow. But now with MySQL database and proper jdbc connection the speed has improved considerably and hence selecting data from the table based on date / symbol is not an issue any more.

Also changed the calculation etc to happend through sored procs in mysql. It took some amount of effort to get good performance but finally managed to accomplish a decent performance. Now the program can automatically download data from the last day of updation till date and do the calculations at around 5 minutes per every day's records.

Simultaneously, created a new program for downloading data from nse site throughout the day. It works fine and takes about 12-14 minutes for downloading for all the shares at a network speed of 128 KBps.

No comments: