After waiting weeks of meaning to add this feature I finally did it. It actually took me less than an hour when I thought i was going to have to write all sorts of new code and that everthing would somehow end up being far more complex than I wanted it to be. Simple feature added, now instead of telling the system to crawl an entire site, you can tell the system to add a single page to the database. This makes it easier when finding an article, that links to entirely useless data, but should be added. So I am glad i finally took the time to add this simple feature. It was also good to see that I still remember alot more of the code structure on the spidering system than i would have thought I remembered. Starting next week I am going to finish making the system entirely automated. I should be able to finish that in a couple days. Then I am going to make the system very general so it doesn’t have to remain so specific to special education and then the same code base for newsshaker would be adoptable to other systems such as the HAMCOD project (which is a horrible name, but since I am more interested in just working on the idea for now I am not going to spend any time working on a name until success full. Man I could make some amazing progress on the system if i could get about 3 people coding on these machine learning systems. Oh well it is good to be back and making some progress again.



blog comments powered by Disqus
Dan Mayer Profile Pic
Welcome to Dan Mayer's development blog. I primary write about Ruby development, distributed teams, and dev/PM process. The archives go back to my first CS classes during college when I was first learning programming. I contribute to a few OSS projects and often work on my own projects, You can find my code on github.

Twitter @danmayer

Github @danmayer