Dan Mayer's Blog

Well we did paris. It was fast but we saw a ton of stuff. Also going up the eiffel tower was cooler than i would have expected. So it was a good time. We did a very quick run through Amsterdam for one ni ght we leaev for germany later this afternoon. We have been able to get around very easily through out europe. We have been having a blast and just kind bouncing all over the place it has been great. Some places are just so different. I can't get over having to pay for bathrooms. It just is so wierd to me. Also w ater is worth it's weight in gold here... It is so hard to get water anywhere. We did get a full case of beer for 7 bucks. That was pretty cool. Anyways I guess i should get off the computer now and see what it up next.

P.S. Happy Birthday I am sure you will see this... hehe

Hey i am in munich now. I saw a concentration camp today which was wierd and scary. It was a good thing to do though i think. Like one of the walls there said, `Never Forget'. i figure that is something important to remember. I have been having a blast everywhere i have been. The overnight sleeper train car with all four of us packed it in was an adventure and a half it was hilarious. (we were in that little room for about 12 hours). We are going to see castles around here pretty soon either tomorrow or the day after.

We have done a good job balancing partying and seeing stuff. I havent written enough in the journal with me but i guess you really cant capture a trip like this anyways. I have done more reading than expected which is cool. Oh well Lots of thinking and ideas during this trrip... it is going to be hard to go back to work after this trip. It has been very cool. There are tons of odd little differences between here and home. I will never get over having to pay to go to the bathroom. wierd!

I am back and it was a wonderfull time. I really thing i learned alot had time to reflect, time to party, and all around just had an amazing time. I am sure this trip will affect me in a positive way for the rest of my life. It was almost scary at the end of the trip to realize how much I have actually grown up in the last four years. (as well as all of the people I was travelling with.)

Some Wierd little things from Europe:
*You have to pay for ketchup.
*You have to pay for toliets.
*You get good free breakfast at every hotel/hostel.
*Wine and Beer are far cheaper than water.
*French tap water smells like ham.
*British people shouldn't be allowed to cook.
*French restraunts rename their cheese, munster apparently means vomit smelling Brie.
*German train conductors are scary as hell.
*You learn to appreciate Mcdonalds a bit just because you know what the food is.
*You can be a veggie in europe although it is a bit harder. I am entirely sick of cheese sandwhiches.
*Everyone in Europe seems to love their beer far more than we do. Everyday we saw locals drinking large beers with their lunches. I don't think getting a mild buzz on with lunch would be considered a good thing in most american companies.
*The Hofbrauhaus is an insanely large german bar, that servers beer a litre at a time.
*It isn't that america has no culture, it is just europe has far more history. They have some massive amazing buildings that you can tour but if they were built in the modern times would be considered a gross miss use of funds. Castles like those will never be made by regular people again and that is probably a good thing.

Well I guess i am am adjusting to being back. I am on a almost normal sleep schedule now. I have still been extremely tired earlier than normal though. It has been kinda cool that I haven't had problems sleeping as I normally do. I have a big presentation at work on monday so i have been nervous about that and kinda laying low until I have that taken care of. If it goes well i might get to work with a bunch of really smart people on continueing work to expand my senior project from last year.

This morning was a crappy experience. Our power went out, along with our power went out my server. So since the power has been out for a couple hours my server has been down awhile. (Not the server my blog is on), but a good section of my website is currently missing in action and that makes me upset. Hopefully when I get home from work we will have power back and I can get my server up and running again.

Party on wayne, party on Garth.

A moment of awkward silence as the Olsen twins turned 18.

Care read more: Post on the olsens

I did a bunch of cool art today. I also ordered a ton of frames so i can frame my art and some of my cool posters so they don't get torn up (more torn up) during the upcoming move. I also helped nicole clean the hell out of our house.... yeahhh not as messy as usual.

I forgot some funny things about bathrooms in europe. Most toliets have two flush modes, half flush and full flush. So you don't have to waste water if your only peeing. The second thing is even the full flush or the pipes or something don't do as well as around here, our trip through out europe was also a trip leaving clogged toliets behind in every country. It was nuts seriously they clogged so easy. I guess the entire area of europe is on the double flush rule.

Anyways just thought i would share.

Imagine desperately holding on to a rope the only keeping you from falling down a pit. Imagine that rope breaking. Imagine falling lovingly holding on the rope that let you fall because it is all that you have left. Imagine...

A zillion thoughts have blown my mind in the last 20 hours. The best example would be: 2 six year old girls dressed like Brittany spears singing sir mixalot's baby got back on top of a boulder on pearl street. This shows where our society is headed.

A personal email from me to a business: My letter to gardenburger

Liz huffman defends football players calling a female player a cunt. Saying that the word can be used as a term of endearment. Don't beleive how bad this is read the full article.

After going through a little less than 3000 emails. I have finished testing nad doing any work on SVMMail. It still is sitting at a 97.5% accuracy. I am sure this could be increased, but I need to move all of my focus back to my primary project, News Shaker.

On the News Shaker front. I have added about 350 new manually categorized sites across the database. I am going to rebuild all of the models and see if the increased training data brings my percents up to a more reasonable level. Then once I have a little better percent accuracy I will begin all of the auto categorization code and just start to let the system go crazy and see how many sites it can categorize correctly when left to its own devices. Should be an interesting time next week. That is if the machine boots up. Someone was working on my system and now it freezes on boot up. I am sure all my data is still there and I have a fairly recent back up but hopefully this can be sorted out before the begining of next week.

I also have began reading Managing Gigabytes which is about compressing and indexing documents and images. I also ordered a new book about machine learning and artifical intellegence that i will begin reading soon. Perhaps they will provide me with some new ideas on how to improve my system.

It seems that when i was younger i learned more and more about how life worked. How people interacted and what friendship love and everything else ment. As I now grow older I think i can't really learn anything because there is far to many exceptions to every rule. I mean nothing about life can actually be known. You can't prove love, you can't prove you made a mistake, you can show who you should really be with, it is all just your beliefs and if someone elses beleifs don't match up with yours, your just screwed. Perhaps that is why I am into computers so often, I can prove somehting, I can figure things out. They never just disappear and leave me alone when I am still up and thinking.

A friend of mine posted that this is what i think about before you even get up (which is a quote from mallrats). I think the more appropriate version for me is this is what I think about after the world has fallen asleep.

I don't know if i will ever really know anything about life again. I used to know certain things. I used to believe certain things about people. Now I haven't known anything for sure about people for along time, especially people that I haven't know for years. Even those people tend to throw huge suprises and issues once in awhile.

I recently learned that someone that I have known for months was capable of something I would have never expected. I don't think anyone would have. They were nice, kind, and always shy and friendly. Turns out they will probably be going to jail for awhile now, because of their actions.

So what can you do about this? Nothing, all you can do is push yourself out in the world each morning and hope. Hope that you will make friends, find work, and meet someone special who shares enough of your beliefs and misconceptions about the world that you can be honest with them.

This is a description of a idea that may be the next phase of my project as I continue my work with text categorization. It right now is just a initial idea and outline of an idea so that I have some thought to begin working with when I get to the next point.

HAMCOD
Human Assisted Machine Categorized Open Directory

A collaborative human assisted machine categorized directory. That will extend the functionality of the ODP (http:www.dmoz.com) project. This project will combine a already large and extensive base of human knowledge, with text categorization and social collaboration techniques to increase the amount of well categorized and defined websites, without the need of such high levels of human interaction that are required for DMOZ. This projects goal is to eventually use machine learning techniques to replace the slow and time consuming process of human categorization.

This will have humans interact with the machine learning process as it contributes to and works with the categorized directory. This will have humans say when something is miss categorized and they will be able to recommend new re-categorizations. Or recommend deletion from the directory. The spider would crawl for new pages that aren't listed in the category, if a new page is discover by the spider, the system will attempt to categorize it to the lowest level of a category that it can.

So if there is a main category like "shoes" it might have a hierarchy like this:

1.______Shoes____________________Car______________________Cheese
2._________|________________________|_________________________|
3.Nike___Reebok__Vans________Ford____Toyota_________American____Cheddar

The system would first do categorization on the level of is this page about shoes. If it is determined to be about shoes it would them use different models knowing that it is a shoe to try to determine what shoe company the page is about.

In its early stages it will only work through the Computers category of the DMOZ project. Which is already contains 149,512 websites. To first determine if a site belongs in the computers category, I will need to get about 400,000 websites at random from other parts of the DMOZ directory. Alternatively, I could assume that if I can't find an appropriate sub category in computers that the site doesn't belong in the computers category at all.

Initially I will design the system with no log in or registration required. Since use will be practically non existent. Once use begins I will require an email address and to have that email address to be confirmed. This will make it rather hard for companies or individuals to artificially increase their rankings with in the directory. With in each category the sites will need some sort of ranking system initially I won't worry about ranking the sites. I will just assume that as you get specific enough there will only be a small number of sites in each category. (This seems to be true for DMOZ)

Liz "c-word" Hoffmann begins to cry when asked about her remarks on the use of C-word. Says she was pushed into it by a lawyer. I say she shouldn't have been protecting the asses of all sorts of athletic officals at a cost of millions of dollars to CU students. Now that I know it is ok, if i ever get a chance to speak to Liz, I will know what to call her.

Yeah, I am so ready to be done with CU.

Also if you have a good idea about something wrong going on, like many "un patriotic" americans have in the last few years, it is likely what you saw now wil get you hated and booed, and months from now it will be considered the popular belief as the facts come out. As with micheal moores booed off oscar speech and Hunter Thompsons critizism of Bush for leading america with similiar propaganda tactics of hitler. It is scary how many lies have been told to the public... Then only if found to be false, were considered a mix up.

After waiting weeks of meaning to add this feature I finally did it. It actually took me less than an hour when I thought i was going to have to write all sorts of new code and that everthing would somehow end up being far more complex than I wanted it to be.

Simple feature added, now instead of telling the system to crawl an entire site, you can tell the system to add a single page to the database. This makes it easier when finding an article, that links to entirely useless data, but should be added. So I am glad i finally took the time to add this simple feature. It was also good to see that I still remember alot more of the code structure on the spidering system than i would have thought I remembered.

Starting next week I am going to finish making the system entirely automated. I should be able to finish that in a couple days. Then I am going to make the system very general so it doesn't have to remain so specific to special education and then the same code base for newsshaker would be adoptable to other systems such as the HAMCOD project (which is a horrible name, but since I am more interested in just working on the idea for now I am not going to spend any time working on a name until success full. Man I could make some amazing progress on the system if i could get about 3 people coding on these machine learning systems. Oh well it is good to be back and making some progress again.

Well things are going really well at work right now. I just finished adding in a huge part of my project. Now my text categorization software has all of the pieces seperately fully automated. Now i just need a time management system the kicks off the process with the right amount of time and lets them finish before started the next part of the process. It should be pretty cool.

Well tonight at midnight is the official I dont live with a 12 year old anymore night. Thats right Dom turns 21, hell yeah and about time. hehe I dont think much is going on tonight because he has school tomorrow, but you should talk to me before tuesday night because i think a bunch of people are going out and it should be a good time. Yeaaa everyone is 21 dom's b-day party. It should be fun, but i actually have alot going on at work this week lets hope that i can balance everything out.

Well Dom turned 21 and it was a great time. We partied really hard core. Dom threw up in two of the bars bathrooms. He did make it to the bathroom and keep it clean both times so go dom. Also he refused to go home until the bars closed. We got home and dom threw up a bunch more while others played beer pong and foosball. It was a ton of fun and we got a bunch of good pictures. Here are just a few to show some of the night. Dom's ft collins friends couldn't come down but Matt and Jesse came down to represent that other college town in colorado... hehe

Anyways now we have to do all the 21 stuff that is normal... Coors tours, Old C's, liqour mart, denver bars, ft collins bars, gambling, and such... It should be fun to have a partner in crime to hit the bars with.

Click it to make it bigger!

The sun was gone
the nights are long
and i was left while the tears fall.

Swing Swing - the all american rejects

This is possibly the best prank i have every pulled. Well Dom, Nicole, and I did this. Scott was in europe and leaves soon for airforce so we decided we had to do something special to say good bye. So here are some photos of us putting over 100 lbs of newspaper in his room. We covered all his walls and all his items and filled his floor up about 4 feet up with crumbled newspaper. It was hilarious, he didn't even know what to think when he first opened his door. It was worth the 4 hours of our lives it took. Hehehe yeah Scott were going to miss you good luck in all that you do.

The last couple of weeks i have done alot of work on news shaker. I have done lots of testing. I all of the categories (about 12) to an approximated error average of 88%. For 12 categories this is really good. First i began by adding more and more data to the categories and rebuilding the models. This initially was increasing the percents but it ceased to help after all of the categories had about 90 documents in them. I then began to play with the weight of the positive terms. This was highly successful after increasing the weighting on all of the positive training vectors I could successfully take all of the training data and recategorize it with 88% accuracy with the remaining documents not wrongly categorized but declared to be of an unknown category. I then started real world testing giving all of my category unseen documents that were hand categorized. The results for the few real world tests i have done so far have been fairly poor, showing only 15-20% accuracy. I am not sure why that varying how the model is made dramatically increases percent of categorization of known documents but seems to have no effect on unseen documents. This currently is the problem i am working on. It is possitive to get known values accuracy for my models to range from 85% to 93%. After a little more real world testing and some other discussion i might be able to come to a conclusion as to what is going on between known and unknown examples.

Scary enough this makes alot of sense. Apparently many of my problems in life can all be traced to the fact that i am left handed. I should also die between2 and 9 years earlier (depending on the study you read). So i guess i have a reason now but does it just make it a self fullfilling prophesy (i know i can't spell and yet i still dont care or do anything about it.).

Lefties, have high frequencies of depression, drug abuse, bed-wetting, attempted suicide, lower-than-normal birth weight, sleeping disorders, and autoimmune diseases.

Oh well Scott has been home awhile it has been nice. Stuff at work has been going pretty well. MTRoom stuff is doing amazingly well. Other stuff has been a little boring lately, but hopefully i can remedy that situation. Tonight is guys night. Saturday is Scott's going away party / Nicoles birthday party. Starting with a nice fancy dress up dinner and then ending with all our friends draining a keg of beer (except me who has plans for some serious white russian drinking). That is all for now.