Main

Resources Archives

December 17, 2011

Cleaning up files after hacking

This blog was hacked awhile ago. It was annoying, I hadn't updated my blog software for awhile and there was a security hole. It was a good learning experience of how to deal with the intrusion. It was also nice to be able to quickly whip together some scripts to help clean everything up.

This let me see which files were recently modified

find . -type f -mtime -3 | grep -v "/Maildir/" | grep -v "/logs/"

The exploiters were modifying my .htaccess and adding a few files of their own, this deleted their files

This code helped remove exploit code that was injected into all of the pages on the site.

After upgrading my software and installing security patches, a bit of other cleanup, and file permission fixes everything was back to normal.

October 29, 2011

Git clone from local filesystem to fetch local branches

So I am working on a project that has a very long test suite. Yes, that is a problem and we should start to do something about it. For now that problem exists but it leads to wanting to do some weird things. I wanted to work on code in a branch, while my tests were running in another. I had cloned the project twice so I had two versions, of the project, the issue was the branch I wanted to work in hadn't been pushed remote. I then realized since it is git I could easily solve the problem by pointing my second version of my project at the first and fetching all of my local branches! Git is pretty great that you can solve things like this. It also means you could easily just run git off a shared network filesystem, no reason to really setup a fancy git server.

Here are the steps make a second version of a project and then fetch the local branches from your first version

# if you haven't clone a second version of the project do that from the filesystem or the git server
> cd project_2
> git clone /Users/danmayer/projects/project_1
> Cloning into project_1...

# in the second version of your project add a remote repo pointing at the original version of the project on your filesystem
> cd project_1
> git remote add local /Users/danmayer/projects/project_1/
> git fetch local
> git branch #=> a list of the branches on your first projects directory

This is a pretty simple thing to do, but when I ran into the issue it took me a bit of time to realize how simple it was to solve this problem. It also just illustrates just how versatile git really is.

July 4, 2011

Google Closure Compiler And Ruby

I have repeatedly needed to compress / minify a collection of javascript files. Which I had been doing by hand with Google's closure compiler. I started to get annoyed with this process and went looking for a better solution. If you are using Rails, there is a great solution using Jammit asset packager library for Rails. I was not using rails, and just needed something that could write the files on demand. A bit of quick searching came up withh this post Compressing Javascript Code With Google Closure Compiler And Ruby. It basically met my needs, but needed to be modified a bit to take a collection of files and write that collection of compresses files back to disk.

Here is the solution I ended up with:


June 5, 2011

Emacs swap buffer window method

So I really like split screen in emacs. I am in that mode pretty much all the time. One thing that annoyed me was opening a file on the side of the screen I wanted. Sometimes I want code left, tests right others it is project A left project B right or often server code left, client code right. Anyways I often opened a file on the wrong side of the screen and wanted to move the buffer. Previously I always closed the buffer, then switched to the other emacs window (C-x o) then reopened the file. This annoyed me to know end, so I finally went looking for a solution. I found something that was close, but not what I wanted, which swaps two buffers positions left and right, so I started with that code and modified it to do just what I want. I then bound the function to "C-x /" which wasn't in use and I can now swap buffer windows with ease. Also, kind of fun to do a bit of lisp/emacs hacking and breaking out of my Ruby world a bit.

If you notice I have it also close the current tab for the buffer and move the cursor context to the new buffer on the other side of the screen. Most people hate and don't use tabs in emacs so that one tab closing could be commented out. I love tabs because I am a very spacial person and the tabs let me keep the physical location of the buffers in my head. This was written to run in Aquamacs, but will work equally well in standard emacs. Just drop it in your .emacs file or emacs config directory.

Code can be found at this gist, Emacs swap buffer window

May 14, 2007

Good Startup Resources

Here is a list of some good start up resources that I have begun to check out on the web.

Paul Graham's essays

Clay Shirky's essays

OnStartups

Small Business Hub

Startup news

programming.reddit.com

Good start up books to read

Hopefully I will grow this list a bit more shortly, but for now that is some good information to get your started.

April 5, 2005

CVS stuff

I always am forgettting this and that about CVS and this is just a really good resource on all the CVS stuff you would need to know when working with a regular CVS repository.

Command Line CVS

November 8, 2003

OpenGL Resources

This is a collection of various tips and things I use or have done in OpenGL. This is actually up on the web more as a refrence to myself than anything else. This is a collection of good links, programs, tutorials, and source code. That I have found usefull, created, or thought would be usefull to others. So enjoy any and all of my OpenGL info.

Projects:

3D shooter I wrote, it could obviously be improved alot but it is a good easy start.

3dshoot.jpg

You can download the source and the exe and everything and mess with it yourself. This was written in Visual Studio.net Get it


I also made a fractal program into a openGL winamp plug in. If your interested in doing conversions to a winamp plug in yourself, you can download the source and the DLL for winamp and mess around with this.

Get the winamp code


Links:

A group of scanned 3D object models, and how to read and write them.

NEHE the best place to go for open GL resources.

November 2, 2003

Linux / Unix Resources

This is a collection of various tips and things I use in linux. This is actually up on the web more as a refrence to myself than anything else. I keep learning this stuff, but since I dont use it all that frequently I tend to forget how to do something exactly the next time i need to. So here are little commands I use and some tiny descriptions. If you have some tips tricks or think i am doing something the hard way please feel free to comment and share. If you have any Linux / Unix questions please feel free to post them if you think it is something I might be able to help you with. That said here is what I got...

Commands:
// secure copy. This allows you to copy a file to another computer over SSH
scp filename compname:~/. (exactly like this no spaces may occur anywhere in this)

//if something is in the background or you accidently suspended it you
//can bring it back to the front by typing
fg (in the command line)

//if you have sudo access you can do about anything on the system
sudo cmd (any command you want, then it will ask for your password)

//to quickly view a file
less filename.txt

//to open and edit a file
emacs filename

//to add to your path and edit many other settings this in your home directory
emacs .cshrc

//to get to your home directory
cd (or sometimes cd $HOME)

//FTP stuff
ftp domain.com
ftp> cd directory
ftp> put filename
ftp> get otherfile
ftp> quit

//To add users to the system
sudo adduser

//edit a users password if your root or can sudo
sudo passwd username

//to change your users password while logged on the system
passwd

//to give users sudo acces
sudo visudo
//to run a program and keep the console available
program &

//if you dont have the proper permissions for a file
chmod 666 filename (this gives read and write access to everyone, so it isn't secure, man chmod if you need a specific file rights)

TomCat:
To start tomcat run �bin/start-tomcat� from tomcats home directory
Tomcat installs /var/tomcat/webapps
Tomcat really /usr/local/Jakarta-tomcat

MySQL resources

This is a collection of various tips and things I use in MySQL. This is actually up on the web more as a reference to myself than anything else. I keep learning this stuff, but since I don't use it all that frequently I tend to forget how to do something exactly the next time i need to. So here are little commands I use and some tiny descriptions. If you have some tips tricks or think i am doing something the hard way please feel free to comment and share. If you have any MySQL questions please feel free to post them if you think it is something I might be able to help you with. That said here is mysql information...

MySQL Commands:
//to start mysql on the command line
Mysql �u root �p (It will then ask for your password)

//To start mysql server
Sudo bash
�bin/safe-mysqld� from the mysql root directory

//How to select data using rules involving more than one table from MYSQL:
Select * from user leftjoin link on user.id = from Left join msg on to=msg.id where user.name = �john�;

Database:mysql -u root �p *** //snowmass

//to create a text dump of a database
mysqldump �-user [user name] �-password=[password] [database name] > [dump file]

//to create a new database
mysql> create database somedb;

//granting privileges to a user
grant all privileges on *.* to NewsShaker@�localhost" > identified by 'passwrd';

//shows all the tables of a database
show tables;

//creates a new table in the database your currently using
create table "tablename" ("column1" "data type", "column2" "data type", "column3" "data type");

//creates a link table (which is really the same as other tables but just
//a way to associate data between tables
create table link (catID bigint, pageID bigint);

//deletes an entire table
drop table Pages;

//deletes an entry from table idgen where the item has a uid of 12
delete from idgen where uid=12;

//show only the colums of a table not the data
show columns from tablename;

//To change the data of an already existing entry
update tablename set columname = "whatever" where columnname = "something";

//to alter a table if you need to add a new field
alter table tablename add column_name column_type after column_name2;

//creates a dump of database called newsshaker
./mysqldump �u root �p newsshaker > ../../../ddmayer/newsshakerdb.txt
//at command line in the mysql/bin directory

//to recreate the database from the dump file
./mysql -u root -pPASSWORD newsshaker < ./newsshakerdb.txt
//at the command line in the mysql/bin directory

//to get the count in a category since a date:
select count(*) as co from Pages as p left join link as l on p.UID = l.PageID left join categories as c on c.UID = l.CatID where c.name like 'autism' and p.date > 20031201000000 group by c.UID;

//to get only 5 in order as the very newest:
select p.UID,p.date,c.UID from Pages as p left join link as l on p.UID = l.PageID left join categories as c on c.UID = l.CatID where c.name like 'autism' and p.date > 20031201000000 order by p.date DESC limit 5;

//to get a whole category:
select p.UID,p.date,c.UID from Pages as p left join link as l on p.UID = l.PageID left join categories as c on c.UID = l.CatID where c.name like 'autism';

//add a new user to your mysql system
mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'localhost'
-> IDENTIFIED BY 'some_pass' WITH GRANT OPTION;

//getting random results from a mysql query or mysql table
mysql_query("SELECT * FROM table ORDER BY RAND() LIMIT 1")

//renaming multiple tables at the same time.
RENAME TABLE old_table TO backup_table,
new_table TO old_table,
backup_table TO new_table;


Links:

MySQL home

My SQL commands tutorial

Java MySQL tutorial

Great java Mysql developer tutorial

Java Resources

This is a collection of various tips and things I use in Java. This is actually up on the web more as a refrence to myself than anything else. I keep learning this stuff, but since I dont use it all that frequently I tend to forget how to do something exactly the next time i need to. So here are little commands I use and some tiny descriptions. If you have some tips tricks or think i am doing something the hard way please feel free to comment and share. If you have any Java questions please feel free to post them if you think it is something I might be able to help you with. That said here is some java stuff...

First off i think the best way to work with java code and projects is using J Builder. It is available free for personal use. I highly recommend it. It is available for linux and windows. get it from Borland

If your going to develop web applications I also highly recommend apache tomcat. It is a free java (JSP) server. It is also available for both windows and linux which is great. get it from Apache Tomcat

If a java program keeps running out of memory, this is easy to fix. The java virtual machine is only given so much memory. So running:
"java -Xmx256m PROGRAMNAME" should fix the problem by giving the virtual machine 256 mb of memory. You will see this error as a java.lang.outofmemory error. If your using J builder go to project->run->Java VM parameters: and add the -Xmx256m to the parameter line.

Here's how you could rewrite that statement using the ?: operator:

System.out.println("The character " + aChar + " is " +
(Character.isUpperCase(aChar) ? "upper" : "lower") +
"case.");

The ?: operator returns the string "upper" if the isUpperCase method returns true. Otherwise, it returns the string "lower". The result is concatenated with other parts of a message to be displayed. Using ?: makes sense here because the if statement is secondary to the call to the println method. Once you get used to this construct, it also makes the code easier to read.

Files:
One of the first Java projects I did the source and some info on a AOL Instant messenger (AIM) chatterbot. Built using simpleaim, megahal and some of my own work. Most of the work was in combining the programs learning how they worked and such though. I did add a nice logger to it as well. This still needs alot of work to be that impressive. download MegaDan.zip

A part of my machine learning, for project News Shaker. Text2SVM converts text documents into the proper format for use with SVMlight. It is fairly well commented and contains the full source code. This is a good easy way to learn to work with reading and writting files since it has alot of IO. Visit this projects own page, Text2SVM.

My edited and improved version of weblech. Weblech is a web spider. I added many features and costumized it for my own purposes. This crawler uses google to find files that are relavant to whatever topics your working on. Download the improved weblech from here

Links:
A great starting point for building a java web spider (Weblech)

My edited and improved version of weblech

A great Java Word Counter / dictionary builder

Stop List

Parsing Text

Java MySQL tutorial

Java Porter Stemming Algorythm

Java Lancaster Stemmer

Sorting a hash table in java

Java sort class (sorts most types)

a great java machine learning site

Pretty much everything you would need to work with java Email

How to work with multipart email messages

Great Java Swing Tutorials, especially the stuff on the grid bag layout

Great webservices and java XML info

The 10 best and unervalued least known about java libraries. some of these would be great to use in many projects without reimplementing there feautres yourself.

Web 2.0 craziness

View Dan Mayer's profile on LinkedIn


I Power Seekler
I Power Seekler

www.flickr.com
This is a Flickr badge showing public photos and videos from mayer_dan. Make your own badge here.

Creative Commons License
This weblog is licensed under a Creative Commons License.