HunterDavis.com

Ok, so here’s the deal. I write a lot of C in my line of work, so when I get the chance to write some clever shell script, I relish it. I do a lot of long supercomputer simulations, which tend to be hard to debug (especially when a problem arrises 3 days into a 4 day run). This is where logfiles come in handy, I know I’m not alone in this. Unfortunately, for really long runs these logfiles can add up to hundreds of gigs of space, which is a hard to come by commodity on supercomputing clusters. I found numerous solutions online, all of them tending to be long and overly complex shell scripts… No thanks!, when I want something done on shell it needs to follow the shell paradigm, small and powerful.

The problem:
I want to see the logfiles as they are created, but I also want them stored and compressed and not taking up space.

The solution:
———————————————————
| tee >(gzip > logfile.tgz)
———————————————————
Let’s break it down one bit at a time
| tee
Standard output is piped through tee, the unix t-shaped pipe, pretty much comes standard with all *nixes. Tee has two outputs, first it outputs to a file (or file handle), and second it pipes the output to the screen.

>()
This right caret/parenthesis pair comes in very handy. It opens a sub shell, leaving an implicit file handle that can be piped to. Tee sees this as a regular file, and begins piping standard output to it.

gzip > logfile.tgz
Gzip defaults to standard in for the input if no file handle is given. It uses lz77 encoding along with Huffman trees in a 32k sliding window. Decoding data (Huffman trees) are placed at the beginning of each block. This means

——————————————————–
| tee >(gzip > logfile.gz)
——————————————————–
Putting it all together, we pipe standard output to tee which pipes to screen as well as the implicit file handle created by the sub shell. The sub shell is gzipping the piped standard input at maximum compression and outputting the resultant gzip file to logfile.gz. Just pipe any huge log file through this bit of code, and you’ve taken your space requirements from gigs to megs, with virtually no cpu/memory overhead. Handy, and fits easily into most launch scripts.

Tags: Automation and Robotics, Programs // Add Comment »

Ok, so here’s a fun bit of perl code I wrote last night. It’s called snesaver, and you can download it here. Here’s how it works,

1. You’ll need to be running some flavor of *nix (self=ubuntu), with perl installed
2. You’ll need to have xscreensaver set up as your screensaver (though it should be simplicity itself to do this for most any other screensaver)
3. You’ll need to have zsnes installed
4. You’ll have to supply your own roms (legality), and record your own rom state movies in zsnes (simple simple!)

And that’s about all you’ll need. Just open up your ~/.xscreensaver file and put an entry for snesaver.pl (should be in your path) under the “programs” section. Then edit the script to point to /your/rom/directory/structure/ and you’re all set.

Here’s a youtube video of it in action.

/H

Tags: Multimedia, Programs // Add Comment »

Easy Execute - An extremely easy to use C++ library for spawning processes and executing arbitrary commands.
This library is intended for developers who wish to simply execute some command, without worrying about the underlying
fork/exec/system calls, string parsing, setting up timers, recording the run time of the command, etc. This library
makes it all completely plug and play. GPLv3.

Tags: Programs // 2 Comments »

I’ve uploaded a video to youtube of casey (my pointer hound) doing some tricks. Fun!

Tags: Multimedia, Personal // Add Comment »

This should significantly increase our capture and processing speed. First, install libcv1 in ubuntu, I like to get the documentation and python bindings as well, and these will install libcv1 anyway. sudo aptitdue install python-cv opencv-doc Now we’ll start writing some C code. As I have the benefit of writing this article after the code, I know that one can render/display 30fps without processing, and can render/display without lag with processing as well (thank you compiled code!). You can compile opencv code in linux using the following command:

gcc `pkg-config --cflags opencv` `pkg-config --libs opencv` -o MY_PROJECT_RUNME MY_PROJECT.cpp

Well that’s simple enough. The structure of the program has a few changes.

1. Program takes first command line argument as the image filter size (try 2-10 for good results)
2. Program no longer does image subtraction for a mask over the image, it’s direct processing now
3. Program is significantly (10x) faster, depending on speed of camera frame grabs
4. Check out the openCV tutorial for grabbing images from a camera, then check out the code below..
5. You can snag it here
6. Also, please note that ‘escape’ will end the loop and finish the program

Tags: Automation and Robotics // Add Comment »

Ok, now that we have a working home automation server setup (see article 1), let’s plug in a lamp module and have it flash when our gmail arrives. First up, plug in the light module, select your house code (’heyu info’ will tell you your house code) and an unused x10 number on the dial.
Find a lamp you’d like to have flash (or turn on, or dim etc etc) and plug it into any other outlet. Turn on the lamp, you want to be sure the light is actually on before you start cursing at your x10 module. Now plug the ‘on’ light into the x10 module. Let’s try turning on the module with a heyu on A3 Where A is your house code and 3 is your module number. Voila, you should have light. a quick heyu off A3 will get us back to the off state. There are many heyu options (dimming !) to try out, but they all work in this manner. Now, let’s install gmail-notify with a sudo aptitude install gmail-notify Gmail notify is a great python script to notify you when you have gmail. We will insert an execute command into the python code (very simple) so when gmail-notify runs and we have an unread email, flash the light (or dim the light, or turn on your lava lamp, etc.)

Fire up your favorite editor (I prefer vim, but most guides prefer nano) sudo nano /usr/lib/gmail-notify/notifier.py Now head down to line 208 (it was 208 as of 9/07). You are looking for this block of code: if attrs[1]>0: print str(attrs[1])+” new messages” We will change it to: if attrs[1]>0: path = ‘/usr/bin/gmflash.sh’ os.system(path) #execute gmflash print str(attrs[1])+” new messages” What we’ve done is told gmail-notify to execute the script /usr/bin/gmflash.sh when there is new gmail. Now we’ll need to create this script with a sudo nano /usr/bin/gmflash.sh Here we’ll tell the x10 light to flash on, then off. It takes my x10 module 1 second to deactivate a light and 1 seconds to activate one. It takes my light 2 seconds to ‘warm up’ to bright. We’ll want our script to pause for at least (1 + cycle time + bulb time) seconds, so in my case that’s 4. Insert into your editor window: heyu on A3 sleep 4 heyu off A3 Save it, close it, and make it executable with: sudo chmod +x /usr/bin/gmflash.sh

That’s all there is to it. Fire up gmail-notify, put in your gmail settings, and send yourself a test email to check. If your light doesn’t flash, try executing /usr/bin/gmflash.sh. If this doesn’t work, time to re-check your setup. Fin.

Tags: Automation and Robotics // Add Comment »

A little tic/toc ery has shown that without an image draw scilab can process 3 fps. So here’s a loop unrolled version that delays image rendering till 3 dots have been drawn.
//”paint” a scene with a laser pointer in realtime
//speed of update depends on speed of camera + processor

//could also easily be used on a video file

n = camopen();

for idx=1:15,

//give the camera time to auto white balance

im1=avireadframe(n);

end;

im3prime = rgb2gray(im1);

//save our “primary scene”

im3 = im3prime; imshow(im3);

//show us our primary scene

r=x_message([’Baseline Set’],[’Ok’]);

//let us know when to laser pointer

for ido=1:15, mask = zeros(im3);

//clear out our mask quickly

for idx=1:3,

//or however many frames/sec you can process

//tic;

im2=avireadframe(n);

//read a frame in

//subtract the greyscale current image from the primary scene

//then take that logical array, convert to numerical and

//use it as a mask over im3

mask = mask + bool2s(imsubtract(rgb2gray(im2), im3prime) > 50);

//imshow(im3);

//toc

end;

im3(mask == 1) = 255;

imshow(im3);

end;

avicloseall();

Tags: Automation and Robotics // Add Comment »

For some time now I’ve been into computer vision. However, much of computer vision is done on matlab, a 1000$ piece of software that doesn’t jive with my open source philosophy. Luckily, there are a number of open source alternatives. I prefer scilab, though octave is a good alternative. However, scilab has the advantage of an open source video toolbox. In this first article, I’ll show you how to setup scilab, opencv (the intel open computer vision library), and the sivp toolbox. Then we’ll verify its working by processing video from an avi file or in this case, a live webcam stream.

1. First, lets start by installing scilab and preparing our system to compile ffmpeg and opencv

sudo aptitude install scilab

2. Now follow this guide to get opencv and ffmpeg compiled. Remember to make sure ffmpeg, v4l, and v4l2 are compiled options, or we’ll be unable to process video, webcam, or newer webcam video
3. Now head on over to the SIVP webpage and follow the typical ./configure && make && sudo make install
4. At this point we should have video streaming in scilab. Fire up scilab from the command line scilab
5. Here, we’ll want to make sure the video/image toolbox is loaded. Hit the toolbox menu, then the sivp toolbox. You should get a message about it being loaded
6. Now, hit the examples button, and make sure your main scilab window stays open. From here click the sivp section, and we can grab straight from a videocam, image, video etc.
7. The best part about the examples is the (very simple) code is displayed in your main scilab window, very cool!

From here the sky is the limit!

Tags: Automation and Robotics // Add Comment »

Ok, so i’ve got this old clamshell ibook. The main weakness of these models was the screen, a paltry 800×600 resolution. The main sweetness of the project:

Control your house lights from your phone, any web browser, any ssh client, etc

Automate your lights, run complex temperature analysis and face detection.

Run ubuntu linux, fully support all hardware functions on ppc architecture, failsafe in case of power failure .

Dns forwarding from easy to remember address.

Future development: laser calibration.

Ok, first thing’s first, what will we need?
1. ibook clamshell
2. 1x usb->serial adapter (~1$)
3. 1x x-10 heyu compatible adapter (~10$ ebay)
4. whatever x10 controllers you’d like (lights, power outlets) cheap on ebay
5. any old usb camera should work (firewire maybe? stay tuned!)
The ibook has the following specs, but any computer ~ these specs would be fine:
366mz g3 ppc proc.
368mb ram
10gb hdd
800×600 lcd
1x usb 1.1 port
1x firewire port
Ok first of all, let’s get linux installed on here. Because the cd-rom drives tend to die in these units, this was the case here. The solution was to burn an ~12mb iso image to a cd - The ubuntu mini ppc.iso image, and keep retrying this boot disk till the laptop booted the cd (about 20 tries). You could also use any other macbook in firewire host mode (check the mac forums for that). The feisty image (works great!, install xubuntu) supports the airport card and the Ethernet port naively, so that was nice. Remember to hold the ‘c’ key on your mac to boot from the cdrom.

While you have that running, let’s do some multitasking. First head over to dyndns and get yourself a dns forwarding address (if you don’t want to have to remember your constantly changing ip address). While you’re at the site, check out their guide to inadyn, some cool open source software to update dyndns with your dynamic ip. Also, if you have a router, make sure to forward the ports for whatever services you want (20-25 ssh, telnet, ping etc, 80 or 8080 etc for web, 5090-5092 vnc, etc)

Ok so at this point we’ve got an ibook running xubuntu feisty, let’s install some packages. We’ll want to:
Rock open a terminal and break out your su hat. sudo aptitude install the following packages and the firewall software of your choice.
openssh-server //this is if you want ssh access to your machine
inadyn //this will automatically update your dydns
tightvnc-server //if you’re into vnc
gnome-power-preferences, gnome-power-settings //easy power profiles (i.e. close the lid, blank the screen)

Now it’s time to plug in you usb->serial adapter. I broke down and paid 5$ on ebay for one, but I’ve seen them locally for about 1$. In a terminal, cat /var/log/messages, and look for lines like ‘usb 1-1, pl2303 (or your chipset) converter now attached to ttyUSB0′. This is your new serial port, and what you’ll specify to hey-u (software to control your serial port controlled x-10 power line controller). Now go download hey-u. In terminal, run the whole configure/make/sudo make install shebang. The install is pretty user friendly, and they ask you for your serial port (which we got earlier from /var/log/messages). Time to try ‘heyu info’ Fingers crossed…
It works!

It should be immediately apparent if your device is found, you’ll get firmware info as well.

At this point things are coming together. We’ve got remote access from any ssh capable device to a command line interface to all the x10 power devices in our house. Next up, if you have a way to get jar files onto your phone, or your phone has a browser, I highly recommend midpssh from here.

At this point the sky is the limit, but first thing I recommend is installing a web frontend to hey-u like the one found here. So for this, we’ll need apache and php. So get your fix with a ’sudo tasksel install lamp-server’. I also recommend that right after installing apache, you ’sudo touch /var/www/index.html’, so your web directory isn’t open. How about a fusion of web/ssh access?
I recommend grabbing the isnetwork release of mindterm ssh for java applet from here
Just move everything in the applet directory in the zip into a folder on your website, change the netscape.html to index.html, and you’ve got an ssh client available from any computer that has java. Sweet.

Tags: Automation and Robotics // Add Comment »

Sometimes I feel like all the world’s problems can be solved with shell scripts. Here’s one that’s short and sweet. You give it a PID, interval, and outfile, and it keeps track of the PID given till the PID dies (or you kill the script). These values are plotted against time with gnuplot. I wrote it while keeping an eye on ‘top’ on a server. Snag it here

Tags: Programs // Add Comment »