Half a line of shell to display to screen while compressing output… The most useful shell I’ve written in forever
Ok, so here's the deal. I write a lot of C in my line of work, so when I get the chance to write some clever shell script, I relish it. I do a lot of long supercomputer simulations, which tend to be hard to debug (especially when a problem arrises 3 days into a 4 day run). This is where logfiles come in handy, I know I'm not alone in this. Unfortunately, for really long runs these logfiles can add up to hundreds of gigs of space, which is a hard to come by commodity on supercomputing clusters. I found numerous solutions online, all of them tending to be long and overly complex shell scripts... No thanks!, when I want something done on shell it needs to follow the shell paradigm, small and powerful.
The problem:
I want to see the logfiles as they are created, but I also want them stored and compressed and not taking up space.
The solution:
---------------------------------------------------------
| tee >(gzip > logfile.tgz)
---------------------------------------------------------
Let's break it down one bit at a time
| tee
Standard output is piped through tee, the unix t-shaped pipe, pretty much comes standard with all *nixes. Tee has two outputs, first it outputs to a file (or file handle), and second it pipes the output to the screen.
>()
This right caret/parenthesis pair comes in very handy. It opens a sub shell, leaving an implicit file handle that can be piped to. Tee sees this as a regular file, and begins piping standard output to it.
gzip > logfile.tgz
Gzip defaults to standard in for the input if no file handle is given. It uses lz77 encoding along with Huffman trees in a 32k sliding window. Decoding data (Huffman trees) are placed at the beginning of each block. This means
--------------------------------------------------------
| tee >(gzip > logfile.gz)
--------------------------------------------------------
Putting it all together, we pipe standard output to tee which pipes to screen as well as the implicit file handle created by the sub shell. The sub shell is gzipping the piped standard input at maximum compression and outputting the resultant gzip file to logfile.gz. Just pipe any huge log file through this bit of code, and you've taken your space requirements from gigs to megs, with virtually no cpu/memory overhead. Handy, and fits easily into most launch scripts.
Snesaver – Zsnes screen saver for linux
Ok, so here's a fun bit of perl code I wrote last night. It's called snesaver, and you can download it here. Here's how it works,
1. You'll need to be running some flavor of *nix (self=ubuntu), with perl installed
2. You'll need to have xscreensaver set up as your screensaver (though it should be simplicity itself to do this for most any other screensaver)
3. You'll need to have zsnes installed
4. You'll have to supply your own roms (legality), and record your own rom state movies in zsnes (simple simple!)
And that's about all you'll need. Just open up your ~/.xscreensaver file and put an entry for snesaver.pl (should be in your path) under the "programs" section. Then edit the script to point to /your/rom/directory/structure/ and you're all set.
Here's a youtube video of it in action.
/H
Easy Execution Library – c++ class library for spawning processes
Easy Execute - An extremely easy to use C++ library for spawning processes and executing arbitrary commands.
This library is intended for developers who wish to simply execute some command, without worrying about the underlying
fork/exec/system calls, string parsing, setting up timers, recording the run time of the command, etc. This library
makes it all completely plug and play. GPLv3.
| Author | Hunter Davis email |
| Operating Systems Supported | Linux, Unix, OSX, Cygwin, any POSIX type system |
| Included with Application | C++ class definition and header, examples |
| Target Audience | Begining developers, any developer who does not wish to involve themselves with mid to low level process code, timing code, etc. |
| ScreenShot | N/A |
| Documentation | Included with archive. Driver program/unit tests also included |
Shell Script to Monitor PID cpu usage over time
Sometimes I feel like all the world's problems can be solved with shell scripts. Here's one that's short and sweet. You give it a PID, interval, and outfile, and it keeps track of the PID given till the PID dies (or you kill the script). These values are plotted against time with gnuplot. I wrote it while keeping an eye on 'top' on a server. Snag it here
Ping Rebooter
Ping Rebooter is a simple but useful application, basically it pings a website at some time interval if the website does not return, it uses wget to execute your cable/dsl modem's reset string. The default reset string is: http://192.168.100.1/gscan.cgi?freq=331000000 This works for my linksys cable modem and has been tested on a few others. I find this useful
- if your modem stops working due to power outage
- if your modem gets disconnected due to torrent use, file sharing, etc.
This is simply a open source linux remake of the windows version I released to a few friends 2 years ago. Since then i've decided to completely get rid of all MS software, so it was time for a ground up re-write. This version has not yet been tested on many systems, but it's fairly simple code so not much to worry about. Download it Here, it should work on posix type systems (linux,unix,macos) and cygwin, it requires wget and ping to be installed.
SSRSS – Super Simple RSS
One of the more popular programs I released around 3 years ago, this one made it to the big-time. Magazines, download sites, and advertising revenue! SSRSS allows you to easily create and edit rss files.
Development of SSRSS has been taken over by a group of enterprising programmers at sourceforge. Go check out their site and the current state of SSRSS here.
- Features: Create and Manage RSS 2.0 files
- Features: Simplify the process of RSS file creation
- Features: Easily manipulate RSS files created
csserver adventure!
Here is a program I wrote freshman year of college (1999). A text-based roleplaying game based on people I had met in the cs department. Fun, dated, text-based.
Title Bar Scroller – Read Books on ANY application window title, and so much more!
This is a program I wrote about 4-5 years ago which received some popularity on various international websites and magazines. While not as popular as ssrss, it still made lockergnome and shell city if I remember correctly. You use it to scroll or appear lines of text from a book or text file (or alarm text items you set up, features!) on the current window titlebar of whatever application you are using at the time. Source code available gplv3.
Building the Perfect Batch File Processor for Converting Videos with VLC for Xbox 360 playback.
vlcwmvhgd.batThis may not be useful anymore after the fall dashboard update, but for those who still have files the 360 can't play, here's a nice vlc batch script I wrote a while back.
I added in shift-looping (so you can drag as many files as you like to this bat file initially and let them run overnight), as well as adding the vlc:quit command to the vlc playlist after every file conversion. This ensures the vlc gui closes after encoding so the batch file can move on to the next video to encode w/o human interaction.