Posts Tagged ‘Geek’
If you’ve tried to download Oracle 10g express database (or any oracle rpm) from their website using wget, you’ve probably hit a bit of a brick wall that looks like an oracle login page. Then, when you try to wget using the link to the rpm you’ve also been unsuccessful. So, Here’s your solution:
$ wget http://download.oracle.com/otn/linux/oracle10g/xe/10201/oracle-xe-univ-10.2.0.1-1.0.i386.rpm
https://profile.oracle.com/jsp/realms/otnLogin.jsp?remoteIp=xxx.xxx.xxx.xxx&globalId=&redirectUrl=http%3a%2f%2fdownload-llnw.oracle.com%3a80%2fotn%2flinux%2foracle10g%2fxe%2f10201%2foracle-xe-univ-10.2.0.1-1.0.i386.rpm
Take note of this and craft the following wget:
wget -c --no-check-certificate "https://profile.oracle.com/jsp/reg/loginHandler.jsp?remoteIp=174.xxx.xxx.xxx&globalId=&redirectUrl=http:%2F%2Fdownload-llnw.oracle.com:80%2Fotn%2Flinux%2Foracle10g%2Fxe%2F10201%2Foracle-xe-univ-10.2.0.1-1.0.i386.rpm&username=USERNAME&password=PASSWORD"
You will still need an username/pass for this, so create yourself a free account on their website. Best of luck!
Having a ball playing around with the just-released GoogleCL tool, which offers command line access to Google Calendar, contacts, Docs, Picasa, Blogger, and YouTube. With Python-based GoogleCL installed, you can do things such as list today’s events on your GCal right in the terminal, like so:
$ google calendar today title
Coffee with Michael and Samir
Dozing off
Lunch at Flingers
Instant use case: Add echo “Next 24 hours:”;google calendar today title to your ~/.bash_profile file to see what you’ve got scheduled for the day when you launch a new Terminal window. Some more GoogleCL fun inside.
If you just type google at the command line, you launch an interactive terminal that lets you try all the various commands. In the interactive terminal, type command-name help to see its options, like help calendar.
Each command has several parameters that aren’t immediately apparent. For example, in calendar, you can omit the long and hairy event URL by using the title parameter. You can list events for a particular day using the data parameter (–date 2010-06-16), and you can get events from a particular calendar and by keyword search term.
For example, to see all my trips to Melbourne on my Work calendar, I’d use the command:
$ google calendar list --cal Work --query Melbourne
Remember the beauty of the command line: you can easily chain commands together with the pipe, so you can sed, awk, and grep output to your heart’s content, and then write it to a file if needed, using >. Before I discovered the title parameter on the calendar command, I was planning to use sed to filter out the calendar URLs from the output. (Thanks to lightening-fast sed and awk experts on Twitter, I was prepared to do just that.)
While I’ve mostly only played with calendar, the Docs access is pretty useful, too. With it, you could easily schedule cron’ed backups of your Google Docs, or push data into a new doc on a regular basis. Same deal with Picasa and YouTube. I like the idea of cron’ing a job that backs up my Google contacts to a CSV file on my local computer weekly, too. I don’t see myself ever blogging from the command line, but it’s neat that you can.
When installing I had an issue when running the google command, There was some python errors followed by:
ImportError: No module named gdata.service
I resolved this by downloading the gdata-python-client package, untar’ing and installing.
I find that whenever I get a large data file from somewhere (i.e. extract some data from a database, crawl some sites and dump the data in a file) I always need to do just that little bit of extra processing before I can actually use it. This processing is always just non-trivial enough and I do it just uncommonly enough for me to always forget exactly how to go about it. Of course, this is to be expected, if you learn something and want it to stick you have to keep doing it. It’s all part and parcel of how our brain works when it comes tolearning new skills, but that doesn’t make it any less annoying.
Back to our data file, for me I find that I almost always need to do 3 things (amongst others) before doing anything else with my file.
- delete the first line (especially when pulling data out of the database)
- delete the last line
- remove all blank lines
Don’t ask me why but for whatever reason, you always get an extraneous first line and unexpected blank lines (and less often an extraneous last line) no matter how you produce the file 
.
Anyways, my tool of choice in the matter is bash - it is just too trivial to use anything else (plus I love the simplicity and power of the shell). So, to make sure I never forget again here is the easiest way of doing all the three things above using sed:
sed -e 1d -e ‘$d’ -e ‘/^$/d’ input_file > output_file
Of course since we’re using bash, there should be numerous ways of doing the above.
You can remove the first line using awk:
awk 'FNR>1'
but I don’t know how to remove the last line using awk. Anyone?
You can use head or tail to get rid of the first and last line:
head --lines=-1 input_file | tail --lines=+2
but not to remove blank lines.
You can use grep to remove blank lines
grep -v "^$" input_file
but it would be silly to try and use it to remove the first and last line (possible though).
If you know of an easier way to do the above three things in a one-liner using bash – do share it.
What are some of the most common (but non-trivial enough) things that you find yourself doing with bash when it comes to pre-processing that large data file?
Code names have been around for a long time. Remember the Manhattan project in the 1940s? That turned out to be the atomic bomb. Thankfully, not all code names hide such sinister projects.
Code names can be about secrecy, but when it comes to software development, it’s usually not so much about secrecy as it is about the convenience of having a name for a specific version of a software. It can be very practical to have a unique identifier for a project to get everyone on the same page and avoid confusion.
And we want to name our darlings, don’t we?
Read More
Sometimes there is a very quick need to determine what user(s) are causing high load to a particular page. Instead of tailing high-speed logs and giving yourself a headache, Throw in a one-lined piped cat command to give you the info you’re after without the foreplay.
cat /path/to/access.log | awk '{print $1}' | sort | uniq -c | sort -n | tail