Showing posts with label Unix. Show all posts
Showing posts with label Unix. Show all posts

January 23, 2014

CoLT: Copy hyperlink text/details

CoLT is a very useful Firefox add-on to help you copy the text and the actual URL of a hyperlink in a variety of ways. Let me just put in the add-on’s brief straight from its homepage:

CoLT (short for "Copy Link Text") makes it easy to copy either a link's text, or both the link text and its URL in a format you specify. This non-invasive extension simply appears as context menu items when you right-click a link. Fully customizable!

The add-on can be installed from https://addons.mozilla.org/en-US/firefox/addon/colt/.

January 11, 2014

lsof: A very useful Unix command to list the open files

lsof – shortened form for “list open files” – is the command that can be used to report the list of open files on Unix-based systems. There are a number of options available that can be used to control how the command behaves: list all the open files on a system, find out which processes and users have a file opened, see what files does a user or program have open etc.

Just to draw parallel, this Unix command is equivalent to Windows’ Process Explorer that was previously shared as a Techbit.

http://www.catonmat.net/blog/unix-utilities-lsof/ has given lot more details on the various usages of the command in a very user friendly way.

January 3, 2014

Unix command: screen

Consider this scenario: you’re connected to a remote Unix-based server (like Solaris or Linux); you invoke a program; the process starts running; while the process is still running and has not yet completed its execution, you lose the network; the process dies! The program you invoked could be anything like an installer which is just halfway through. How do you make sure you do not end up in b-e-a-utiful situations like this?

One way is to use nohup command that detaches the process from the terminal and makes the process ignore the HUP/SIGHUP signal (which is sent when the terminal session closes and the process usually exits as part of signal handling). This explains the origin of the command name too: no HUP ~ no hup ~ nohup. Other ways to detach include setting shell specific process control options like disown. However, this mechanism only detaches the process i.e. once detached, it cannot be attached to a terminal. This does not also handle processes that are interactive i.e. which expect input from the user where the ability to pause/resume or plug back into the session is required.

screen is a wonderful command that addresses all the aforementioned shortfalls and provides many more new features in the context of remote session management capabilities. It can be used to attach & detach sessions, share a session with another user, split the window horizontally/vertically and so on. You can create as many sessions as you want and manage as per your wish.

What the command essentially does underneath is that the process is detached from the connected terminal and rather managed by the screen process. Note that screen may not be available by default on your Unix system and may need to be installed explicitly.

Links

Good tutorial: http://www.rackaid.com/resources/linux-screen-tutorial-and-how-to/
Short introduction: http://www.pixelbeat.org/lkdb/screen.html
Official Website: http://www.gnu.org/software/screen/

December 13, 2013

EPIC: Eclipse-based Perl IDE

Are you a programmer? And, is Perl one of the languages you use? If so, then this Techbit is for you.

EPIC is the point to start from for your Perl programming. It is imperative that one uses IDE for programming and it is precisely where EPIC comes as aid in the case of Perl-based development activities. As an IDE, EPIC provides all the standard functionality like syntax highlighting, syntax checking, content assist, debugging aids etc.

EPIC is located at http://www.epic-ide.org/.


For a quick head start, you can go through http://web.archive.org/web/20120113222246/http://www.cs.wcupa.edu/~rkline/Ubuntu/eclipse-perl.html (archived version of the no longer accessible http://www.cs.wcupa.edu/~rkline/Ubuntu/eclipse-perl.html). The detailed documentation can be found at the official website listed above.

December 4, 2013

Command Line Snippet Store

Straight from the words as described on the site itself, http://www.commandlinefu.com is the place to record those command-line gems that you return to again and again. Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

August 6, 2013

Unix: more or less?

Commonly seen in Unix-based operating systems is usage of the command more to view paginated outputs. However, this command is quite old.

There exists the less command which has lot more functionality (like both forward and backward navigation) and is better performing as well (by not reading the entire file contents at the time of command invocation itself).

So, next time you want to use more, be frugal and use less instead – it is not useless though :)

July 23, 2013

Unix command: script

What is "script" command?

script is a very useful command that can be used to capture all that appears on a terminal. The output thus captured from the interactive session may then be useful while sharing with another colleague or team, or even used in a variety of debugging scenarios.

Next time you find yourself in a position to grab the screen/terminal contents, make use of this wonderful utility instead of relying on ad-hoc output redirections and screenshot captures.

What is its syntax and how do I use it?

Refer to the man pages on your system for the exact syntax/details, or an online resource like the one at http://unixhelp.ed.ac.uk/CGI/man-cgi?script. Typical usage however is along the lines of:
script [-a] [-c COMMAND] [-f] [-q] [-t] [file]

Are there any interesting points to be noted?

One standard note is that interactive commands, such as vi, create garbage in the output file. Be aware of it as such and use "script" for capturing plain text.

Few oft missed aspects, though, are that it:
  • Invokes a new shell based on what is populated in the environment variable SHELL
  • Creates a new login session as can be seen from the output of last command

Why don't you show me a simple invocation and what does it result in?

Sure, I can :)

Here's a sample invocation of the command:
sh-3.2# script
Script started, file is typescript
sh-3.2# ls
bin  boot  dev  etc  home  lib  lib64  lost+found  media  misc  mnt  net  opt
proc  root  sbin  selinux  srv  sys  tftpboot  tmp  typescript  usr  var
sh-3.2# exit
exit
Script done, file is typescript
And, this is what is logged into the file:
sh-3.2# cat typescript
Script started on Tue 23 Jul 2013 04:10:04 PM IST
sh-3.2# ls
bin  boot  dev  etc  home  lib  lib64  lost+found  media  misc  mnt  net  opt
proc  root  sbin  selinux  srv  sys  tftpboot  tmp  typescript  usr  var
sh-3.2# exit
exit

Script done on Tue 23 Jul 2013 04:10:20 PM IST
sh-3.2# 

July 9, 2013

MobaXterm

I wanted to include a one-liner in the title of this Techbit than just the tool name i.e. MobaXterm - it was very difficult though. There are so many wide-ranged features integrated into one single app.

Listed below are some of the features/findings:
  1. Tabbed Console with UNIX-like interface – replacement for Console & accommodates part of Cygwin
  2. Tabbed Console with support for embedded terminal server sessions – replacement for PuTTY Connection Manager
  3. X-server client – replacement for X-based commercial tools like Exceed or Reflection-X
  4. Multi-Execution Mode – you want the same command to be invoked in parallel on several servers? Then, this is what you want.
  5. Ability to download files (SFTP/FTP) from the terminal session – In general, if you want to fetch a file while working on a terminal server session, then you open a separate file transfer app (like FileZilla) and then download it. This can all be done from the same session in this tool.
MobaXterm is free and available for download at http://mobaxterm.mobatek.net/.

April 27, 2013

SSH or Telnet using Java

Ever wanted to find out how one can programmatically connect to a server or a network element like router, switch, firewall etc. using SSH or Telnet – specifically if the programming language is Java? If so, then JSch and Apache Commons enable you to realize it.

SSH in Java using "JSch"

JSch can be downloaded from http://www.jcraft.com/jsch/.
Examples can be found at http://www.jcraft.com/jsch/examples/.

Telnet in Java using "Apache Commons"

Apache Commons can be downloaded from http://commons.apache.org/.
Example can be found at http://commons.apache.org/proper/commons-net/examples/telnet/TelnetClientExample.java.

April 22, 2013

JD-Eclipse: Java Decompiler

JD-Eclipse is a plug-in for the Eclipse platform. It allows you to display all the Java sources during your debugging process, even if you do not have them all.

JD-Eclipse is free for non-commercial use. This means that JD-Eclipse shall not be included or embedded into commercial software products. Nevertheless, this project may be freely used for personal needs in commercial or non-commercial environments.

JD-Eclipse >> http://java.decompiler.free.fr/?q=jdeclipse

February 24, 2013

How to find the count of physical processors, cores and threads on Solaris?

Let’s first understand the terms and relationship between the same quickly.

Physical processor – processor that is actually / physically present
Core – further division of physical processor i.e. a physical processor can contain multiple cores
Thread – a distinct execution path provided by a core i.e. a core can support multiple threads
Virtual processor – can be considered equivalent to a thread

That means, Number of virtual processors in a physical processor = count (cores) * count(threads).

psrinfo -p gives the number of physical processors

psrinfo -pv gives the number of virtual processors per each physical processor

But, how does one find the number of cores / threads on a physical processor? The only way that this could be found out is by going to the specs page on Sun/Oracle website. However, recent updates of Solaris 10 did include a fix to report this kind of detail. In case the fix is not there, it can still be found out from the command “kstat cpu_info”. For more details, you may refer to http://blogs.oracle.com/sistare/entry/cpu_to_core_mapping.

February 16, 2013

Unix: Why do “df” and “du” report different disk usages?

Let's understand the fundamental difference between the two commands first.

  • "df" reports usage based on details available at filesystem-level (inodes)
  • "du" reports usage based on the actual contents (files) of a directory

There are many reasons for the discrepancy between the numbers reported by the two commands. However, it is _mostly_ because of removal of a large file, the file handle of which is still present with a process (the file was opened by the process). When a file is removed, "du" no longer takes it into consideration although "df" counts it as the file inode is still present because of the open file handle - which is why "du" reports free usage where as "df" does not.

How do you fix the discrepancy in such cases? Simplest is by not removing the files that are held open by some process. However, if you do run into a situation of accidentally removing such files, restart the application/process that held the open file descriptor - this is the proper way to clean up. There is also a workaround of clearing the file descriptor manually/directly although it is not recommended procedure.

Another reason for the mismatch is that "df" counts the complete blocks (8 KB blocks, for example, as seen in the above techbit) where as "du" counts the actual file sizes (like 1 KB if the file is actually only 1 KB). If there is a large number of such small files, then the difference can show up significantly.

There are many other possible reasons as well for the mismatch. Provided below are few links for further reading:

http://linuxshellaccount.blogspot.com/2008/12/why-du-and-df-display-different-values.html
http://blog.thilelli.net/post/2008/10/18/Discrepancies-Between-df-And-du-Outputs
http://sysunconfig.net/aixtips/df_du_diff_out.txt

December 8, 2012

How to find filesystem block size on Solaris?

Firstly, what is a filesystem block?

It is the smallest piece/size of the disk allocated or accessible on the filesystem.

Can you explain a bit more?

Having a larger block size helps in achieving better performance due to lesser number of disk reads and lower metadata overhead. However, it is a disadvantage for files of smaller size. For example, assuming block size of 8 KB, 8 KB is allocated even if the file size is only 1 KB.

Tell me how do I find the filesystem block size on Solaris?

There’re couple of ways:

  1. From “df -g”; example: df -g | grep "block size" | awk '{print $1 " " $4}'
  2. From “fstype –v”; example: fstyp -v /dev/dsk/c0t1d0s0 | grep ^bsize

October 22, 2012

Techbits #25: Cygwin – Unix on Windows!

Have you ever wished if you had the ability to use UNIX shells (like “bash”), commands or scripts on Windows to make yourself more productive from the resulting power and flexibility? And, should it be something free and not commercial like “MKS Toolkit”? If so, then this Techbit is for you.

Cygwin is what you are looking for and accessible from http://www.cygwin.com/. There are a huge number of packages available that you can install additionally to spruce up and spice up your Cygwin setup.

Go on and wake up your Windoze with Cygwin! :)

October 11, 2012

Techbits #24: “nslookup” works but “ping”, “telnet” etc. don’t?

This Techbit is for those souls that work on Solaris platform. It may be applicable on other Unix-based platforms as well but I've not verified it.

A friend of mine told me he has configured the name servers correctly in /etc/resolv.conf after which he could get "nslookup" working but not the other commands like "ping", "telnet" etc. Why?

The answer lies in that "nslookup" does DNS lookups using /etc/resolv.conf where as "ping", "telnet" etc. refer to /etc/nsswitch.conf to find out the mechanisms to use for name resolutions.  If DNS is not listed here, then /etc/resolv.conf is not going to be referred to. Therefore, make sure DNS is listed as a source for host resolution in /etc/nsswtich.conf.

Another command to do such verification is 'getent hosts' - this is like nslookup but pays attention to /etc/nsswitch.conf additionally.

Thus, to summarize which configuration file does each of the commands depend on:

nslookup - /etc/resolv.conf

ping - /etc/nsswtich.conf

telnet - /etc/nsswtich.conf

getent hosts - /etc/resolv.conf, /etc/nsswtich.conf

July 21, 2012

Techbits #18: Editing ClearCase files from Windows leads to ^M characters?

A common problem reported by developers accessing ClearCase VOBs/views from Windows is that the edited text files (like source code, scripts, configuration files etc.) contain Ctrl+M (^M) characters when they are checked-in eventually (or) when transferred to Unix-based servers like Solaris or Linux. This could lead to further build problems.

Most people try to get around this issue by replacing the additional Ctrl+M characters in the files on the Unix machines by using commands like "dos2unix" or "sed 's/^M$//g' ...". However, the issue has to be fixed at the source (editing in the first place) and not at the destination.

How to then fix the problem? By setting the file type properly in your text editor and ensuring that the file type is retained correctly in case of existing file edits. If your text editor (like Notepad or Wordpad) does not support setting these properties, you should consider switching to something else.

If you're using Notepad++, then this can be specified by following Settings >> Preferences >> New Document/Default Directory and setting the parameter "Format" to "Unix". Note that Notepad++ needs to be restarted to effectuate the change. Once restarted, you can verify by seeing UNIX in the status bar. In a similar manner, if you’re editing any existing UNIX-based text files, the status bar will show UNIX.

Similar options can be found in standard IDEs like Eclipse and others.

April 7, 2012

Techbits #13: Unix "vi" Editor - ZZ

Given the depth of the previous Techbit, let's make this a short one.

If you're working on Unix-based systems, you must have used the "vi" command at one or other point of time.

While using "vi" in escape mode, you can use ZZ (shift+zz) to save the file. I found this much more convenient than its equivalent ":wq" or ":wq!". However, the percentage of people whom I've seen using this is very less.

Now, what could be the reason as to why this did not become very popular? Some say that it could be because Ctrl and Shift keys are adjacent to each other due to which one might accidentally press Ctrl instead of Shift - this then pushes the current process (vi) to background due to Ctrl+z (the first z itself). All of a sudden, the user will see himself/herself at the command line prompt. This might confuse the users and hence it might not have been published or popularized so widely.

March 25, 2012

Techbits #11: Avoiding "Useless use of" commands in Unix

Let's check out another interesting Unix-related Techbit related to avoiding the usage of commands when they're not necessary.

Such usages are also fondly referred in "Useless use of ..." category awards :-) There're many varieties under this class like "Useless use of cat", "Useless use of grep & awk", "Useless use of wc -l" etc. Provided below are few.

Useless use of "cat"

cat file.txt | grep pattern can very well be expressed as grep pattern file.txt

Useless use of "grep"

ps -ef | grep java | grep -v grep can be ps -ef | grep '[j]ava'

ps -ef | grep java | awk '{print $2}' can just become ps -ef | awk '/java/ {print $2}'

Useless use of "wc"

ls -1 | grep pattern | wc -l can be written as ls -1 | grep -c pattern

March 17, 2012

Techbits #10: PuTTY Connection Manager

Ever wanted a "tabbed" version of the favourite Telnet/SSH terminal emulator PuTTY? Then this is for you.

"PuTTY Connection Manager" is built on top of PuTTY and has additional features like tabs, credential storage (encrypted) for auto-login etc. which help increase your productivity by a great deal, especially if/when you do not need the X GUI and CLI is more than sufficient to perform your tasks.

What's more?  It's completely free - not just for personal use but for commercial usage as well. And, it's a very good free alternative to a licensed product like SecureCRT.

You can download this utility from http://puttycm.free.fr/.

March 11, 2012

Techbits #9: Use "exec"

If you are working on Unix-based systems like Solaris and Linux, then this Techbit is for you.

When you are no longer interested in the current process and want to work on some other process, use exec to replace the existing process with new the one.

Where is it helpful? I often see people logging on to servers where they find that the shell is Bourne and type bash immediately. Nothing is done with the previous shell process plus there's an overhead to exit twice while disconnecting from the server. In such cases, use exec bash - not only are you going to be more graceful and respectful to using system resources but also increasing productivity by saving yourself from typing exit twice each and every time!

An addendum to the provided example is that if you find yourself repeatedly changing the shell every time you log in to a server, then you should change the default login shell once for all. Of course, this should be done only if the specific login is not being used by any process or application - changing the login shell in such cases would result in the application experiencing unexpected behaviour because it could possibly miss the shell and its start-up parameters thereby. This is a corner case scenario but it needs to be taken into consideration nonetheless.